std.zlib

Compression over bytes.

import std.zlib

packed = zlib.gzip(data)
data   = zlib.gunzip(packed)

raw    = zlib.deflate(data)
data   = zlib.inflate(raw)

Which format

gzip / gunzip for anything crossing a boundary - a file on disk, an HTTP body, something another tool will read. Gzip has a header and a checksum, and everything understands it.

deflate / inflate for raw compressed data with no header, when a format or protocol specifies it. Smaller by a few bytes, and not self-describing.

If you are unsure, use gzip.

In and out are bytes

Both directions work on bytes. Compress a string by encoding it first:

packed = zlib.gzip(bytes(text))
text   = string(zlib.gunzip(packed))

string() on the way back raises on invalid UTF-8, which is the behaviour you want

• silently substituting replacement characters would corrupt the round trip. See Bytes.

What compresses

Text, JSON, logs and source compress very well - often 5 to 10 times. Already compressed data - JPEG, PNG, video, a zip - does not, and gzipping it wastes CPU to produce something marginally larger.

Errors

Corrupt or truncated input raises rather than returning partial output. Catch it when the data came from outside:

try { data = zlib.gunzip(body) } catch (e) { reject("bad encoding") }

Decompression bombs

A small compressed input can expand enormously: deflate amplifies about 1000:1, so a 1.4 MB input can stand for 1.5 GB. gunzip and inflate stop at ECKO_ARCHIVE_MAX_UNPACKED (default 1 GiB, the same bound std.archive extraction uses) and raise an error that names it, so a bomb is a catchable failure rather than the OOM killer. Set the variable lower where the input is untrusted and the legitimate sizes are known. The package fetcher applies the same rule with its own bound, ECKO_PKG_MAX_UNPACKED.