“We already compress our logs with gzip.” It is one of the most common responses to the idea of lossless telemetry compression, and it is worth taking seriously. gzip is everywhere, it is free, and it does shrink log files. So why is generic compression not enough to change the economics of observability? This post compares lossless vs. gzip for logs, explains why a few-times reduction does not move the needle, and describes what a purpose-built, property-aware approach does differently.
What gzip does, and where it stops
gzip (DEFLATE) is a general-purpose compressor. It finds repeated byte sequences within a sliding window and encodes them more efficiently. On text-heavy logs it typically achieves a few times reduction. That is genuinely useful for cold archives, but it has two limits that matter for observability:
- The ratio is modest. A few-times reduction lowers storage somewhat, but it does not turn observability from a spiraling cost into a predictable one. The order-of-magnitude change is where the economics flip.
- It is opaque to queries. A gzip file is a blob. To search it you decompress it first, which means you either re-inflate huge volumes at query time or re-ingest into an index and pay for that separately. Generic compression saves storage but not search.
Why telemetry deserves a purpose-built compressor
Logs, metrics, and traces are not random text. They are highly structured and highly repetitive in ways a general compressor cannot fully exploit. The same message templates appear millions of times with only a few fields changing. Timestamps march forward predictably. Numeric fields cluster and trend. Fields of the same type share statistical properties. A compressor that understands these properties can do far better than one treating the stream as anonymous bytes.
That is the idea behind the patent-pending, property-aware compression at Sasquatch Labs. Rather than a single generic pass, it applies multiple layers tuned to the structure of telemetry: separating stable message templates from their variable fields, encoding columns of like-typed values together, and modeling the patterns within each field. The result reaches roughly an order of magnitude or more on logs, traces, and metrics, and higher still on security data, well beyond what gzip achieves on the same input.
The part that matters most: staying queryable
A high compression ratio is only half the story. If searching your data means inflating it first, you have traded a storage bill for a compute bill. The important property is that the compressed data remains directly queryable. The Snowman query engine reads the compressed chunks in place and speaks LogQL, SPL, TraceQL, and PromQL natively, so there is no decompress-and-re-ingest step and no separate search-compute line item. You get both the storage savings and full-fidelity search from the same compressed data.
gzip makes your logs smaller. Property-aware, queryable compression makes your observability cheaper without making your data harder to use.
gzip vs. lossless telemetry compression, at a glance
- Ratio: gzip a few times smaller; property-aware compression an order of magnitude or more.
- Queryable: gzip must be decompressed to search; property-aware compression is queried in place.
- Data loss: both are lossless. Neither drops data, unlike sampling. See sampling vs. lossless.
- Economics: gzip trims storage; a high, queryable ratio changes the whole cost model. See the math.
If your only compression is gzip, you are leaving most of the savings, and all of the query benefit, on the table. Learn how purpose-built lossless compression works at sasquatchlabs.io.