Gorilla Compression

Gorilla (Facebook, 2015 paper) is the standard for TSDB compression. Key ideas:

Timestamp compression (delta-of-delta):

1. First timestamp: store full 64-bit value.
2. Second: store delta (D1 = T2 - T1).
3. Subsequent: store delta-of-delta (DoD = (T_i - T_{i-1}) - D_{i-1}).
4. Encoding:
   • DoD = 0: write 0 (1 bit).
   • DoD in [-63, 64]: write 10 + 7 bits.
   • DoD in [-255, 256]: write 110 + 9 bits.
   • DoD in [-2047, 2048]: write 1110 + 12 bits.
   • DoD larger: write 1111 + 32 bits.

For regularly-sampled data (every 10s), most DoDs are 0 → 1 bit per timestamp.

Value compression (XOR):

1. First float: store full 64-bit IEEE 754.
2. Subsequent: XOR with previous; encode the XORed bits.
3. Encoding:
   • XOR = 0: write 0.
   • Else: write 1 + leading_zeros (5 bits) + meaningful_bits (6 bits) + meaningful bits.
   • If meaningful range matches previous: skip the size header.

For slowly-changing values (most metrics), XORs have many leading + trailing zeros → few bits.

Combined: ~10-15 bits per (timestamp, value) pair on average. Vs raw 16 bytes (128 bits) — 10x compression.

Decoding is symmetric.

Blocks of 4k points are the typical size. Stream-friendly: produces a bit array; finalize closes the bit writer.