Time-Range Index

Logs are time-ordered — most queries are "show me errors in the last hour", not "all errors ever". A time index narrows the chunk set.

Per stream, store chunks sorted by start_time. Binary search for range:

Most systems also keep a time-bucketed index: bucket every chunk into 1-hour buckets:

2026-05-05T12:00 .. 2026-05-05T13:00
  -> [chunk_a, chunk_b, chunk_c (stream s1), chunk_d (stream s7), ...]

Lookup for [12:30, 13:30] -> read 2 buckets (12:00 and 13:00 ones).

Storage: index lives in a small fast DB (Bolt, Cassandra, DynamoDB, Postgres). Tens of MB for billions of entries.

Pruning: dropping old chunks = drop bucket records older than retention. Cheap.

Compaction: small chunks merge into bigger ones over time → fewer index entries to scan.