Compressed On-Disk Storage

You can hash objects and parse objects; now they need to live somewhere. Git's loose-object store is a masterpiece of doing the simplest thing that works: one file per object, zlib-compressed, filed under its own hash. No database, no index, no daemon — a directory layout so dumb it has survived unchanged since 2005.

The layout

An object named 95d09f2b10159347eece71399a7e2e907ea3df4f lives at:

.git/objects/95/d09f2b10159347eece71399a7e2e907ea3df4f
             ─┬ ────────────────┬───────────────────
     first 2 chars          remaining 38

The two-level split (256 possible subdirectories, 00 through ff) exists for a mundane, real reason: filesystems degrade when a single directory holds hundreds of thousands of entries. The fan-out keeps each subdirectory small. Given any hash, the path is pure string manipulation — no lookup required; that's content addressing paying rent again.

zlib, whole-file

The file's contents are the full object — <type> <size>\0<payload> — compressed with zlib (the deflate algorithm; Python's zlib module, Go's compress/zlib, every language has it). Note the order of operations, because it's a real design decision:

• Hash the uncompressed bytes (identity must not depend on compression settings — the same content compressed at different levels must keep the same name).
• Store the compressed bytes (source code and text compress 3–10×; a repository stores its entire history in less space than you'd guess).

The write path and the read path

Write (hash_object with persistence):

1. Build <type> <size>\0<payload>.
2. SHA-1 → 40-hex name.
3. If the object file already exists — done; content addressing means it's necessarily identical. (Dedup isn't a feature you build; it's a check you skip.)
4. Else zlib.compress and write to objects/xx/yyyy…. Real Git writes to a temp file and renames — an atomic-write habit worth copying so a crash mid-write can't leave a corrupt object under a valid name.

Read (read_object):

1. Split the hash into path components; read the file.
2. zlib.decompress.
3. Parse with last lesson's function.
4. Verify: re-hash the decompressed bytes and compare with the name you looked up. Mismatch = the disk lied = refuse loudly. This one check is Git's entire corruption-detection story for loose objects, and it costs one hash.

Round-tripping — write an object, read it back, get identical bytes — is the exercise, and it's also the moment your object store becomes real: after this lesson, hash-object and cat-file (next lesson) are just thin CLIs over these two functions.

Loose vs packed, a preview

One-file-per-object is beautiful and wasteful: ten thousand commits of a file that changed slightly each time store ten thousand full copies (compressed, but whole). Real Git periodically repacks loose objects into packfiles — delta-compressed archives covered in this chapter's later lessons. The mental model to carry: loose objects are the write-optimized format; packfiles are the read-optimized one. Your Git speaks loose first, because everything else is an optimization of it.

Your exercise: Round-trip an Object Through Storage

Implement write and read exactly as above — path fan-out, zlib both directions, header parse, hash verification — and prove read(write(content)) == content. Test the edges the grader will: writing the same content twice (second write is a no-op, same hash out), and reading back binary-ish payloads intact. When the round-trip holds, you own a durable, self-verifying, deduplicating object database in ~40 lines — which is, quietly, most of what .git is.