Packfile Header: Reading the Bytes

The previous lesson explained why git packs objects. This one reads a real pack at the byte level: the 12-byte header that announces what is inside, the 20-byte footer that proves the file is intact, and the variable-length header that starts the first object. After it your program takes the hex of a whole .pack file and reports what git would see before it inflates a single object.

The layout of a pack

offset 0    4 bytes   "PACK"                     magic
offset 4    4 bytes   version, big-endian        2 (3 is accepted, git writes 2)
offset 8    4 bytes   object count, big-endian
offset 12   ...       the objects, one after another
last 20     20 bytes  SHA-1 of every byte before it

All multi-byte integers are big-endian, the byte order of network protocols, because packs are what fetch and push send over the wire. The smallest legal pack holds zero objects: 12 header bytes plus a 20-byte footer, 32 bytes in all.

The footer is the integrity check

Git verifies the footer before it trusts anything in the file. Hash all bytes except the last 20 and compare. A truncated download, a flipped bit or a version-skewed writer all fail here, which is why a pack is never half-accepted.

Each object starts with a size-and-type varint

An object is not prefixed by <type> <size> NUL like a loose object. Its header is packed into as few bytes as possible:

first byte:      C TTT SSSS        C = continuation bit, TTT = type, SSSS = low 4 bits of the size
later bytes:     C SSSSSSS         7 more size bits each, least significant group first

While the continuation bit (the top bit) of a byte is set, another byte follows. The size is assembled little-endian: the first byte supplies bits 0 to 3, the next supplies bits 4 to 10, the next bits 11 to 17, and so on.

A worked example on a tree of 1000 bytes. The type is 2, so the type bits are 010. The size 1000 is 0x3e8: the low 4 bits are 8, and the rest, 62, fits in one more group of seven bits.

byte 1:  1 010 1000   = 0xa8    continuation set, type tree, size bits 0-3 = 8
byte 2:  0 0111110    = 0x3e    last byte, size bits 4-10 = 62
size = 8 + 62 * 16 = 1000

Right after the header comes the zlib data (or, for deltas, the base reference and then zlib data). This lesson stops there: the exercise needs only the first object's type and size.

Common mistakes

• Reading the version or the count little-endian. 00 00 00 02 is version 2.
• Computing the size with a 32-bit shift. A packed object can be larger than 4 GB, and a size built from 7-bit groups overflows quickly in a 32-bit integer.
• Hashing the whole file including the footer, or only the first 12 bytes.
• Reading the type from the wrong bits. The type is bits 4 to 6 of the first byte, above the 4 size bits and below the continuation bit.
• Running past the footer. If a continuation bit is set on the last body byte, the header is cut off, and the next 20 bytes are the checksum, not data.

Your exercise

Each input line is the hex of a complete pack file. Print one line per file in the format given in the exercise, or the first error in the order listed there. The test packs include ones written by real git pack-objects, so a parser built on the examples in this lesson alone will miss the size varint and the checksum.