What Protobuf Solves

Protocol Buffers (protobuf, by Google) is a compact, schema-driven binary format for structured data.

JSON's problems:

• Verbose: field names repeated in every message.
• Slow: parsing strings.
• Type ambiguity: 123 could be int or float.
• No schema enforcement: receiver doesn't know if data is correct.

Protobuf:

• Schema defined in .proto files.
• Compiled to native code (Go, Java, Python, C++, Rust, etc.).
• Compact wire format (varint, length-prefixed strings, no field names).
• Schema evolution rules (add field = backward-compatible).

Schema:

syntax = "proto3";

message Person {
    string name = 1;
    int32  age = 2;
    repeated string emails = 3;
}

= 1, = 2: tag numbers. Used in wire format. NEVER reuse or change after deploy.

Compiled with protoc to a per-language stub:

Real systems using protobuf:

• gRPC: RPC framework on top of protobuf.
• Google internal: everything.
• Kubernetes: API serialization.
• etcd: state.
• TensorFlow models.

Comparable formats:

• Apache Avro: schema-driven, used in Hadoop/Kafka. Schema embedded with data.
• Thrift (Facebook): similar, less popular.
• MessagePack: schemaless binary JSON.
• Cap'n Proto (Sandstorm): zero-copy, even faster.
• FlatBuffers (Google): zero-copy alternative to Protobuf.

Trade-offs:

• vs JSON: 3-10x smaller, 5-100x faster, but requires schema sharing.
• vs FlatBuffers: smaller schemas + better tooling, but parse overhead.