Producers

A producer sends messages to topics. Configurable for throughput vs latency vs durability.

Send modes:

• Fire-and-forget: send, don't wait. Fastest, may lose messages.
• Synchronous: send, wait for ack. Slow, definite confirmation.
• Asynchronous (callback): send, register callback for ack. Best balance.

Batching:

• Producer accumulates messages per partition.
• Sends batch when: size threshold OR time threshold (linger.ms).
• Improves throughput dramatically.

Acks (durability levels):

• acks=0: don't wait for any ack. Fastest, may lose messages.
• acks=1: wait for leader. Standard.
• acks=all: wait for all in-sync replicas (ISRs). Strongest.

Idempotent producer:

• Auto-retries duplicate detection via per-producer sequence numbers.
• Configured: enable.idempotence=true.
• Default in modern clients.

Transactions:

• Multiple sends + offset commit in one atomic transaction.
• Used for exactly-once processing in stream apps.

Compression:

• gzip: high ratio, slow.
• snappy: fast, low ratio.
• lz4: balanced.
• zstd: modern best.
• Compresses batches; CPU on producer + decompress on consumer.

Common patterns:

• Per-key ordering: send with consistent key.
• Spread load: random key (or no key + round-robin).
• Attempt-tracking: include retry count in headers.