Replication

Each partition is replicated across N brokers. One is leader; others are followers.

Partition 0:
  Leader:    Broker 1
  Followers: Broker 2, Broker 3

All reads/writes go through the leader.

Followers periodically fetch from leader. Catch up to leader's high water mark.

ISR (In-Sync Replicas):

• Replicas caught up within replica.lag.time.max.ms (default 30s).
• Only ISRs are eligible to become leader.
• Producer with acks=all waits for all ISRs.

If a follower lags too long: kicked out of ISR. Continues fetching but is "out of sync." When caught up: re-added.

Replication factor:

• 1: no replication. Single point of failure. Test only.
• 2: one failure tolerated.
• 3: two failures tolerated. Standard.
• Higher for critical data.

Network bandwidth:

• Replication multiplies write traffic by RF.
• Inter-broker bandwidth must be sufficient.

Min in-sync replicas:

• min.insync.replicas=2 (with RF=3): producer rejected if fewer than 2 ISRs.
• Tradeoff: availability vs durability.

Leader election:

• On leader failure: controller picks new leader from ISRs.
• Preferred leader: original leader. Auto-balance brings it back when ISR.

Unclean leader election:

• If no ISR available: pick non-ISR replica (data loss).
• Disabled by default (unclean.leader.election.enable=false).

KRaft (no ZooKeeper):

• Controller quorum (3-5 brokers) runs Raft.
• Stores cluster metadata.
• Replaces ZK in Kafka 3.3+.
• Faster failover, simpler ops.

Cross-DC replication:

• MirrorMaker 2: replicate topics between clusters.
• Used for DR or geo-distribution.