TCP State Tracking

For TCP, the firewall tracks the FSM:

                NEW
                 |
                 | SYN
                 v
              SYN_SENT
                 |
                 | SYN+ACK
                 v
            ESTABLISHED
                 |
                 | FIN
                 v
            FIN_WAIT_1
                 |
                 | ACK
                 v
            CLOSE_WAIT
                 |
                 | FIN
                 v
              LAST_ACK
                 |
                 | ACK
                 v
              CLOSED

Conntrack mirrors this. NEW packets must have valid SYN; mid-sequence SYNs may be REJECT'd.

Out-of-state packets:

• ACK without prior SYN: drop.
• SYN to existing connection: drop or RST.
• Mid-sequence packet: validate sequence numbers (advanced).

UDP:

• Stateless protocol. Conntrack uses (src, dst, port pair) tuple.
• "ESTABLISHED" = saw response from dst.
• Timeout-based cleanup.

ICMP:

• Echo request → reply: track pairs.
• Other ICMP types not really stateful.

Connection rate limiting:

iptables -A INPUT -p tcp --syn -m connlimit --connlimit-above 50 --connlimit-mask 32 -j REJECT

Limit concurrent connections per source IP. Defends against scan/DoS.

SYN cookies (kernel feature):

• When SYN flood happens, kernel doesn't allocate connection state immediately.
• Encodes state in TCP sequence number (cookie).
• Validates on ACK.
• Saves memory under attack.

Synproxy:

• iptables module: terminate TCP handshake on firewall, validate, then connect to backend.
• Mitigates many attack patterns.

Window scaling, timestamps, SACK:

• Modern firewalls track these per-connection.
• Mismatch = invalid packet.