Skip to content
Lesson 5 of 8

Step 1 of 5 · Reading · ~1 min

Read

Failure Detection

Phi-Accrual Failure Detector

Binary alive/dead is too brittle. φ-accrual detector (Hayashibara et al. 2004): output a CONFIDENCE value.

Each heartbeat arrival is logged. Inter-arrival times form a distribution. When you query "is X alive?", compute:

φ(t) = -log10(P_later(t - last_heartbeat))

Where P_later(d) is the probability that a normal heartbeat takes longer than d seconds. Use the inter-arrival distribution.

φ values:

  • 0: heartbeat is on time
  • 1: 90% confident node is dead
  • 2: 99%
  • 3: 99.9%
  • ...

Configurable threshold (typically φ > 8) triggers "node is dead". Adapts to network conditions: if heartbeats normally take 100ms, a 200ms gap is suspicious. If they normally take 5s, a 200ms gap is fine.

Used by Cassandra, Akka cluster, Hazelcast.

Why is this better than fixed-timeout?

  • A bursty network with occasional 1s gaps doesn't trigger false positives.
  • A normally-instant network with a 100ms gap DOES trigger.

Adaptiveness > fixed thresholds.

Up nextConvergence TimeFailure Detection

Discussion

Ask a question, share an insight, or help someone who’s stuck.

Sign in to post a comment or reply.

Loading…