fsync and Durability: When Is a Write Safe?

When a program writes to a file, the data usually has not reached the disk. The operating system keeps it in its page cache and writes it out later, and a power cut in between loses it, even though the write call returned success. fsync is the system call that does not return until the data is on stable storage. It is slow (milliseconds on a spinning disk, about 0.1 to 1 ms on an SSD), so a database must decide how often to pay for it. After this lesson your log has a sync policy, and you can see exactly which writes a crash loses under each one.

Two places a record can be

The simulator keeps two lists:

• the buffer: records handed to the operating system. They survive a process crash but not a power cut. CRASH here models a power cut: the buffer is lost along with memory.
• the log: records that have been fsynced. They are durable and survive CRASH.

fsync moves everything in the buffer into the log and counts one fsync call. Calling it with an empty buffer does nothing and is not counted.

The policy

SYNC chooses when a new record is fsynced right after it is appended to the buffer:

batch is the idea behind group commit: many writers share one fsync. A real server batches records from concurrent transactions and acknowledges all of them after the single fsync, so they are all durable and the cost is shared. Here, with a single writer, batch n simply shows the trade: the fewer the fsyncs, the more a crash can take with it. SQLite exposes the same dial as PRAGMA synchronous (FULL, NORMAL, OFF).

What changes in your code

Only the write path: append_record now puts the record in the buffer (taking the next LSN as before) and then calls fsync if the mode says so. FSYNC and CHECKPOINT (which must make everything durable before it snapshots) already call the provided fsync.

After a crash and RECOVER, the next LSN continues from the last durable record. LSNs given to lost buffered records are handed out again, which is correct: those records never existed as far as the surviving log is concerned.

Commands

• SYNC always, SYNC never, SYNC batch <n> (n at least 1): set the mode, OK. Anything else is ERR usage: SYNC always | never | batch <n>. Changing mode does not flush. The default is always.
• FSYNC: OK synced=<records moved to the log>.
• STATUS: durable=<records in the log> buffered=<records in the buffer> fsyncs=<fsync calls that moved something>.
• LOG, TEAR and CORRUPT only see the durable log.

Common mistakes

• Counting a no-op fsync. Only a call that moves at least one record counts.
• Losing the LSN order: the record gets its LSN when it is appended, whether or not it has been fsynced yet.
• Putting the record in the log first and then the buffer, so that never mode still looks durable.
• Forgetting that batch 1 equals always.

Your exercise

The starter has the buffer, fsync, the commands and the checkpoint code. You write the new append_record. The mistake to avoid: writing straight to the durable log and skipping the buffer.