OverlayFS Internals

The last two lessons treated the layer stack as "topmost wins", which handles reads and writes. It cannot handle deletion — you cannot remove a file from a read-only layer, and yet rm /etc/hosts inside a container plainly works. OverlayFS answers with two markers that live in the upper layer and mean something to nothing but the kernel.

Whiteouts

Delete a file that exists only in a lower layer and OverlayFS creates, at that path in the upper layer, a character device with major:minor 0:0. It is not a file, it holds nothing, and its only job is to be recognised:

upper/etc/hosts   ->  char device 0:0        "this path is deleted"
lower/etc/hosts   ->  the real file          (still there, still shared)
the container sees:   ENOENT

The lower file is untouched, because it must be — the other containers on this host are still reading it. Deletion inside a container is a statement about that container's view, never about the image.

Opaque directories

rm -rf /var/log would need one whiteout per entry, and per entry of every subdirectory, for a tree you may never list. Instead OverlayFS marks the upper directory with an extended attribute:

trusted.overlay.opaque="y"

The rule is blunt: when listing this directory, do not merge the lower layers at all.

upper/var/log/  (opaque)
   my-app.log
lower/var/log/
   syslog, auth.log, dpkg.log, ...
the container sees:  my-app.log

One xattr hides an arbitrarily large subtree in constant time. It is also why a directory you deleted and recreated inside a container comes back genuinely empty, rather than repopulated from the image.

Copy-up, once more with the cost visible

Every first write to a lower-layer file promotes the whole file to upper before the write lands. Combined with whiteouts this gives the container's writable layer a precise meaning: it holds the files you changed, the files you created, and a marker for each file you deleted. Nothing else. That set is what docker diff prints and what docker commit turns into a new layer.

Why the exercise is a simulator

The markers are a char device and an xattr — neither creatable without privileges the grader sandbox does not grant. The semantics are a small set of visibility rules, and those rules are where all the surprising behaviour comes from.

Your exercise: OverlayFS with Whiteouts & Opaque Dirs

The ordering rule that decides most cases: upper, then whiteout, then lower. A path in upper is served from upper and no marker matters; a whiteout with no upper entry means ENOENT even though the lower file is right there; only a path with neither falls through to lower. WRITE after DELETE must clear the whiteout, because the file is genuinely back — that round trip is worth testing yourself before you submit.

For LS, compute the upper children first, then add lower children unless the directory is opaque, skipping any that carry a whiteout. Children are the entries whose parent directory is exactly the one being listed — not everything underneath it.

Reference: kernel Documentation/filesystems/overlayfs.rst.