Skip to content
Lesson 1 of 24

Step 1 of 5 · Reading · ~4 min

Read

Boot & Memory

What an OS Kernel Does

A kernel is the one program the CPU trusts completely. Everything else you run, your shell, your browser, the programs you will load in this course, executes with the hardware's permission system pointed at it. The kernel is what that permission system points away from. After this lesson your program will replay a trace of CPU events and say which privilege ring the processor is in at every moment, including the instant a user program tries something it is not allowed to do.

Rings: privilege in hardware

Every x86 CPU keeps a 2-bit current privilege level (CPL) in the low bits of the CS register. Rings 1 and 2 exist but no mainstream OS uses them.

CPLNameWhat it can do
0kernel mode ("ring 0")every instruction, every byte of memory
3user mode ("ring 3")ordinary instructions on memory the kernel mapped for it

Ring 0 is special because of the privileged instructions. Executed at CPL 3 they do not run: the CPU raises a general-protection fault, exception vector 13, written #GP, and hands control to the kernel instead.

InstructionWhat it would let a user program do
hltstop the CPU until the next interrupt
cli, stiswitch interrupts off and on
in, outtalk to device ports (with IOPL 0, as every mainstream OS runs)
lgdt, lidt, ltrreplace the descriptor tables and task register
mov cr0, mov cr3, mov cr4change paging, i.e. the entire memory map
rdmsr, wrmsrread and write model-specific registers
invlpg, wbinvdinvalidate translations and flush caches

If one of these worked in ring 3 the isolation would be over: mov cr3 swaps page tables, cli plus hlt freezes the machine, out reprograms the disk controller. Ordinary instructions (mov, add, jmp, nop) are the same in both rings.

How the CPU moves between rings

User code can never jump to ring 0. There is no "promote me" instruction. Every 3-to-0 transition is the CPU itself transferring control to an entry point the kernel registered earlier:

  • System call. The program asks for service with syscall. The CPU sets CPL to 0 and jumps to the one address the kernel wrote into a model-specific register at boot. The program chooses when; the kernel chose where.
  • Interrupt or exception. A device (timer, keyboard, disk) or a fault (page fault, divide by zero, the #GP above) suspends whatever was running and vectors into the kernel's handler at CPL 0. The vector number selects which handler.

Exactly two instructions go the other way, and only the kernel can usefully run them: sysret (the partner of syscall) and iret, which restores the interrupted context. If the saved context was ring 3, execution continues in ring 3.

            syscall / interrupt / exception / #GP
 ring 3 ------------------------------------------>  ring 0
 ring 3 <------------------------------------------  ring 0
                  sysret / iret

A worked trace

A user program makes a system call, then tries cli, then the kernel (now in ring 0) returns to it:

EventCPL afterOutput
(power-up of the user program)3
SYSCALL0
EXEC cli0none, allowed in ring 0
SYSRET3
EXEC cli0#GP, the fault itself lands in the kernel
PROBE0ring 0

The last two rows are the surprise: the failed instruction does not leave the program in ring 3. The fault is a transition to ring 0, and the kernel will decide whether to kill the process or fix things up.

What the kernel does with that monopoly

It owns memory (which frames each process may touch), CPU time (who runs next, enforced by the timer interrupt), devices (every port and memory-mapped register) and isolation (a crashing process damages only itself). Keep one split in mind for the whole course: mechanism versus policy. The context switch (mechanism) does not care which process runs next; the scheduler (policy) decides.

Common mistakes

  • Treating sysret/iret as allowed from anywhere. SYSRET executed in ring 3 is itself a #GP.
  • Leaving the CPU in ring 3 after a fault. Every exception enters the kernel.

Your exercise

Replay the event trace and answer each PROBE. The starter reads the trace and prints PROBE answers; you write the transition function that decides the new ring and whether the event faulted. Lines you do not recognise are skipped, not errors.

Up nextBoot ProcessBoot & Memory

Discussion

Ask a question, share an insight, or help someone who’s stuck.

Sign in to post a comment or reply.

Loading…