Chapter 1: The CPU's Contract§

Authors: Shrey Patel and Jay Patel, Coconut Labs
Book: Think Like an OS. See book/00-INDEX.md

After this chapter you can explain what "kernel mode" physically is: a hardware state the CPU checks on every instruction. You can name the privilege levels on x86_64 and ARM64 and say what runs at each, list the concrete operations the CPU refuses to unprivileged code, and describe how Meltdown punched through this boundary in 2018 and what fixing it cost. You will also have run programs that get killed by the CPU for trying, and watched the privilege level change live under a debugger.

Two kinds of reader need this chapter, for different reasons. If you are heading for kernel work, this is the floor everything later stands on: page tables, syscalls, credentials, and Coconut's capability tokens all assume the CPU enforces one piece of state that software cannot forge. If you are heading for low-latency systems work, trading included, this is where your syscall bill comes from. Every crossing from user code into the kernel is a hardware privilege transition. Its price is set by silicon you do not control, and the Meltdown section below watches that price get rewritten across the industry, mid release-cycle, because of one disclosure. That is the mechanism under every "count your syscalls" rule you have ever been handed.

The problem [fundamental]§

An operating system's most basic job is refereeing: many programs share one machine, and the OS decides who gets the CPU, which memory belongs to whom, and who may touch the disk and the network. Chapter 0 called this the protection referee role.

Here is the problem: the OS is just code. It sits in the same RAM as everything else, executes on the same CPU, and has no magical substance that makes it special. If any program could execute any instruction, then any program could overwrite the referee's code, redirect the referee's data structures, or switch the referee off. A referee that the players can rewrite is not a referee. It's a suggestion.

Software alone cannot fix this. You could try making every program promise to behave. That was literally the plan in early systems. Classic Mac OS and Windows 3.x used cooperative multitasking: programs voluntarily yielded the CPU, and one buggy or hostile program could freeze or corrupt the entire machine. Every crash took the whole system down because nothing enforced the boundary between one program's mistakes and everyone else's memory.

The only way out is to put the enforcement below software, in the silicon. The CPU itself must maintain a notion of "who is currently running: the referee, or a player?" and must check that state before executing dangerous instructions or touching protected memory. That hardware state, and the rules the CPU enforces around it, is the contract this chapter is about. Every other protection mechanism in this book (virtual memory, system calls, credentials, Coconut's capability tokens) is built on top of this one bit of hardware truth.

Unfolded [fundamental]§

Privilege is a hardware state, not a software convention [fundamental]§

Every modern application-class CPU carries, at all times, a current privilege level. It is not a variable in memory that code could overwrite. It is part of the processor's internal execution state, like the program counter. On x86_64 it lives in the bottom two bits of the CS register (the code segment selector, a small register that describes what kind of code is currently running). On ARM64 it is the current exception level, tracked by the core itself.

The CPU consults this state constantly:

Two consequences fall out of this design, and they are the whole story:

  1. Unprivileged code cannot raise its own privilege. There is no instruction that says "make me privileged". The instructions that do change privilege level simultaneously transfer control to a code address the kernel chose in advance. You can become privileged only by jumping into the referee's own code, at the referee's chosen entry point.
  2. The kernel is ordinary code plus a hardware state. "Kernel mode" is not a place or a substance. The same physical CPU runs your web browser and the scheduler; the only difference is the privilege state and, because of it, what the CPU will permit.

Vendors name these states differently. x86 says rings, ARM says exception levels. The shape is identical: a ladder of privilege, hardware-checked at every rung.

x86_64: four rings, two used [fundamental]§

Intel's protected mode, introduced with the 80286 and carried forward ever since, defines four privilege rings numbered 0 through 3. Ring 0 is the most privileged, ring 3 the least. The current ring is called the CPL, the current privilege level, and it is stored in the low two bits of CS.

The original vision was layered: OS core in ring 0, drivers in ring 1, system services in ring 2, applications in ring 3. Almost nobody shipped that. Linux, Windows, and the BSDs all settled on two levels: kernel in ring 0, everything else in ring 3.

RingOriginal intentWhat Linux actually runs there
0OS kernelThe entire kernel, including drivers
1DriversNothing
2System servicesNothing
3ApplicationsEvery userspace process, root included

Why did rings 1 and 2 die? Two reasons compound:

So on the machines Coconut OS targets, the practical model is binary: ring 0 is the kernel, ring 3 is everyone else, and "supervisor vs user" is the only distinction the memory hardware draws. Portability reinforced this. Most other architectures only ever offered two levels, so a portable kernel could not lean on four.

ARM64: exception levels EL0-EL3 [fundamental]§

ARM's 64-bit architecture (AArch64, the ISA of every modern phone, Apple Silicon Mac, and ARM server) makes the ladder explicit and, mercifully, numbers it in the intuitive direction. There are four exception levels, EL0 through EL3, and higher numbers mean more privilege: EL0 < EL1 < EL2 < EL3.

LevelWho runs thereNotes
EL0Applications (userspace)The unprivileged level; equivalent of ring 3
EL1OS kernelLinux lives here; equivalent of ring 0
EL2HypervisorKVM's low-level world-switch code; virtualization support
EL3Secure monitor firmwareGateway to TrustZone; the only level that can switch the security state

Unlike x86's decorative middle rings, EL2 and EL3 earn their keep. EL2 exists so a hypervisor can sit genuinely below the kernels it hosts: a guest OS at EL1 cannot touch hypervisor state at EL2, by the same hardware logic that keeps EL0 out of EL1. EL3 runs the secure monitor: vendor firmware that arbitrates between the normal world and TrustZone's secure world, where things like key stores run. Hardware designers may omit EL2 and EL3; EL0 and EL1 are mandatory. One wrinkle to carry forward: ARMv8.1's Virtualization Host Extensions let Linux itself run at EL2 when it is acting as the hypervisor host, which is how KVM commonly runs on modern cores. Chapter 36 returns to this.

   least privileged
  ┌─────────────────────────────┐      ┌─────────────────────────────┐
  │ x86_64 (long mode)          │      │ ARM64 (AArch64)             │
  ├─────────────────────────────┤      ├─────────────────────────────┤
  │ Ring 3   your processes     │      │ EL0   your processes        │
  │ Ring 2   (unused)           │      │ EL1   Linux kernel          │
  │ Ring 1   (unused)           │      │ EL2   hypervisor (KVM)      │
  │ Ring 0   Linux kernel       │      │ EL3   secure monitor fw     │
  └─────────────────────────────┘      └─────────────────────────────┘
   most privileged at the BOTTOM        most privileged at the BOTTOM
   (note: ring numbers DECREASE          (EL numbers INCREASE with
    with privilege, x86 counts            privilege, ARM counts up)
    down)

The mapping to remember: ring 3 ≈ EL0, ring 0 ≈ EL1. When this book says "user mode" and "kernel mode," it means those pairs.

What privileged operations look like concretely [working]§

"Privileged instruction" stays abstract until you see the actual mnemonics. Here are the ones you will meet repeatedly in this book, and what each controls.

On x86_64:

InstructionWhat it doesWhy it must be privileged
mov %rax, %cr3Writes CR3, the register holding the physical address of the current page tablesWhoever writes CR3 chooses the entire memory view. A user process writing CR3 could map anything anywhere
rdmsr / wrmsrRead/write a model-specific register: ECX selects which MSR, EDX:EAX holds the 64-bit valueMSRs configure the machine itself: the syscall entry address, power states, mitigation toggles. rdmsr raises a general-protection fault (#GP) unless CPL is 0
hltStops the core until the next interruptA user process halting a shared CPU is a denial of service in one byte. Requires CPL 0; #GP otherwise
in / outRead/write an I/O port, the legacy channel to devicesDirect device access bypasses every kernel policy. Gated by the IOPL field in RFLAGS plus a per-process port bitmap; Linux exposes exceptions via ioperm(2)/iopl(2), which require CAP_SYS_RAWIO

The kernel runs these constantly and legally. Every context switch writes CR3 (Chapter 3 and Chapter 9). The idle loop is, in effect, hlt executed forever, waking on each interrupt. During boot the kernel writes an MSR called LSTAR (number 0xC0000082) with the address of its own syscall entry code. That single wrmsr is what makes the syscall instruction later jump to the kernel's chosen address and nowhere else.

On ARM64, the same territory is covered by system registers, accessed with two instructions: mrs x0, <sysreg> reads a system register into a general register, and msr <sysreg>, x0 writes one. (Yes: ARM's MSR instruction and x86's "MSR" registers are an unfortunate acronym collision, Move to System Register versus Model-Specific Register.) The register names encode the minimum privilege in their suffix. TTBR0_EL1 (the page-table base, ARM's CR3), SCTLR_EL1 (system control), and VBAR_EL1 (the exception vector base, where the CPU jumps on any trap) are all _EL1 registers, so EL0 code touching them takes an undefined-instruction exception, which Linux delivers to the process as SIGILL. There is one narrow, deliberate exception: Linux traps and emulates EL0 reads of a few read-only CPU-identification registers so programs can discover CPU features. That is the kernel choosing to answer on the hardware's behalf, not the hardware letting EL0 through.

Notice the pattern in both columns. The privileged operations are exactly the ones that define the environment other code runs in: which memory exists, where traps land, whether the core runs at all. The contract in one sentence: code at user privilege can compute anything but configure nothing.

The moat runs both ways: SMEP, SMAP, PXN, PAN [working]§

So far the boundary keeps user code out of kernel resources. Around 2012, CPU vendors started adding checks in the opposite direction: features that constrain what the kernel can do with user memory. That sounds backwards until you see the attack they kill.

A classic kernel exploit works like this: the attacker finds a kernel bug that lets them redirect a code pointer, say by corrupting a function pointer the kernel will call. Kernel memory is hard to write to, but the attacker fully controls their own process's memory. So they place their payload in their own user pages and aim the corrupted kernel pointer at it. The CPU, running at ring 0, happily executes attacker-supplied instructions with full privilege. The technique is called ret2usr (return to userspace), and it turns one bad pointer into total compromise.

The hardware answer: make it a fault for privileged code to execute, or even casually read, user-accessible memory.

FeatureArchWhat it blocksMechanism
SMEP (Supervisor Mode Execution Prevention)x86, Ivy Bridge 2012Kernel executing instructions from user-accessible pagesCR4 bit 20; instruction fetch at CPL < 3 from a user page faults
SMAP (Supervisor Mode Access Prevention)x86, Broadwell 2014Kernel reading/writing user-accessible pages by accidentCR4 bit 21; data access at CPL < 3 to a user page faults unless the kernel temporarily sets the AC flag with stac, clearing it after with clac
PXN (Privileged eXecute Never)ARM64Kernel executing from user pagesA per-page attribute bit in the page-table entry; Linux sets PXN on every user page mapping
PAN (Privileged Access Never)ARM64, v8.1Kernel loads/stores to user-accessible memoryA processor state (PSTATE) bit; when set, EL1 accesses to user-accessible pages fault. Dedicated unprivileged-access instructions (ldtr/sttr) exist for the deliberate cases

The escape hatches are the interesting part. The kernel must legitimately touch user memory all the time: every read() copies data into a buffer the caller supplied. With SMAP or PAN live, those copies happen only inside narrow, explicitly bracketed windows (copy_to_user() and friends wrap the stac/clac or PAN toggling). Anything outside a bracket that touches user memory is, by definition, a bug, and it now crashes loudly during development instead of becoming an exploit in production. This is a theme you will see across kernel hardening: convert "silently exploitable" into "immediately fatal."

Meltdown: when the boundary leaked [working]§

For thirty years, the ring 0/ring 3 contract looked airtight. Then on January 3, 2018, a week ahead of the planned coordinated date and after observers noticed suspicious patches landing in the Linux kernel, researchers disclosed Meltdown (CVE-2017-5754) and Spectre (CVE-2017-5753, CVE-2017-5715). The industry learned that the boundary had been leaking through a side door the whole time.

The side door is speculative execution. Modern CPUs do not wait to find out whether an instruction is allowed before starting work on the instructions after it; they guess, race ahead, and if the guess was wrong they throw the results away and pretend it never happened. The architectural state, meaning registers and memory as the program can observe them, is rolled back perfectly. But the microarchitectural state is not: work done speculatively leaves traces in the CPU's caches, and cache state is measurable, because a cached read is much faster than an uncached one.

Meltdown weaponized a specific gap on affected CPUs (primarily Intel, plus a few others such as ARM's Cortex-A75): the privilege check on a load could resolve after speculative execution had already used the loaded value. A user process would read a kernel address. That read is architecturally forbidden, and the instruction does eventually fault. But in the speculative shadow before the fault, the secret byte briefly existed in a register, and a second dependent load used that byte to pick which of 256 cache lines to touch. The fault fires, the registers roll back, the process catches SIGSEGV and shrugs. Then it times accesses to those 256 lines. The one that comes back fast reveals the secret byte. Repeat at kilobytes per second and a normal unprivileged process is reading arbitrary kernel memory: keys, passwords, other processes' data.

Why did the kernel's memory even resolve to something readable from a user process? For performance, Linux (like every major OS) kept the kernel mapped in every process's page tables. It was marked supervisor-only, so architecturally untouchable, but present, so syscalls and interrupts did not need an expensive page-table switch. Meltdown turned that decades-old optimization into a liability: "mapped but forbidden" collapsed to "mapped."

The fix, KPTI (Kernel Page Table Isolation), removed the assumption: while a process runs in user mode, the kernel is not mapped at all. Only a minimal trampoline remains: the entry and exit code, plus the few structures that must exist to take interrupts. Every user↔kernel transition now switches page tables, both directions. KPTI was merged for Linux 4.15-rc6 and backported to 4.14 stable. Merging something this invasive mid release-cycle was nearly unprecedented, which tells you how bad the bug was.

The cost is real and permanently instructive: extra work on every single syscall, interrupt, and page fault. Measured overhead ran around 5% for typical workloads and up to roughly 30% for syscall-heavy ones, heavily dependent on whether the CPU supports PCID (a tag that lets translation caches survive a page-table switch instead of being flushed; Chapter 3). Read that 30% as a claim about your own hot path rather than a historical footnote. If your workload is syscall-heavy, part of its latency was set by a mitigation decision made in someone else's building. The lesson for OS designers: the privilege boundary is not free, its price is set by hardware behavior you do not control, and it can be repriced overnight by a vulnerability disclosure.

Spectre, disclosed the same day, is the harder sibling: it tricks victim code into leaking its own secrets through the same cache side channel, doesn't depend on the Meltdown gap, affects nearly every speculating CPU (Intel, AMD, ARM), and KPTI does not fix it. Its mitigations are a running theme in later chapters; the honest summary is that the industry is still paying for it.

Crossing on purpose: a preview [fundamental]§

If ring 3 code cannot touch anything interesting, how does your program ever open a file? It asks. There are exactly two kinds of legitimate crossing from user to kernel mode, and both land at addresses the kernel chose in advance:

  user mode (ring 3 / EL0)                 kernel mode (ring 0 / EL1)
 ─────────────────────────────            ─────────────────────────────
  your code: syscall / svc  ──────────▶   entry point the kernel
             (voluntary)                  registered at boot
                                          (LSTAR MSR / VBAR_EL1)
  your code: <interrupted>  ──────────▶   interrupt/exception vector
             (involuntary)                (kernel-chosen, Chapter 2)
                                                     │
  your code resumes         ◀──────────   sysret / eret

On x86_64 the voluntary door is the syscall instruction: it atomically switches to ring 0 and jumps to the address in that LSTAR MSR the kernel wrote during boot. On ARM64 it is svc (supervisor call), which raises a synchronous exception into the EL1 vector table at VBAR_EL1. Involuntary crossings (a timer interrupt, a page fault, or the #GP you will trigger in the lab) go through the same vector machinery. In all cases privilege and control transfer in one indivisible step; there is no moment where code is privileged but still running attacker-chosen instructions. Chapter 2 dissects the vectors; Chapter 7 walks the syscall path instruction by instruction.

The real thing in Linux [working]§

Where this chapter's machinery lives in the Linux 6.12 tree (the tree Coconut OS forks). Paths verified against v6.12 source:

Coconut tie-in [working]§

Coconut OS (currently in spec phase, pre-implementation) is a hard fork of Linux 6.12 LTS, and it inherits this chapter's machinery byte-for-byte unchanged. That is a design position, not an accident: AI agents in Coconut are first-class kernel primitives, but they are not privileged ones. An agent process runs at ring 3 (EL0 on later ARM64 targets) exactly like any other process. No new ring, no agent-special privilege state.

What Coconut adds are new doors through the same wall. The agent_* syscall family occupies numbers 472-479: agent_spawn (472) and agent_attest (473) are wired in the spec; 474-479 are reserved -ENOSYS stubs. Each is one more kernel-chosen entry point reachable via the same syscall instruction dispatching through the same entry_SYSCALL_64 path, which is Chapter 7 territory. The enforcement that makes agents governable all lives on the ring 0 side of the boundary: capability checks in security/coconut/, the agent registry in kernel/agent/, BLAKE3-chained audit emission from kernel/audit/coconut/. That placement is this chapter's referee argument applied directly: the code being supervised (the agent, at ring 3) must be physically unable to rewrite the code supervising it, and hardware privilege is the only mechanism that guarantees it.

Two practical consequences from the spec (04-HLD, 05-LLD): v1.0 targets x86_64 only, so ring 0/3 plus SMEP/SMAP is the enforcement substrate v1 assumes. The ARM64 story (EL0/EL1, PAN/PXN) arrives with v1.1's server target. And the CI gate you will meet in Chapter 31, kunit-coconut, boots the fork's kernel in QEMU x86_64 on every push, which is the same rig as the lab below.

Lab [fundamental]§

Goal: get personally rejected by the privilege check, then observe the privilege level directly. Host is macOS, so everything runs inside a Linux environment: the x86_64 QEMU guest from Chapter 0's lab bench, or (steps 1-3 only) any Linux x86_64 Docker container such as docker run --rm -it gcc:14 bash on an x86_64 host. The signals in question come from the CPU+kernel contract, so a container on a native x86_64 kernel and a QEMU guest both show the real behavior.

Three of the four steps take about a minute each, and two of them end with a dead process. That is the point. You are going to be refused by the hardware, on purpose, and then go look at the state that refused you.

1. Execute hlt from ring 3 [fundamental]§

c
// halt.c - attempt the privileged hlt instruction at CPL 3
#include <stdio.h>

int main(void)
{
    printf("about to hlt at ring 3...\n");
    fflush(stdout);
    __asm__ volatile("hlt");
    printf("unreachable\n");
    return 0;
}

Predict first. The CPU will refuse, but which signal kills the process? Most people guess SIGILL ("illegal instruction"). Write your guess down.

sh
gcc -O0 -o halt halt.c
./halt; echo "exit status: $?"

Expected:

about to hlt at ring 3...
Segmentation fault (core dumped)
exit status: 139

SIGSEGV, not SIGILL. Status 139 is 128+11, and 11 is SIGSEGV. The instruction is perfectly legal; running it at CPL 3 is not. The CPU raises a general-protection fault (#GP), which vectors into the kernel (an involuntary crossing, exactly as in the preview diagram), and Linux's #GP handler delivers SIGSEGV to a faulting user process. Meanwhile, the kernel itself executes hlt all day in its idle loop, legally, at ring 0.

2. Read an MSR from ring 3 [fundamental]§

Aim higher: try to read MSR_LSTAR (0xC0000082), the register holding the kernel's syscall entry address.

c
// readmsr.c - attempt rdmsr of MSR_LSTAR at CPL 3
#include <stdio.h>
#include <stdint.h>

int main(void)
{
    uint32_t lo, hi;
    printf("about to rdmsr 0xC0000082 at ring 3...\n");
    fflush(stdout);
    __asm__ volatile("rdmsr" : "=a"(lo), "=d"(hi) : "c"(0xC0000082u));
    printf("LSTAR = 0x%08x%08x\n", hi, lo);   // never reached
    return 0;
}

Predict: same or different signal as hlt?

sh
gcc -O0 -o readmsr readmsr.c
./readmsr; echo "exit status: $?"

Expected:

about to rdmsr 0xC0000082 at ring 3...
Segmentation fault (core dumped)
exit status: 139

Same #GP → SIGSEGV path: rdmsr faults unless CPL is 0. There is a legitimate route. As root, the kernel's msr driver (modprobe msr, then the rdmsr utility from msr-tools reading /dev/cpu/0/msr) will read it for you, because then ring 0 code does the read on your behalf, subject to kernel policy. That asymmetry is the entire contract in miniature.

Two dead programs in, you can answer something that catches out experienced developers: a SIGSEGV does not always mean a bad pointer. It means the CPU refused, and a #GP from a privileged instruction is one of the ways it refuses.

3. Inspect the hardening flags [fundamental]§

sh
grep -o -w -E 'smep|smap' /proc/cpuinfo | sort -u

Expected:

smap
smep

If either is missing in your QEMU guest, your virtual CPU model is not advertising it. Relaunch QEMU with -cpu max (or -cpu host with hardware acceleration) and re-check. When present, the booting kernel's setup_smep()/setup_smap() enabled them via CR4, and the ret2usr technique from this chapter is dead on your machine.

4. Optional: watch CPL change under the debugger [advanced]§

QEMU embeds a gdb server. Launch your Chapter 0 guest with two extra flags: -s (gdbstub on TCP port 1234) -S (freeze at the first instruction), then from the host:

sh
gdb -ex 'target remote :1234' -ex continue

Let the guest boot to a shell. Now, in the guest, run a pure-userspace spin: while :; do :; done. Hit Ctrl-C in gdb to freeze the machine mid-spin and look at CS:

(gdb) p/x $cs
$1 = 0x33
(gdb) p $cs & 3
$2 = 3

Expected: low two bits 3. You froze the CPU in ring 3. Kill the spin loop, let the guest sit idle, and Ctrl-C again:

(gdb) p $cs & 3
$3 = 0

Expected: ring 0. An idle guest is almost always inside the kernel's hlt idle loop, the very instruction that killed your program in step 1. (QEMU's monitor offers the same view: info registers prints the full hidden segment state including a CPL= field.) You have now watched the privilege state this whole chapter is about, flipping in real time.

Bridge notes [fundamental]§

For the reader arriving from microprocessor/microcontroller coursework and compiler design:

You already knowWhat is new at this scale
8051/AVR: one flat privilege domain. Any code writes any SFR or any address, and a wild pointer takes down the machineThat world is cooperative-multitasking-era PC computing in miniature. Hardware privilege exists precisely because "all code is trusted" stops scaling past one program. The 8051 has no referee because it runs exactly one team
Cortex-M: handler mode vs thread mode; CONTROL.nPRIV makes thread mode unprivileged; unprivileged code cannot write CONTROL and must go through an exception (e.g. SVC) to regain privilegeThe same ladder, two rungs tall. Cortex-M's SVC-into-handler-mode is architecturally the same move as svc from EL0 into EL1: privilege rises only through a hardware-vectored entry point. What Cortex-M lacks is an MMU, so there are no per-process address spaces, only an optional MPU with a handful of regions. EL0/EL1 plus paging is that idea industrialized: thousands of processes, each in a private virtual world (Chapter 3)
Memory-mapped I/O and vector tables on small coresSame concepts, now privilege-gated: device MMIO sits behind supervisor-only mappings, x86 additionally has legacy port I/O (in/out) behind IOPL/ioperm, and the vector base itself (IDT / VBAR_EL1) is privileged state. The referee decides where traps land
Compilers: calling conventions, a rigid contract for crossing a function boundary (who saves what, where arguments go)The syscall boundary is a calling convention enforced by silicon: fixed entry address (LSTAR / VBAR_EL1), fixed register contract (Chapter 7), and, unlike a function call, a callee that distrusts the caller completely. Every argument gets validated, and SMAP/PAN bracket every touch of caller memory
Compilers: undefined behavior, which the compiler assumes never happensMeltdown is what happens when hardware "as-if" breaks: speculation was supposed to be invisible like compiler reordering, but caches made it observable. Same class of bug as a miscompilation, with the optimizer leaking through the abstraction

Three things you can do now that you could not do before this chapter: read a crash whose signal does not match its apparent cause, name the exact hardware state that "permission denied" refers to at this layer, and check on any machine you touch whether SMEP and SMAP are actually enabled. Chapter 2 takes you through the other door in the diagram, the involuntary one, and shows what the machine does in the microseconds after it stops running your program.

Sources [working]§