Chapter 3: Memory Hardware§

Authors: Shrey Patel and Jay Patel, Coconut Labs
Book: Think Like an OS. See book/00-INDEX.md

After this chapter you can take a virtual address, walk it through the x86_64 page tables by hand, name every bit in the page-table entry you land on, and explain why the same load can cost 1 nanosecond or 100. You will measure all three claims yourself: a pointer-chasing benchmark that exposes the L1/L2/L3/DRAM latency cliffs, a program that translates one of its own virtual addresses to physical via /proc/self/pagemap, and a perf experiment showing huge pages cutting TLB misses. This is the single biggest hardware delta between the microcontrollers you may have programmed in school and the servers Coconut OS targets.

Both of this book's destinations meet here. If you are heading for kernel work, page tables are the data structure the kernel spends the most care on, and the fault path in Chapter 11 stays unreadable until the walk below is automatic for you. If you are heading for low-latency work, this chapter is most of the job: the nanoseconds-per-load ladder in Lab 1, the TLB-miss counter in Lab 3, and the false-sharing section at the end are three mechanisms you can act on from userspace, without a kernel patch or anyone's approval. Nothing in this chapter is a metaphor. Every claim has a lab that measures it.

The problem [fundamental]§

Run two programs on a machine with no memory hardware between the CPU and RAM, and every load and store hits physical memory directly. Both programs were linked to start at address 0x400000, so one has to move. But its pointers are baked into its instructions, so "moving" means rewriting the binary. Worse: program A can write to program B's memory, deliberately or by bug; a stray pointer in a text editor can corrupt your database. Worse still: both programs together must fit in physical RAM, and if both use the C library, each carries its own copy.

Every one of those failures was normal in early computing, and every one still exists on flat-memory microcontrollers. General-purpose machines fixed all of them with one piece of hardware, the memory management unit (MMU), and one idea: put a translation step between the addresses a program uses and the addresses the memory chips see.

Unfolded [fundamental]§

Four problems, one mechanism [fundamental]§

The MMU translates every address a program generates (a virtual address) into an address in physical RAM (a physical address). The kernel controls the translation tables; the hardware applies them on every load, store, and instruction fetch. That indirection solves four problems at once:

ProblemWithout translationWith translation
RelocationPrograms must be linked for the exact physical address they occupyEvery process sees the same virtual layout; the kernel maps it anywhere in RAM
ProtectionAny process can write any byte of RAMA process can only touch memory its tables map; everything else faults
SharingShared code (libc) is duplicated per processOne physical copy mapped into many processes' tables
Illusion of sizeProgram memory must fit in RAMPages can be missing; the kernel fills them in on demand

The fourth row is the deepest. A translation entry can say "not present." When the CPU touches such an address it doesn't crash. It raises a page fault, a hardware exception that hands control to the kernel, which can fetch the data from disk, allocate a fresh page, or kill the process (a segfault). Demand paging, swap, copy-on-write fork, and memory-mapped files are all built from this one trick. Those kernel policies come in later chapters; here we stay on the hardware.

A brief history: segments, and why paging won [working]§

Translation was not always page-based. The 8086 (1978) had segments: a segment register shifted left 4 bits plus a 16-bit offset. The 80286 made segments real protection objects, each with a base, a length limit, and permission bits. Protection came per variable-sized logical region: code, data, stack.

Segments lost decisively, for three reasons. Variable-sized regions fragment physical memory: after enough churn you have free RAM but no contiguous hole big enough, while fixed-size pages fit any free frame. A segment swaps in and out as a unit; a page system can evict any 4 KiB it likes. And C treats memory as one flat array of bytes; nobody wanted the pointer-plus-segment programming model.

The 80386 added paging alongside segmentation, and Unix-family systems immediately configured segmentation into irrelevance: every base 0, every limit maxed, all real work done by pages. AMD made it official in x86-64: in 64-bit "long mode" the hardware treats the CS, DS, ES, and SS segment bases as zero and skips limit checks entirely. Only FS and GS survive, as two extra base registers set via model-specific registers. Linux uses GS for per-CPU kernel data and FS for thread-local storage. The %fs:0x28 stack-canary read in compiler output is the last living fossil of segmentation.

Pages, frames, and page tables [fundamental]§

Paging chops both address spaces into fixed-size chunks. A page is a chunk of virtual address space; a page frame is a chunk of physical RAM; on x86_64 and default ARM64 configs both are 4 KiB. The page-to-frame mapping lives in a page table in ordinary RAM, written by the kernel and read by the MMU.

A flat table won't work: 48-bit addresses at 4 KiB granularity mean 2^36 pages, which is 512 GiB of 8-byte entries per process. The fix is the one a compiler engineer would reach for: a radix tree, keyed on address bits. Address spaces are sparse (a process maps a few slivers of an enormous range) and a tree only materializes branches actually in use. x86_64 uses a 4-level radix tree; each node is exactly one 4 KiB page holding 512 eight-byte entries. 512 entries needs 9 index bits, four levels consume 36, plus 12 bits of byte-offset within the final page: 48 bits total.

The x86_64 four-level walk, step by step [fundamental]§

The root of the current process's tree lives in the CPU register CR3; reloading CR3 is how the kernel switches address spaces. From CR3, the hardware page-table walker performs up to four dependent memory reads per translation.

Take the virtual address 0x00007f1234567890 (a typical user-space address). Slice its low 48 bits into 9+9+9+9+12:

virtual address 0x00007f1234567890, low 48 bits:

  011111110 | 001001000 | 110100010 | 101100111 | 100010010000
   PGD idx      PUD idx     PMD idx     PTE idx      offset
    = 254        = 72        = 418       = 359       = 0x890

           CR3
            │  physical address of the top-level table
            ▼
   ┌─ PGD (level 4) ─┐
   │  entry[254] ────┼──► physical addr of next table
   └─────────────────┘
            ▼
   ┌─ PUD (level 3) ─┐
   │  entry[72] ─────┼──► physical addr of next table
   └─────────────────┘
            ▼
   ┌─ PMD (level 2) ─┐
   │  entry[418] ────┼──► physical addr of next table
   └─────────────────┘
            ▼
   ┌─ PT  (level 1) ─┐
   │  entry[359] ────┼──► physical frame number (PFN)
   └─────────────────┘
            ▼
     physical address = (PFN << 12) + 0x890

(PGD/PUD/PMD/PT are Linux's names; Intel says PML4/PDPT/PD/PT.) Each arrow is a real read from RAM: the walker reads the entry, extracts the physical address of the next table, and indexes it with the next 9 bits. At the bottom, append the untranslated 12-bit offset to the frame address and you have the physical address. If any entry is marked not-present, the walk aborts and the CPU raises a page fault.

Note the cost: one memory access can require four extra memory reads just to learn where the data lives. The TLB (below) exists to make that almost never happen.

With 64-bit registers but 48 translated bits, bits 48 to 63 must be copies of bit 47 (sign extension). Such addresses are canonical; anything else faults. That splits the space into a low half (0 to 0x00007fffffffffff, user space, ~128 TiB) and a high half (0xffff800000000000 up, kernel), with an enormous non-canonical hole between. This is why user pointers start 0x00007f... and kernel pointers 0xffff....

Five levels: 57-bit addresses [working]§

128 TiB per process sounds unbounded until you meet machines with tens of terabytes of RAM. Intel's LA57 extension adds a fifth level, extending virtual addresses from 48 to 57 bits (128 PiB virtual, 4 PiB physical, per the kernel's 5-level paging documentation), and it first shipped in Ice Lake processors. Mechanically it is one more 9-bit index and one more read in the walk. The kernel sets bit 12 (LA57) of the CR4 control register when the CPU advertises the la57 flag in /proc/cpuinfo, and builds support with CONFIG_X86_5LEVEL=y. Linux names the extra level P4D, slotted between PGD and PUD; on 4-level hardware P4D is "folded" (compiled to a no-op), so one kernel source serves both.

Inside a page-table entry [working]§

Every entry at every level is 64 bits: a physical address in the middle (bits 12 to 51), control bits at the edges. The ones the OS lives by, with their x86_64 positions:

BitNameMeaning
0Present (P)Entry valid. If clear, access faults and hardware ignores the other 63 bits, so the kernel stashes bookkeeping there (e.g., swap location)
1Read/Write (RW)If clear, writes fault. Copy-on-write hangs off this bit
2User/Supervisor (U/S)If clear, user-mode access faults. This one bit is the kernel/user memory boundary
5Accessed (A)Hardware sets on any access; kernel clears and checks later. This is how page reclaim finds what is in use
6Dirty (D)Hardware sets on write (final level only): page must be written back before its frame is reused
7Page Size (PS)At PMD/PUD level: stop the walk, because this entry maps a huge page directly
63No-Execute (NX)If set, instruction fetch faults. Stack and heap get NX; a genre of code-injection exploits died with this bit

Notice which bits the hardware writes: Accessed and Dirty. The entry is a two-way channel: the kernel writes policy downward, the MMU reports usage upward. Every serious memory-management algorithm reads those two bits.

The TLB [fundamental]§

Four extra reads per access is a 5x tax nobody actually pays, because the MMU caches completed translations in the Translation Lookaside Buffer (TLB): virtual page number in, physical frame plus permissions out, in about a cycle. Typical sizes are dozens of entries at the first level and low thousands at the second, split between instruction and data sides (hence iTLB and dTLB in perf output).

A TLB hit makes translation free. A miss triggers the hardware walker, which does the reads (themselves served from the data caches, mercifully) and installs the result. The performance model that falls out: touch a small set of pages repeatedly and you run entirely from the TLB; skip across more pages than the TLB holds and you pay a walk per touch. TLB reach at 4 KiB is meager: a 1,536-entry dTLB covers 6 MiB. That mismatch drives the next two sections.

Tagged TLBs and shootdowns [working]§

Two hard problems come with caching translations. First: translations belong to a process, so a CR3 reload makes every cached entry potentially wrong. Flushing everything per context switch makes each process start cold. The fix is tagging: entries carry an address-space ID and lookups match only the current tag. x86 calls the tag a PCID (Process-Context Identifier, 12 bits, written alongside CR3); ARM has had the equivalent, ASIDs, for much longer. Linux enabled PCID in kernel 4.14, keeping a small per-CPU pool of tag slots (six in current x86 code) for recent address spaces, so bouncing between a few processes no longer wipes translation state.

Second, nastier: each core has its own private TLB, and hardware does nothing to keep them consistent. If the kernel unmaps a page while thread A runs on core 0 and thread B on core 7, core 7 may still hold the stale entry, and thread B keeps writing to a page the kernel thinks is gone. The kernel handles this itself: the changing core sends an inter-processor interrupt (IPI) to every core that might cache the mapping; each interrupts its work, invalidates, and acknowledges. This is a TLB shootdown, a classic hidden scalability cost: munmap and mprotect on a big box become broadcast interrupt storms. Systems literature measures a single shootdown across 120 cores at roughly 100 microseconds, which is an eternity in kernel time. If anything on your hot path calls munmap or mprotect, that is the number to carry around: one unmap can interrupt every core that has touched the mapping.

Huge pages [working]§

The PS bit is the escape hatch from TLB poverty. A PMD-level entry with PS set stops the walk and maps a whole 2 MiB region (the 9 PTE-index bits merge into the offset). A PUD-level entry with PS maps 1 GiB. One TLB entry now covers 512x or 262,144x more memory, and the walk is a level shorter.

The costs are fragmentation-shaped: the kernel must find physically contiguous, aligned 2 MiB or 1 GiB chunks, and a huge page is all-or-nothing: dirty one byte and the whole 2 MiB is dirty. Linux exposes them two ways: explicitly, via mmap(MAP_HUGETLB | MAP_HUGE_2MB) (or MAP_HUGE_1GB) from a reserved pool, and transparently (THP), where the kernel assembles 2 MiB pages behind your back, steered by madvise(MADV_HUGEPAGE). Databases, JVM heaps, and (relevantly for Coconut) multi-gigabyte model weights are the canonical winners.

ARM64 translation [working]§

ARM64 plays the same game with different constants, worth knowing because Coconut v1.1 targets ARM64 servers. Three differences:

Configurable granule. The base page size, called the translation granule, is 4 KiB, 16 KiB, or 64 KiB, selected via fields in the TCR_EL1 control register. The kernel's AArch64 memory documentation lists the shapes: 4 KiB pages with 3 levels gives 39-bit (512 GiB) virtual addresses; 4 KiB with 4 levels, 48-bit (256 TiB); 64 KiB with 2 levels, 42-bit (4 TiB); 64 KiB with the LVA extension reaches 52-bit. Most distros ship 4 KiB / 48-bit; Apple's macOS uses 16 KiB.

Two table roots. Where x86_64 has one CR3, ARM64 has two: TTBR0_EL1 translates the low half (user), TTBR1_EL1 the high half (kernel), selected by bit 55 of the virtual address. Context switch swaps only TTBR0; kernel tables never move. x86_64 fakes this by copying kernel entries into every process's top-level table; ARM makes it architectural.

ASIDs from the start, riding in the TTBR0 value. TLB invalidation uses TLBI instructions that can broadcast across cores in hardware, so ARM64 shootdowns don't always need IPIs.

The cache hierarchy [fundamental]§

Translation says where a byte lives; caches decide how long it takes. DRAM costs 50 to 100 ns, which at 4 to 5 GHz is 200 to 400 potential instructions, so CPUs interpose layers of smaller, faster SRAM. Representative numbers for an Intel Skylake-class core (7-cpu.com measurements):

LevelTypical sizeLatencyScope
L1 data32 KiB4 cycles (~1 ns)per core
L2256 KiB to 1 MiB~12 cyclesper core
L3 (LLC)8 to 64 MiB~38 to 42 cyclesshared across cores
DRAMmany GiB~50 to 100 nsshared

Caches move data in fixed units called cache lines: 64 bytes on every x86 part you are likely to meet (some ARM designs use 128). Touch one byte and the whole line arrives, which is why sequential access is fast (the next 63 bytes are already present, and the prefetcher runs ahead) and why pointer-chasing through scattered nodes is the slowest thing you can do to memory: every hop is a full-latency miss with no prefetch help. That asymmetry, not instruction count, is why a flat array beats a linked list by an order of magnitude. The Lab makes you watch it.

Coherence and false sharing [working]§

Private per-core caches raise the staleness problem again: core 0 writes x, core 1 reads x, both hold the line. Who wins? Here hardware does solve it, with a coherence protocol. The MESI family (real chips run variants like MOESI or MESIF) tags each line:

The invariant: many readers or one writer, never both. Before writing a line a core must own it exclusively: it broadcasts a request-for-ownership and every other core invalidates. Reads of a Modified line are served from the owning cache, not stale RAM. Coherence is invisible for correctness but ruinous for performance when two cores alternate writes to one line: it ping-pongs between caches at tens of cycles per transfer.

The trap: coherence works on 64-byte lines, not variables. Put two unrelated counters, written by different threads, in adjacent bytes, and every increment by thread A invalidates thread B's copy, serializing threads that share no data. This is false sharing, the reason kernel per-CPU structures are padded to line boundaries (____cacheline_aligned). If you have ever watched two threads get slower after someone moved their counters closer together in a struct, this paragraph is the entire explanation. It closes the chapter's theme: hardware sells you one flat shared memory, and the entire cost model lives in the machinery maintaining the illusion.

The real thing in Linux [working]§

Where this chapter meets the 6.12-era tree:

Coconut tie-in [working]§

Coconut OS is a Linux 6.12 LTS hard fork, currently in spec phase. Nothing here is shipped; it is design intent from the 04-HLD and 05-LLD.

Coconut inherits this chapter's machinery untouched; what the spec adds is placement policy. v1's mm plan is additive tier-aware hooks: agents are first-class kernel objects (the agent_* syscalls at 472 to 479, with agent_spawn and agent_attest wired), and an agent's dominant memory (model weights, KV caches) is exactly where these constants bite. Read-mostly multi-gigabyte weights are the textbook huge-page client; the tier hooks exist to keep hot KV-cache pages in fast memory while cold ones demote. SCHED_AGENT's tenant-fairness lineage comes from kvwarden, Coconut Labs' CUDA inference broker, and one reason placement keeps an agent's threads on cores sharing an LLC is precisely the shootdown-IPI and line-ping-pong costs above. The audit subsystem (kernel/audit/coconut/, JSON-lines with a BLAKE3 hash chain) is spec'd with per-CPU cache-line-aligned buffers, because false sharing on a hot ring would eat its throughput budget. Since v1.0 is x86_64-only, the 4-level walk you did by hand is the exact target; the ARM64 section becomes load-bearing at v1.1. Every push to the fork already boots in QEMU x86_64 under the kunit-coconut CI gate, which is the environment this Lab assumes.

Lab [working]§

Host is macOS, so run these in a Linux environment: the book's QEMU guest, or docker run -it --rm -v "$PWD":/work -w /work gcc:14 bash. Labs 1 and 2 work anywhere Linux runs. Lab 3 needs hardware performance counters, which Docker Desktop's VM and QEMU-TCG guests do not provide. Use bare-metal Linux or a KVM guest with PMU passthrough; expect <not supported> elsewhere.

Lab 1: see the latency cliffs [working]§

Pointer-chasing: an array of pointers shuffled into one giant random cycle, then followed. Every load depends on the previous one, so there is no overlap and no prefetching, and the time per hop is the raw latency of wherever the working set lives.

Predict first: with a 32 KiB L1, 1 MiB L2, and 8 MiB L3, sketch ns-per-load versus working-set size. You should predict a staircase: ~1 ns until 32 KiB, a step past L1, another past L2, then a cliff to tens of ns past L3.

c
/* chase.c - pointer-chase latency ladder. gcc -O2 -o chase chase.c */
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

static uint64_t now_ns(void) {
    struct timespec ts;
    clock_gettime(CLOCK_MONOTONIC, &ts);
    return (uint64_t)ts.tv_sec * 1000000000ull + ts.tv_nsec;
}

int main(void) {
    size_t sizes_kib[] = {16, 32, 64, 256, 1024, 2048,
                          4096, 8192, 16384, 32768, 65536};
    printf("%10s %12s\n", "size_KiB", "ns_per_load");
    for (int s = 0; s < 11; s++) {
        size_t n = sizes_kib[s] * 1024 / sizeof(void *);
        void **ring = malloc(n * sizeof(void *));
        size_t *idx = malloc(n * sizeof(size_t));
        for (size_t i = 0; i < n; i++) idx[i] = i;
        srand(42);                      /* Fisher-Yates: one random cycle */
        for (size_t i = n - 1; i > 0; i--) {
            size_t j = (size_t)rand() % (i + 1);
            size_t t = idx[i]; idx[i] = idx[j]; idx[j] = t;
        }
        for (size_t i = 0; i < n; i++)
            ring[idx[i]] = &ring[idx[(i + 1) % n]];

        void **p = &ring[idx[0]];
        size_t iters = 20 * 1000 * 1000;
        for (size_t i = 0; i < n; i++) p = (void **)*p;   /* warm up */
        uint64_t t0 = now_ns();
        for (size_t i = 0; i < iters; i++) p = (void **)*p;
        uint64_t t1 = now_ns();
        printf("%10zu %12.1f   (p=%p)\n", sizes_kib[s],
               (double)(t1 - t0) / iters, (void *)p);
        free(ring); free(idx);
    }
    return 0;
}

Expected: (shape is the point; absolute numbers vary by CPU)

  size_KiB  ns_per_load
        16          1.2   (p=0x...)
        32          1.3   (p=0x...)
        64          2.9   (p=0x...)
      1024          4.0   (p=0x...)
      4096         12.8   (p=0x...)
     16384         31.5   (p=0x...)
     65536         92.4   (p=0x...)

Match each step against your CPU's cache sizes (lscpu | grep -i cache). The last cliff is also a TLB story: 64 MiB of 4 KiB pages is 16,384 pages, far beyond dTLB reach, so those ~90 ns hops include page-table walks.

Write the plateau numbers down somewhere you will find them again. Most performance intuition worth having runs on those constants, and yours are now measured on the machine you actually use rather than quoted from someone else's table.

Lab 2: translate a virtual address yourself [working]§

/proc/self/pagemap exposes the kernel's answer for every virtual page: one u64 per page, present bit 63, PFN bits 0 to 54. Predict first: will the physical address resemble the virtual one? (No, but the within-page offset must survive translation unchanged.) Run as root; unprivileged readers get PFN 0.

c
/* virt2phys.c - translate one of our own addresses via pagemap.
   gcc -O2 -o virt2phys virt2phys.c ; run as root */
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    long psz = sysconf(_SC_PAGESIZE);
    char *buf = malloc(psz);
    memset(buf, 0xAB, psz);     /* touch it: no touch, no PTE (demand paging) */

    uintptr_t va = (uintptr_t)buf;
    int fd = open("/proc/self/pagemap", O_RDONLY);
    if (fd < 0) { perror("open"); return 1; }
    uint64_t entry;
    if (pread(fd, &entry, 8, (off_t)(va / psz) * 8) != 8) {
        perror("pread"); return 1;
    }
    uint64_t pfn = entry & ((1ull << 55) - 1);
    printf("virtual   = 0x%lx\n", (unsigned long)va);
    printf("present   = %d\n", (int)(entry >> 63) & 1);
    printf("swapped   = %d\n", (int)(entry >> 62) & 1);
    printf("pfn       = 0x%llx\n", (unsigned long long)pfn);
    printf("physical  = 0x%llx\n",
           (unsigned long long)(pfn * psz + va % psz));
    return 0;
}

Expected:

virtual   = 0x7f30a4b2a010
present   = 1
swapped   = 0
pfn       = 0x1c2f43
physical  = 0x1c2f43010

Check: the low 12 bits of virtual and physical agree (010), so the page offset passes through untranslated, exactly as in the walk diagram. Now comment out the memset and rerun: present = 0, pfn = 0. malloc gave you virtual space, but no PTE existed until you touched it. That is demand paging, observed directly from userspace.

Lab 3: dTLB misses, 4 KiB vs huge pages [working]§

A random walk over 1 GiB touches 262,144 distinct 4 KiB pages, and no TLB covers that. With 2 MiB transparent huge pages the same buffer is 512 translations, which fits easily. Predict first: which counter changes, by what factor, and what happens to wall-clock time?

c
/* tlbstress.c - random pokes across 1 GiB; "huge" arg requests THP.
   gcc -O2 -o tlbstress tlbstress.c */
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>

int main(int argc, char **argv) {
    size_t len = 1ull << 30;
    char *m = mmap(NULL, len, PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (m == MAP_FAILED) { perror("mmap"); return 1; }
    if (argc > 1 && !strcmp(argv[1], "huge"))
        madvise(m, len, MADV_HUGEPAGE);          /* hint: back with 2 MiB THP */
    memset(m, 1, len);                            /* populate */
    uint64_t x = 88172645463325252ull, sum = 0;
    size_t npages = len / 4096;
    for (long i = 0; i < 100 * 1000 * 1000; i++) {
        x ^= x << 13; x ^= x >> 7; x ^= x << 17;  /* xorshift64 */
        sum += m[(x % npages) * 4096];            /* one byte per random page */
    }
    printf("%llu\n", (unsigned long long)sum);
    return 0;
}

Confirm THP is available (cat /sys/kernel/mm/transparent_hugepage/enabled must show always or madvise), then:

sh
perf stat -e dTLB-loads,dTLB-load-misses ./tlbstress
perf stat -e dTLB-loads,dTLB-load-misses ./tlbstress huge

Expected: (illustrative magnitudes; counts vary by CPU and kernel)

# 4 KiB pages:
       412,331,907      dTLB-loads
        98,554,201      dTLB-load-misses   # ~24% of loads walk the tables
       4.91 seconds time elapsed

# huge pages:
       409,887,140      dTLB-loads
         1,203,559      dTLB-load-misses   # ~0.3%
       2.87 seconds time elapsed

Two orders of magnitude fewer misses and a real wall-clock win, from changing nothing but translation granularity. Verify the kernel honored the hint: grep AnonHugePages /proc/meminfo during the huge run should show ~1 GiB. This is the whole huge-page argument in one screenful, and it is why the Coconut spec treats huge pages as the default posture for model weights.

Bridge notes [fundamental]§

You already knowNew at this scale
MCU flat physical memory: the linker script pins .text/.data at absolute addresses; any code can write any byteThe MMU inserts one translation on every access: per-process address spaces, per-page permissions, "not present" as a recoverable state
Cortex-M MPU: a few region registers checked on access, no translationPage tables: translation plus protection, millions of fine-grained regions, stored in RAM as a tree instead of registers
Radix tries from compiler and data-structure workThe page-table walk is a 4 to 5 level radix-trie lookup, executed by hardware, on the critical path of every load
Compilers treat memory as uniform once a value spills from registers"Memory" is a four-rung latency ladder (1 ns to 100 ns); layout and access order dominate constant factors more than instruction selection does
Volatile/DMA discipline: you manually track when memory is "really" writtenMESI does it automatically, so correctness is free, but the coherence cost model (line ping-pong, false sharing) becomes your problem
Interrupts as the MCU's only asynchronyTLB shootdown IPIs: cross-core interrupts as routine bookkeeping, and a scalability tax on munmap/mprotect

The one-sentence bridge: on a microcontroller the address is the location; on a server the address is a key into a kernel-owned data structure, and this chapter is the hardware that makes the lookup nearly free, almost all of the time.

You can now take an address out of a debugger and say what has to happen before the byte arrives: which table reads, which cache levels, and which of them the TLB is likely to have skipped. That is the vocabulary Chapter 11's fault path is written in, and it is also the vocabulary of every serious conversation about why a program is slower than it should be. The next time a benchmark surprises you, run Lab 1 on that machine before you argue about the code.

Sources [fundamental]§