Chapter F.2: Anatomy of a Program§

Authors: Shrey Patel and Jay Patel, Coconut Labs
Book: Think Like an OS (see book/00-INDEX.md)

You type gcc hello.c and get a file you can run. This chapter unfolds everything hidden inside that one command: the four distinct programs that run in sequence (preprocessor, compiler, assembler, linker) and what each one's output actually contains; what a CPU register is and the names of every register you will meet in this book; what the stack physically is and what a function call compiles down to; the System V AMD64 calling convention that every Linux x86_64 binary obeys; and the named sections (.text, .data, .bss, .rodata) where your program's bytes live. By the end you can take a ten-line C file, stop the pipeline at every stage, read each intermediate file, and watch a live stack frame in a debugger. That is the exact skill set the rest of this book assumes.

The problem§

If your background is web, data, or mobile engineering, you have almost certainly shipped software without ever seeing a register or an object file. Your toolchain (npm run build, pip install, Xcode's Run button) collapses the entire journey from source code to executing machine code into one opaque step. That opacity is fine right up until you open this book.

Later chapters break without this one. When Chapter 7 says "on syscall entry the kernel saves your registers into struct pt_regs," that sentence is noise unless you know what a register is and which ones exist. When the scheduling chapter says "a context switch swaps the stack pointer," you need to know what the stack pointer points at. When Chapter 24 dissects the dynamic linker, you need to already know what linking is. And every debugging session in this book, kernel or userspace, eventually lands you in a disassembly listing with a backtrace, which is unreadable without the calling convention and the stack-frame layout in your head.

The paragraph below that explains why optimized builds drop the frame pointer is the same paragraph that explains why a profiler sometimes hands you a truncated stack. That is the shape of this chapter: small facts about registers and frames that decide whether your tools can tell you anything at all. Kernel work needs them because struct pt_regs and every syscall-entry listing in Part II are written in this vocabulary. Low-latency and trading work needs them because "what did the compiler actually emit for this function" is a question you will ask weekly, and answering it means reading a disassembly listing and knowing which register held what.

So this chapter is the prerequisite layer: the anatomy of an ordinary program, from text file to executing instructions, with nothing skipped.

Unfolded§

The pipeline at a glance§

gcc is not a compiler. It is a driver, a small program whose job is to run other programs in the right order. The GCC manual states it plainly. Compilation can involve up to four stages, always in that order: preprocessing, compilation proper, assembly, and linking. The driver lets you stop at any intermediate stage.

 hello.c ──► [1. preprocess] ──► hello.i ──► [2. compile] ──► hello.s
 (C source)     cpp / cc1        (still C,      cc1           (assembly
                                  expanded)                     text)
                                                                  │
                                                                  ▼
 hello    ◄── [4. link] ◄─────── hello.o ◄──── [3. assemble] ─────┘
 (runnable     ld (via            (machine          as
  executable)  collect2)           code with
                                   holes)

Each stage is a real program you can invoke, and each stop point has a flag:

StageToolStop flagInputOutputWhat it does
1. Preprocesscppgcc -E.c text.i text (still C)Textual expansion: #include, #define, comments removed
2. Compile propercc1gcc -Spreprocessed C.s assembly textTranslates C into human-readable CPU instructions
3. Assembleasgcc -c.s assembly.o object fileEncodes instructions into machine-code bytes, leaves "holes"
4. Linkld(default).o files + librariesexecutableResolves symbols, patches holes, lays out the final file

Run gcc -v hello.c and the driver prints, per its documentation, "the commands executed to run the stages of compilation." You will see it launch cc1 (the compiler proper, which on modern GCC also runs the preprocessor internally), then as, then collect2. That last one is a GCC utility that arranges for initialization functions to run at program start; per GCC's internals documentation, collect2 is installed in place of ld in the compiler's directory and forwards the real work to the actual linker.

Now each stage, unfolded.

Stage 1: preprocessing, pure text surgery§

The C preprocessor (cpp, per its man page, "a macro processor that is used automatically by the C compiler to transform your program before compilation") knows nothing about C semantics. It performs textual edits: every #include <stdio.h> is replaced by the full text of that header (which itself includes other headers, recursively), every #define macro is expanded, comments are stripped, and #if/#ifdef blocks are kept or deleted.

The output, conventionally a .i file, is still C source code. It is enormous. A ten-line program that includes <stdio.h> becomes tens of thousands of lines, because the entire declaration surface of the C library rode in on that one #include. #include is not "import a module." It is "paste a file here."

Stage 2: compilation proper, C becomes assembly§

gcc -S stops after the compiler proper and leaves a .s file: assembly language, which is the human-readable spelling of the CPU's instruction set. Each line is (roughly) one CPU instruction: move this value into that register, add these two registers, jump to that label. This is the stage where all the intelligence lives: parsing, type checking, optimization, register allocation. Everything after it is mechanical.

One notational note before you read any .s file on Linux: GNU tools use AT&T syntax for x86 by default (the GNU assembler documentation describes Intel syntax as the mode you must explicitly switch on). In AT&T syntax, register names carry a % prefix, constants carry $, and operands are written source first, destination second, so movq %rsp, %rbp copies rsp into rbp. Intel-syntax listings (common in Windows-centric material) reverse the operand order. This book uses AT&T because it is what Linux toolchains print.

Interlude: registers, from zero§

You cannot read that .s file without knowing what a register is, so here is the ground floor.

A register is a small, fixed-size storage slot physically inside the CPU, not in RAM. On a 64-bit machine, each general-purpose register holds 64 bits (8 bytes). Registers have names, not addresses. They are the fastest storage in the machine, and they are scarce. Arithmetic happens in registers, so the compiler's job is largely a logistics problem: shuttling values between slow, huge memory and fast, tiny registers.

The System V AMD64 ABI document, the rulebook for Linux on x86_64, states that the AMD64 architecture provides 16 general-purpose 64-bit registers. We meet it properly in a moment. Here they are, with their conventional roles:

RegisterHistorical nameRole under the SysV ABI
raxaccumulator1st return value; otherwise scratch
rbxbasecallee-saved (preserved across calls)
rcxcounter4th integer argument
rdxdata3rd argument; 2nd return register
rsisource index2nd argument
rdidestination index1st argument
rbpbase pointercallee-saved; conventionally the frame pointer
rspstack pointertop of the stack (hardware-assisted)
r8-r9none5th and 6th arguments
r10-r11nonescratch (temporary) registers
r12-r15nonecallee-saved

The odd names (rax, rsi...) are fossils: they date to the 8086's 16-bit ax, si and friends, widened to eax (32-bit) and then rax (64-bit). The r8 to r15 set was added when AMD designed the 64-bit extension, which is why those eight get boring numeric names.

Two special registers sit outside the general-purpose set:

For contrast, ARM64 (the architecture inside every Apple Silicon Mac and most phones) at a glance: it has thirty-one general-purpose 64-bit registers named x0 to x30, plus a dedicated stack pointer sp and a program counter pc that is not part of the general-purpose set. Per the AAPCS64 (ARM's official procedure-call standard): x0 to x7 carry arguments and results, x19 to x28 are callee-saved, x29 is the frame pointer, and x30 is the link register. On ARM64, a function call puts the return address in a register rather than pushing it to memory. Same concepts as x86_64, different bookkeeping. This book's listings are x86_64; the mapping transfers.

The stack, unfolded§

The stack is not special hardware. It is an ordinary region of the process's memory plus one convention: the rsp register always holds the address of the current "top." On x86_64 (per the SysV ABI) the stack grows downward, toward numerically lower addresses, so "pushing" makes rsp smaller.

Two instructions maintain it:

That is the entire mechanism. Frames, backtraces, "stack overflow": everything else is convention layered on those two moves.

Function calls are built directly on it. Per Intel's instruction definitions:

So "return address" is not metaphor. It is literally an 8-byte code address sitting in stack memory, placed there by call, consumed by ret. (This mechanical fact is also why memory-corruption exploits love the stack: overwrite that saved address and ret jumps wherever the attacker wrote. Later chapters build on this.)

A stack frame is the slice of stack a single active function call owns: its saved registers, its local variables, its scratch space. The classic frame layout uses rbp as a frame pointer, a fixed anchor into the current frame, so locals live at constant negative offsets from rbp while rsp is free to move. A typical function prologue is:

asm
push  %rbp          # save the caller's frame pointer
movq  %rsp, %rbp    # anchor: my frame starts here
subq  $16, %rsp     # carve out 16 bytes for my locals

and the epilogue undoes it. The SysV ABI's frame diagram pins down the resulting layout exactly: with rbp as frame pointer, 0(%rbp) holds the previous rbp value and 8(%rbp) holds the return address. Locals sit below, at -8(%rbp), -16(%rbp), and so on. (Optimized code often drops the frame pointer and addresses everything relative to rsp, which the ABI explicitly permits, but debuggers and this book start with the rbp discipline because it makes stacks legible.) That omission has a cost you may already have paid: with no rbp chain left to follow, a profiler or debugger has to fall back on other unwind information, and when that is missing the stack it shows you is truncated.

Here is a live stack with two frames, main() having called add(3, 4), with addresses shrinking downward:

  higher addresses
 ┌─────────────────────────────┐
 │  main's caller's frame ...  │
 ├─────────────────────────────┤ ─┐
 │  return address into libc   │  │
 │  saved rbp of caller        │◄─┼── main's rbp points here
 │  main's locals              │  │   main's frame
 │  (alignment / scratch)      │  │
 ├─────────────────────────────┤ ─┤
 │  return addr into main ─────┼──┼── pushed by `call add`;
 │                             │  │   points at the instruction
 │                             │  │   right after the call
 │  saved rbp of main          │◄─┼── add's rbp points here
 │  add's locals: sum          │  │   add's frame     ◄── 8(%rbp) = ret addr
 │                             │  │                   ◄── 0(%rbp) = saved rbp
 ├─────────────────────────────┤ ─┘                   ◄── -8(%rbp) = sum
 │            (free)           │◄──── rsp: the current top
 └─────────────────────────────┘
  lower addresses          stack grows ↓

A debugger's backtrace is a walk of this chain: follow saved-rbp links upward, reading the return address next to each one.

One more ABI fact worth knowing early: the 128 bytes below rsp are the red zone. The SysV ABI reserves this area, and signal and interrupt handlers must not clobber it, so a leaf function (one that calls nothing) may use it for its entire frame without ever adjusting rsp. Hold that thought for the Linux section below, because the kernel pointedly refuses to play this game.

The System V AMD64 calling convention: the contract§

When main calls add(3, 4), both sides must agree on where 3 and 4 travel and where the result comes back. That agreement is the calling convention, and on Linux x86_64 it is defined by the System V AMD64 ABI (Application Binary Interface) supplement. The rules you will use constantly:

ClassRegistersMeaning
Argumentrdi rsi rdx rcx r8 r9first six integer/pointer args, in this order
Returnrax (+rdx)function results
Callee-savedrbx rbp r12 r13 r14 r15 rspcalled function must preserve
Caller-savedrax rcx rdx rsi rdi r8 r9 r10 r11caller must assume they are clobbered

Worked example. Given:

c
long add(long a, long b) { long sum = a + b; return sum; }

unoptimized GCC emits assembly shaped like this (representative -O0 output; exact offsets vary by version):

asm
add:
    pushq   %rbp              # prologue: save caller's frame pointer
    movq    %rsp, %rbp        #   and anchor my frame
    movq    %rdi, -24(%rbp)   # spill argument a (arrived in rdi) to my frame
    movq    %rsi, -32(%rbp)   # spill argument b (arrived in rsi)
    movq    -24(%rbp), %rdx   # reload a
    movq    -32(%rbp), %rax   # reload b
    addq    %rdx, %rax        # rax = a + b
    movq    %rax, -8(%rbp)    # store into local `sum`
    movq    -8(%rbp), %rax    # return value goes in rax
    popq    %rbp              # epilogue: restore caller's frame pointer
    ret                       # pop return address into rip

Every convention above is visible: arguments landed in rdi/rsi, locals live at negative rbp offsets, the result leaves in rax, and ret consumes the address call pushed. (The redundant spilling and reloading is what -O0 looks like; at -O2 this whole function becomes leaq (%rdi,%rsi), %rax; ret-shaped.)

If you had to read that listing twice, that is the normal shape of it, and it stays that way for people who read assembly professionally. What makes it stop being slow is not more staring. It is the lab below, where you break on add and watch rdi actually hold 42.

Stage 3: assembling, text becomes bytes with holes§

The assembler (as, driven by gcc -c) translates each assembly line into its binary encoding and packages the result as an object file (.o), on Linux an ELF relocatable file. Two things make a .o more than "just the bytes":

  1. A symbol table: the names this file defines (add, main) and the names it uses but does not define (printf).
  2. Relocations: placeholder holes in the machine code. When main calls printf, the assembler cannot know printf's address, so it emits the call instruction with a dummy target and records a relocation entry saying "patch these 4 bytes with printf's real location later." objdump -d disassembles the code (the man page: it "display[s] the assembler mnemonics for the machine instructions"), and adding -r prints the relocation entries interspersed with that disassembly, so you can see the holes.

Stage 4: linking, resolving the holes§

The linker (ld, invoked through collect2) takes all the object files plus the C runtime startup objects and libraries, and produces the executable. Conceptually it does three jobs: symbol resolution (match every "I need printf" against exactly one "I provide printf"), layout (merge every input's .text into one output .text, likewise .data and the rest, assigning final addresses), and relocation patching (go back and fill every hole with the now-known addresses).

Static vs dynamic linking, at concept level only (Chapter 24 is the deep dive): static linking copies the needed library code into your executable at link time. The result is self-contained, larger, frozen. Dynamic linking instead records a dependency note ("this program needs libc.so.6") and defers resolution to program launch, when the dynamic linker maps the shared library into memory and patches the last holes. Nearly everything on a modern Linux system is dynamically linked; that is why your 10-line hello-world binary is kilobytes, not megabytes.

Sections: where your bytes live§

Inside object files and executables, bytes are organized into named sections, each with a job. The four you must know, as defined in the elf(5) man page:

SectionContainsWritable?Occupies file bytes?
.textexecutable instructionsno (read + execute)yes
.rodataread-only data (string literals, const tables)noyes
.datainitialized global/static variablesyesyes
.bsszero-initialized global/static variablesyesno

The .bss row is the interesting one. Per elf(5), .bss "holds uninitialized data," the system "initializes the data with zeros when the program begins to run," and the section is of type SHT_NOBITS, a type that "occupies no space in the file." Think about why this works. If a variable's initial value is zero, the file does not need to store a million zero bytes. It only needs to store one number, how many zeros, and let the program loader allocate that much zeroed memory at startup. Declare long big[1000000]; at file scope and your executable grows by roughly nothing, while your process's memory image grows by 8 MB. You will verify this yourself in the lab.

The real thing in Linux§

Everything above has a concrete address in the Linux 6.12 tree and its toolchain:

Coconut tie-in§

Coconut's kernel work lives exactly at the layer this chapter teaches. The agent_* syscall family (472 to 479, per 04-HLD) is ultimately a register-level contract: arguments arriving in the very registers tabled above, snapshotted into struct pt_regs at entry. And reviewing diffs to kernel/agent/ or the security/coconut/ cred shim routinely means confirming what the compiler actually emitted, which is objdump -d fluency straight from this chapter's lab.

Lab§

Environment: Docker Linux container. On any Mac (or Linux host), run the container as linux/amd64 so the output matches this chapter's x86_64 listings. On Apple Silicon that means emulation, which is fine for compiling and disassembling. One caveat: the debugger step uses ptrace, which can be unreliable under emulation on Apple Silicon; if gdb misbehaves there, do step 6 natively on macOS with clang/lldb instead. You will see ARM64 registers, x0/x1 in place of rdi/rsi, and the concept-map table at the end translates.

1. Start the container and install the debugger.

sh
mkdir -p ~/anatomy-lab && cd ~/anatomy-lab
docker run --platform linux/amd64 --rm -it -v "$PWD":/work -w /work gcc:14 bash
# inside the container:
apt-get update && apt-get install -y gdb

2. Write the specimen. anatomy.c is ten lines, one variable per section plus a two-function call chain:

c
#include <stdio.h>

long big_zeroes[1000000];              /* zero-initialized  -> .bss   */
long answer = 42;                      /* initialized       -> .data  */
static const char tag[] = "anatomy";   /* read-only         -> .rodata*/

long add(long a, long b) { long sum = a + b; return sum; }

int main(void) {
    printf("%s: %ld\n", tag, add(answer, big_zeroes[0]));
    return 0;
}

3. Stage 1: preprocess. Predict first: the source is 11 lines; how many lines is the preprocessed output? Write your guess down.

sh
gcc -E anatomy.c -o anatomy.i
wc -l anatomy.c anatomy.i
tail -15 anatomy.i

Expected: anatomy.i is tens of thousands of lines (the entire expanded <stdio.h> chain), yet the tail is your own code, nearly verbatim. That is proof that preprocessing is paste-and-expand, nothing more. If you predicted "a few hundred lines," recalibrate: one #include dragged in the C library's whole declaration surface.

4. Stage 2: compile to assembly.

sh
gcc -S anatomy.c -o anatomy.s
cat anatomy.s

Expected: AT&T-syntax assembly. Find (a) add: followed by a pushq %rbp / movq %rsp, %rbp prologue and spills of %rdi/%rsi into negative %rbp offsets; (b) in main, values loaded into %rdi and %rsi before call add, and the result used out of %rax; (c) directives like .text, .data, .bss, .section .rodata sorting your three globals; (d) the string "anatomy" under the rodata section.

5. Stage 3: assemble, then read the object file. Predict first: big_zeroes is 8,000,000 bytes of zeros. Will anatomy.o be roughly 8 MB or roughly 2 KB?

sh
gcc -c anatomy.c -o anatomy.o
ls -l anatomy.o
size anatomy.o
objdump -dr anatomy.o | less

Expected: the file is a few KB. size reports bss around 8000000 with a tiny data and text. The zeros exist as a length, not as bytes (that is SHT_NOBITS doing its job). In the disassembly, the call to printf has a placeholder target with a relocation line (type R_X86_64_PLT32, symbol printf) interspersed by -r, a hole waiting for the linker.

6. Stage 4: link, watching the driver work.

sh
gcc -v anatomy.o -o anatomy 2>&1 | grep -E 'collect2|ld' | head -3
./anatomy

Expected: a collect2 command line (the wrapper that fronts ld) listing your .o alongside C runtime objects (crt*.o) and -lc. Then the program prints anatomy: 42. Re-run the disassembly on the final executable with objdump -d anatomy | grep -A6 '<main>:', and the call targets are now real addresses (dynamic-linking machinery for printf, resolved fully at run time; Chapter 24 owns those details).

7. Watch a live stack frame. Predict first: when the debugger stops at the top of add, what value is in rdi? In rsi? (Read the call site in main again.)

sh
gcc -g -O0 anatomy.c -o anatomy_dbg
gdb ./anatomy_dbg

Inside gdb:

(gdb) break add
(gdb) run
(gdb) backtrace
(gdb) info registers rdi rsi rsp rbp rip
(gdb) x/2gx $rbp
(gdb) disassemble main

Expected: backtrace shows two frames, add above main, the frame chain from this chapter's diagram walked for you. rdi is 42 (answer) and rsi is 0 (big_zeroes[0]): the SysV argument registers, live. x/2gx $rbp prints two 8-byte values, the saved rbp of main at 0(%rbp) and the return address at 8(%rbp). Compare that address against the disassemble main listing and it lands on the instruction immediately after call add, exactly as the call/ret semantics promise. If your prediction for rdi/rsi was wrong, re-read the argument-register table and check which argument position each variable occupied.

macOS-native alternative for step 7 (Apple Silicon, clang/lldb): same experiment, ARM64 dialect.

lldb ./anatomy_dbg
(lldb) b add
(lldb) run
(lldb) bt
(lldb) register read x0 x1 sp fp lr

Expected: x0 = 42 and x1 = 0 (AAPCS64 puts the first two arguments in x0/x1), and lr holds the return address into main, in a register rather than only on the stack, which is ARM64's link-register twist.

Conceptx86_64ARM64
First two argsrdi, rsix0, x1
Return valueraxx0
Stack pointerrspsp
Frame pointerrbpx29 (fp)
Return addresspushed by callplaced in x30 (lr)

Seven steps in, you can stop gcc at any of its four stages, read what came out of each one, and point at the exact register an argument travelled in. Part II's syscall-entry chapters are written against that picture, and so is every disassembly you will squint at while chasing a stall in code you own.

Bridge notes§

If you have compiler and microprocessor coursework, most of this chapter is a fast lane: skim the four-stage pipeline, sections, and the concept of relocation, all of which you built in class. What is genuinely worth your time here: (1) the SysV AMD64 specifics as recall-level knowledge, meaning the rdi rsi rdx rcx r8 r9 order, the exact callee-saved set (rbx rbp r12-r15), the rsp+8 ≡ 0 (mod 16) entry condition, and the 128-byte red zone plus the kernel's -mno-red-zone refusal, because kernel reviews in later chapters assume you can spot violations on sight; (2) objdump -dr and debugger fluency from the lab, which coursework rarely drills; (3) the correspondence between this chapter's register table and struct pt_regs in arch/x86/include/asm/ptrace.h, the pivot on which every syscall-entry and context-switch discussion in Parts 1 and 2 turns.

Sources§