Chapter F.5: Reading Assembly Without Fear§

Authors: Shrey Patel and Jay Patel, Coconut Labs
Book: Think Like an OS. See book/00-INDEX.md

This chapter teaches you to read disassembled machine code, never to write it. By the end you will look at a screen of x86-64 assembly and see structure instead of noise: here is the function's entry, there is the loop, that cluster is an array access, and that lone syscall instruction is the exact point where userspace hands control to the kernel. That is the entire skill. It takes about ten instructions of vocabulary and five recognizable shapes, and it is the prerequisite for every crash dump, perf annotate session, and kernel entry-path discussion later in this book. Two kinds of reader need it. If you are heading for kernel work, the syscall entry path in Chapter 7 is a hand-written assembly file with no C version to read instead. If you are heading for low-latency trading, the question "which instruction is hot" is answered by perf annotate, and perf annotate prints assembly.

The problem§

You will meet assembly whether you want to or not. A segfault in production gives you a crash dump with a faulting instruction address. objdump turns any binary into assembly listings, including one you have no source for. perf annotate attributes CPU time to individual instructions, so the answer to "why is this function slow" is written in assembly. And the Linux kernel's syscall entry path, which Part II of this book walks through in detail, is literally a hand-written assembly file (arch/x86/entry/entry_64.S); there is no C version to read instead.

In a modern career you will read assembly hundreds of times and write it approximately never. Compilers write better assembly than humans do, and the few humans who still write it (kernel entry code, cryptography, codecs) are not you and do not need to be. Reading is a far smaller skill than writing. You never need to remember which instruction to use, only recognize the one in front of you. It is the difference between reading a menu in French and writing a novel in French.

Without this chapter, Part II's entry-path walkthrough (Chapter 7) would be a wall of symbols. With it, that walkthrough becomes a guided tour of code you can already parse.

Unfolded§

The thirty-second model§

A CPU has a small set of named storage slots called registers. They are the only local variables the hardware has. On x86-64 the general-purpose ones are 64 bits wide and have names like rax, rbx, rdi, rsi, rsp, plus r8 through r15. An instruction is one tiny operation: copy a value between a register and memory, add two registers, compare, jump. Your compiled program is nothing but a long list of these. Disassembly is the textual rendering of that list, and a disassembler (like objdump) is the tool that produces it. Everything below is about pattern-matching that text.

One historical annoyance to clear immediately: x86 assembly comes in two textual dialects that describe the same instructions.

Two syntaxes, one table, then we pick one§

We use AT&T for the rest of this chapter because it is what Linux tooling hands you by default: objdump emits AT&T by default for x86 (switchable with -M intel), gdb's disassembly-flavor defaults to att, and GCC itself defaults to -masm=att. When a blog post or the Intel manual shows you the other dialect, the table above is the entire translation.

The one genuine AT&T trap: because operands are source-first, cmpq %rcx, %rax reads as "compare rax against rcx". The second operand is the thing being tested. After cmpq B, A, a following jle means "jump if A ≤ B". You will internalize this after the first worked example; until then, consult the table.

The vocabulary: ten instructions cover ninety percent of listings§

Each row gives the instruction as you will actually see it, and a one-line C equivalent.

Two footnotes that prevent confusion. First, lea ("load effective address") looks like a memory access because it uses memory-operand syntax, but it never reads memory. Compilers abuse it as a free multiply-and-add unit (leaq (%rdi,%rdi,8), %rax is rax = rdi * 9). Second, x86-64 registers have 32-bit aliases (eax is the low half of rax, edi of rdi); writing the 32-bit alias zero-fills the upper half, so mov $1, %eax and xor %edi, %edi are compact ways to set the full 64-bit register.

jcc in the table above is the family name for conditional jumps. The cc is a condition code: e equal, ne not equal, l/le signed less / less-or-equal, g/ge signed greater, a/b unsigned above/below.

The five shapes§

You do not read assembly instruction by instruction any more than you read English letter by letter. You recognize shapes.

Shape 1: function prologue and epilogue. At -O0, every function opens and closes the same way:

push   %rbp          # save caller's frame pointer
mov    %rsp,%rbp     # establish our own frame
...body...
pop    %rbp          # restore caller's frame
ret                  # return

rsp is the stack pointer; rbp is the frame pointer anchoring this function's slice of the stack. At -O1 and above the compiler usually drops the frame pointer entirely, so optimized functions often begin with real work on line one. You may also see endbr64 as the very first instruction, a control-flow-protection landing pad emitted when GCC's -fcf-protection is enabled (many distro compilers turn it on by default). It computes nothing; skip it when reading.

Shape 2: the loop triangle. Any backward conditional jump is a loop. The shape is: setup above, body in the middle, a compare-and-jump-back at the bottom.

        init (set counters)
          │
          ▼
   ┌──► .L3:  body
   │      │
   │      step (add/increment)
   │      │
   │      cmp  limit, cursor
   │      jne  .L3   ── backward jump
   └──────┘
          │  falls through when done
          ▼
        after the loop

Shape 3: array indexing. The memory operand (%rdi,%rsi,8) means address rdi + rsi*8: base register, index register, scale. That is a[i] where a is in rdi, i is in rsi, and elements are 8 bytes. Scale is limited to 1, 2, 4, or 8, which is exactly why it exists: those are the sizes of C's scalar types.

Shape 4: struct field access. A constant offset from a base register: movq 8(%rdi), %rax is p->field where field lives 8 bytes into the struct. Struct layout in C is fixed at compile time, so field access is always base-plus-known-constant, with no computation and no lookup.

Shape 5: the switch jump table. A dense switch compiles to: bounds-check the value (cmp + ja to the default case; ja is unsigned-above, which catches negatives too), load an entry from a table indexed by the value, jump through it (jmp *%rax). When you see an indirect jump through a table shortly after a cmp/ja pair, you are looking at a switch.

Worked example 1: ten lines of C, line by line§

The C:

long sum_array(const long *a, long n)
{
    long s = 0;
    for (long i = 0; i < n; i++)
        s += a[i];
    return s;
}

Compiled with gcc -S -O1 (x86-64, GCC 13):

sum_array:
    endbr64                      # CET landing pad, ignore
    testq   %rsi, %rsi           # compare n against zero
    jle     .L4                  # if n <= 0, skip the loop entirely
    movq    %rdi, %rax           # rax = a          (cursor pointer)
    leaq    (%rdi,%rsi,8), %rcx  # rcx = a + n*8    (one-past-the-end)
    movl    $0, %edx             # s = 0            (s lives in rdx)
.L3:                             # ── loop body ──
    addq    (%rax), %rdx         # s += *cursor     (load and add, one instruction)
    addq    $8, %rax             # cursor++         (8 bytes = one long)
    cmpq    %rcx, %rax           # compare cursor against end
    jne     .L3                  # not equal → jump BACK: the loop triangle
.L1:
    movq    %rdx, %rax           # return value goes in rax
    ret
.L4:
    movl    $0, %edx             # n <= 0 path: s = 0
    jmp     .L1

Read the calling convention first: on x86-64 Linux (the System V ABI), the first integer arguments arrive in rdi, rsi, rdx, rcx, r8, r9, and the return value leaves in rax. So %rdi is a and %rsi is n before the first instruction executes.

Now notice what the optimizer did to your variables. There is no i. The compiler replaced the counted loop with a pointer walk: a cursor starting at a, an end pointer at a + n*8 computed once with lea, and a loop that runs until they collide. s lives its whole life in rdx and only moves to rax at the end because the ABI demands the result there. This is the single most important lesson of optimized-code reading: the compiler preserves your program's behavior, not its variables or its line structure. You match shapes (there is exactly one loop, it accumulates, it walks 8 bytes at a time), not source lines.

If you can look at that listing and say "one loop, accumulating, eight bytes a step" without tracing registers, you have the skill this chapter sells. It is the same move that turns a perf annotate screen from a wall of mnemonics into a ranked list of what the CPU actually spent its time on.

Worked example 2: a real syscall site§

Here is objdump -d output for a tiny static binary that writes "hello, kernel\n" to stdout and exits. It is the smallest honest program on Linux:

0000000000401000 <_start>:
  401000:  mov    $0x1,%eax          # rax = 1  → syscall number: write
  401005:  mov    $0x1,%edi          # arg1 (rdi) = 1 → file descriptor: stdout
  40100a:  lea    0xfef(%rip),%rsi   # arg2 (rsi) = address of the message
                                     #   (rip-relative; objdump resolves it: # 402000 <msg>)
  401011:  mov    $0xe,%edx          # arg3 (rdx) = 14 → byte count
  401016:  syscall                   # trap into the kernel
  401018:  mov    $0x3c,%eax         # rax = 60 → syscall number: exit
  40101d:  xor    %edi,%edi          # arg1 = 0 → exit status (the zeroing idiom, live)
  40101f:  syscall

The protocol is fixed by the kernel's ABI, documented in the syscall(2) man page: the syscall number goes in rax (the man page's table writes it as eax, since the kernel reads only the low 32 bits), arguments go in rdi, rsi, rdx, r10, r8, r9, and the syscall instruction transfers control to the kernel, which leaves the return value in rax. The numbers 1 (write) and 60 (exit) come from the kernel's syscall table, which you will meet below.

Note the one deviation from the function-call convention: syscall argument 4 travels in r10 where a function call would use rcx. That is because the syscall instruction itself overwrites rcx (with the userspace return address) and r11 (with the saved flags) as part of the trap. When Chapter 7 walks the kernel side of this instruction, the very first thing the kernel code does is deal with exactly these registers.

ARM64 in 90 seconds§

ARM64 (AArch64) is the other architecture you will meet: every Apple Silicon Mac, most phones, a growing share of servers. Three facts and a table make you literate.

Fact one: ARM64 is a load-store architecture. Arithmetic instructions operate only on registers; memory is touched only by explicit load (ldr) and store (str) instructions. The addq (%rax), %rdx from our loop is impossible on ARM64; it becomes an ldr then an add. Fact two: there are 31 general-purpose registers named x0 to x30 (the AAPCS64 ABI spec calls them r0 to r30; w0 to w30 are their 32-bit views), with x30 serving as the link register, so bl (branch-with-link) puts the return address there instead of on the stack, and ret jumps back through it. The stack pointer sp is a separate register. Fact three: ARM64 assembly uses destination-first operand order, like Intel syntax.

The shapes are identical: prologue, loop triangle, table jump, base-plus-offset. Shape recognition transfers across architectures; only the spelling changes. That transfer is why this chapter is short. You are not learning two instruction sets. You are learning one set of shapes and two spellings for them.

The real thing in Linux§

Everything above cashes out in specific files and tools in the Linux 6.12 tree and its toolchain.

arch/x86/entry/entry_64.S is the hand-written assembly where every 64-bit syscall enters the kernel, at the label entry_SYSCALL_64. The comment block above it is a register-state contract you can now read directly: rax system call number, rcx return address, r11 saved rflags, then rdi, rsi, rdx, r10, r8, r9 as args 0 to 5, exactly the convention from worked example 2, as seen from the receiving end. The first real instruction is swapgs, which swaps in the kernel's per-CPU base pointer; Chapter 7 explains why.

arch/x86/entry/syscalls/syscall_64.tbl is the syscall number table, a plain text file mapping numbers to handlers: 0 common read sys_read, 1 common write sys_write, 39 common getpid sys_getpid, 60 common exit sys_exit. The $0x1 and $0x3c in our disassembly are rows in this file. (Coconut's fork extends this same file; see the tie-in below.)

Crash output. A kernel oops prints the faulting instruction pointer and raw code bytes; scripts/decode_stacktrace.sh in the kernel tree translates stack-dump addresses back to source files and line numbers. When the bug is in generated code, the disassembly is the ground truth, and you read it with exactly this chapter's vocabulary.

perf annotate reads a perf.data profile and displays annotated disassembly, each instruction tagged with the percentage of CPU samples that landed on it. It is the standard answer to "which instruction is hot", and it is useless to someone who cannot read the listing.

gdb disassembles with the disassemble command, AT&T flavor by default (set disassembly-flavor intel if you must), and shows you the instruction that took the segfault.

Coconut tie-in§

Coconut's agent_spawn (472) and agent_attest (473) are rows in the same syscall_64.tbl you just learned to read, and their userspace call sites look exactly like worked example 2 with a different number in rax. When the 04-HLD entry-path work lands, reviewing it means reading entry_64.S-style assembly diffs, and holding the cap-token lookup to its performance budget means sitting in perf annotate output. This chapter is the literacy those tasks assume.

Lab§

Two parts. Part A runs in any browser on your macOS host with nothing to install; Part B runs in a Docker Linux container. Each exercise is predict-then-check: commit to a prediction in writing before you look. The written prediction is the point, because the gap between it and the listing is the thing you actually learn.

Part A: Compiler Explorer (browser)§

Open godbolt.org (Compiler Explorer, the interactive lab bench for this entire skill; started in 2012 precisely to show how C++ constructs translate to assembly). Select language C, compiler x86-64 gcc, and put -O1 -masm=att in the compiler options box (-masm=att forces the AT&T dialect this chapter uses; the site otherwise tends to show Intel).

Exercise 1: the cost of -O0. Type:

int add3(int x) { return x + 3; }

Predict: how many instructions at -O0? At -O2? Then check both.

Expected: at -O0, six-plus instructions including a full prologue and a pointless round-trip of x through the stack (movl %edi,-4(%rbp) then reload). At -O2:

add3:
    leal    3(%rdi), %eax
    ret

Two instructions, and the addition is done by lea. This is why we read optimized code at -O1/-O2: -O0 output is honest but bloated. (-O0 is GCC's default. -O2 applies "nearly all supported optimizations that do not involve a space-speed tradeoff", per the GCC manual, which is also why -O2 output can look scrambled: inlining, unrolling, and reordering dissolve your source structure. -O1 is the sweet spot for learning.)

Exercise 2: multiply without multiplying. long times9(long x) { return x * 9; } at -O1. Predict: imul? Check.

Expected: leaq (%rdi,%rdi,8), %rax, which is base + index×8, i.e. x + 8x. No multiply instruction at all.

Exercise 3: struct offsets.

struct conn { int id; int state; long bytes; };
long get_bytes(struct conn *c) { return c->bytes; }

Predict the byte offset in the load (count the fields: two 4-byte ints, then the long). Check.

Expected: movq 8(%rdi), %rax. Now swap bytes to be the first field and predict the new offset before recompiling.

Exercise 4: the loop triangle. Paste sum_array from worked example 1 at -O1. Find the backward jump; identify which register is the cursor, which is the limit, and which is the accumulator, using only the shapes. Then switch to -O2 and watch the same loop get restructured. Confirm you can still find the triangle.

Exercise 5: the jump table. A six-case dense switch on an int (cases 0 to 5, distinct return values, plus a default), at -O1. Predict: chain of compares, or table? Check.

Expected shape: cmpl $5, %edi then ja to the default case, a leaq of a table address, an indexed load, and an indirect jmp *%rax. (You may see notrack decorating the jump, which is more control-flow-protection noise; skip it.)

Part B: objdump a real binary (Docker)§

On your macOS host (on Apple Silicon, keep the --platform flag so you get x86-64 output; on an Intel Mac you can drop it):

docker run --platform linux/amd64 --rm -it ubuntu:24.04 bash

Inside the container:

apt-get update && apt-get install -y gcc binutils
cat > hello.c <<'EOF'
#include <stdio.h>
int main(void)
{
    printf("hello\n");
    return 0;
}
EOF
gcc -O0 -o hello hello.c

Predict, in writing: the first two working instructions of main (this is Shape 1). Then:

objdump -d --no-show-raw-insn hello | grep -A 10 '<main>:'

Expected:

0000000000001149 <main>:
    1149:  endbr64
    114d:  push   %rbp
    114e:  mov    %rsp,%rbp
    1151:  lea    0xeac(%rip),%rax        # 2004 <_IO_stdin_used+0x4>
    1158:  mov    %rax,%rdi
    115b:  call   1050 <puts@plt>
    1160:  mov    $0x0,%eax
    1165:  pop    %rbp
    1166:  ret

Score yourself: prologue after the endbr64 pad, string address into rdi (argument 1), and then a surprise. It is call puts@plt, not printf: GCC noticed your format string had no conversions and swapped in the cheaper function. Also note mov $0x0,%eax: your return 0 placing the result in rax.

Bonus: recreate worked example 2 for real. Save the eight-line _start listing from that example as hello.s (with the .global _start / .text header and the msg data section), then:

gcc -nostdlib -static -o hello2 hello.s
./hello2
objdump -d hello2

Expected: hello, kernel on stdout, and a disassembly matching worked example 2. You have now watched a write(2) happen at the instruction level.

That is the whole skill, in a form you can check on yourself. Handed any listing, you can find the entry, find the loop, name the syscall, and say which register is carrying which argument. It pays first in Chapter 7, where the kernel side of that syscall instruction is the subject rather than the punchline, and after that in every perf annotate screen you open on code of your own.

Bridge notes§

If you have compiler-design and microprocessor coursework, most of this chapter is a dialect refresher, not new material.

Skim: the thirty-second model, the vocabulary table (you know what mov/cmp/call are), the five shapes as concepts. Your 8051/AVR/MIPS coursework already built shape recognition, and that is the transferable skill.

Actually new muscles: (1) AT&T syntax: source-first operand order and the %/$ sigils invert what Intel-manual-trained eyes expect, and the cmp operand order will bite you once; (2) x86's compound memory operands, where 8(%rdi,%rsi,4) folds an address computation into one operand and your MIPS training expects separate lw/sw with simple offsets (ironically, ARM64's load-store discipline will feel more familiar than x86); (3) the x86-64 System V register conventions (rdi rsi rdx rcx r8 r9 / return in rax) and the syscall variant with r10, where coursework ISAs had different and usually simpler conventions; (4) most importantly, optimizer-output reading: coursework disassembly was -O0-style code that mirrored the source. Real binaries are -O2: variables evaporate, loops become pointer walks, multiplies become lea. Worked example 1 and Lab exercises 1 and 4 are the reps that build this; do not skip them even if everything else here bored you.

Sources§

• https://man7.org/linux/man-pages/man2/syscall.2.html: Linux syscall calling conventions: x86-64 (syscall, number in rax, args rdi/rsi/rdx/r10/r8/r9, return rax) and arm64 (svc #0, number in w8, args x0 to x5, return x0).
• https://man7.org/linux/man-pages/man1/objdump.1.html: -d, --no-show-raw-insn, AT&T as default x86 dialect, -M intel/-M att.
• https://sourceware.org/gdb/current/onlinedocs/gdb.html/Machine-Code.html: gdb set disassembly-flavor, default att.
• https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html: -S stops after compilation proper and emits a .s assembler file.
• https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html: -O0 is the default; -O2 performs "nearly all supported optimizations that do not involve a space-speed tradeoff".
• https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html: -masm=dialect, choices att/intel, default att; ENDBR insertion tied to -fcf-protection=branch.
• https://elixir.bootlin.com/linux/v6.12/source/arch/x86/entry/entry_64.S: entry_SYSCALL_64 and its register-contract comment (rax number, rcx return address, r11 saved rflags, rdi…r9 args); swapgs at entry. Content verified against https://raw.githubusercontent.com/torvalds/linux/v6.12/arch/x86/entry/entry_64.S.
• https://elixir.bootlin.com/linux/v6.12/source/arch/x86/entry/syscalls/syscall_64.tbl: syscall numbers: read 0, write 1, getpid 39, exit 60. Content verified against https://raw.githubusercontent.com/torvalds/linux/v6.12/arch/x86/entry/syscalls/syscall_64.tbl.
• https://elixir.bootlin.com/linux/v6.12/source/scripts/decode_stacktrace.sh: stack-dump address → file:line translation script in the 6.12 tree.
• https://man7.org/linux/man-pages/man1/perf-annotate.1.html: perf annotate reads perf.data and displays annotated code/disassembly.
• https://github.com/compiler-explorer/compiler-explorer: Compiler Explorer started 2012; godbolt.org is the primary instance; 30+ languages.
• https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst: AAPCS64: 31 general-purpose registers r0 to r30 (x/w views), r30 as link register, SP separate, parameter registers x0 to x7.