An Introduction to Programming using entity-component-systems & existence-based processing in rust

written by Bjorn Madsen updated: 2026-06-21

Read online: Codeberg · GitHub Pages

Clone source (the public default branch is the rendered book; the runnable code lives on main):

git clone --branch main https://codeberg.org/root-11/intro-book.git
or
git clone --branch main https://github.com/root-11/intro-book.git

Issues: Codeberg · GitHub

This book teaches programming from first principles of data-oriented design, entity-component-systems (ECS), and existence-based processing (EBP). It assumes no prior programming experience and uses Rust as the only language.

The book is structured around forty-three concepts (the DAG) and their canonical wording (the glossary). Sections are short - two to three pages of prose followed by four to twelve compounding exercises. Concepts are named only after they are built: every section earns its vocabulary through working code, not the other way around.

The through-line is a small ecosystem simulator built in stages from one hundred wandering creatures to a hundred million streamed ones. The simulator’s specification is at code/sim/SPEC.md.

This is a work in progress. Section ordering is by the DAG; reading order can be linear (front to back) or by following the cross-links wherever they lead.

Who this book is for

You want to build something. You are either coming to programming from another field, or you tried it before and found that what got taught did not match what you wanted to make. You can read code; you may have written some; you have not been bitten enough to feel that programming is yours yet.

The book is for people who learn by building artifacts and want technical depth that compounds - where each new idea makes the previous one more useful, not just adds another tool to a pile. The through-line is a small ecosystem simulator that grows from a hundred creatures to a hundred million; everything you learn earns its keep on that one program, then transfers everywhere else.

It is not aimed at the median CRUD-application job market. If your goal is “any programming job, fastest,” there are faster paths. If your goal is “the kind of programmer whose programs work,” this is one of them.

Background

You should be comfortable with high-school algebra and a command line - running a command, changing directories, reading error messages without panic. A laptop with internet is enough for the first ten sections; for the rest, you will install a Rust toolchain locally.

You do not need prior programming experience, calculus, a maths degree, or any prior contact with Rust. The book teaches Rust syntax as each section needs it; the language is a vehicle, not the subject.

A first taste

Before any vocabulary is named, here is what an ECS world looks like in fifteen lines of Rust. One hundred creatures, each with a position and a velocity, moving for thirty ticks of simulated time. No structs, no traits, no libraries - four Vecs indexed in lockstep, and a function (the for i in 0..x.len() loop) that advances every creature one step.

fn main() {
    let mut x:  Vec<f32> = (0..100).map(|i| (i as f32) * 0.1).collect();
    let mut y:  Vec<f32> = (0..100).map(|i| (i as f32).sin()).collect();
    let     vx: Vec<f32> = (0..100).map(|i| ((i * 7) % 11) as f32 * 0.01 - 0.05).collect();
    let     vy: Vec<f32> = (0..100).map(|i| ((i * 13) % 7) as f32 * 0.01 - 0.03).collect();

    for tick in 0..30 {
        for i in 0..x.len() {
            x[i] += vx[i];
            y[i] += vy[i];
        }
        if tick % 10 == 0 {
            println!("tick {tick}: creature 17 at ({:.2}, {:.2})", x[17], y[17]);
        }
    }
}

Click play. The simulator runs in your browser, prints three lines, and stops. That is the entire shape of what the rest of the book grows: tables (the Vecs), a tick (the outer loop), a system (the inner loop). Everything that follows is the discipline that lets this same shape carry a hundred million creatures without falling apart.

Running the code

Most code blocks in the early chapters have a play button that runs the code in your browser via the Rust Playground. Click it, edit, see the result. No setup required. The deck-game exercises in §5, §9, and §10 are designed to be run this way - open the page, hit play, work through the exercises in the editor that appears.

From the simulator chapters onward, the exercises stop being self-contained snippets. They build the through-line: a working Rust program that grows from one hundred wandering creatures to a hundred million streamed ones. Running them needs a local Rust toolchain, a project that holds state between runs, and the ability to time loops on your own hardware. By that point you will want a clone of the book’s repo:

git clone https://codeberg.org/root-11/intro-book.git
cd intro-book
cargo run --release --bin sim

For the timing exercises in §1, the play button works but the numbers it produces are not yours - they come from a shared server the playground happens to be running on. The exercise asks “how fast does your machine run this?”, and that question only has a real answer locally. Click play for a first taste; then run on your own hardware for the numbers the rest of the book references.

The threshold between playground and local is fuzzy by intent. A reader on a phone or in a classroom can stay in the browser through §10. Beyond that, treat a local toolchain as part of the curriculum.

The companion edition

If you want to read the same book in a slow language and see what discipline must replace what the type system here enforces for you, the Python edition (codeberg,github) covers the same forty-four sections in Python and numpy. The architecture is identical; the language differs. Many readers find Python a useful contrast: every borrow-check error here is a runtime mistake there, and the per-chapter Python commentary names the cost.

Nomenclature

Quick reference for symbols, notation, and abbreviations the book uses. Concept definitions live in the glossary; this page covers the shorthand only.

Symbols

Text formatting

Variables you will see across chapters

Rust types used in code

Abbreviations

EBP is this book’s shorthand. The spelled-out term - existence-based processing - is Richard Fabian’s, from Data-Oriented Design; §17 introduces it from the simulator. An acronym index will not list “EBP” because the source literature spells the term out rather than abbreviating it.

Naming in code

• snake_case for variables, functions, fields.
• PascalCase for types and traits.
• SCREAMING_SNAKE for constants.
• File names mirror their dominant content: creatures.rs defines the creature table, motion.rs the motion system.

1 - The machine model

Concept node: see the DAG and glossary entry 1.

Most explanations of “how a computer works” use a diagram with a CPU and a single big block called memory. The diagram is wrong. Memory is many things at different speeds, and which one your data sits in decides whether your program is fast or slow.

Inside the CPU there is L1 cache - small, sometimes only 32 KB per core, but a read from it costs about one nanosecond. Around it sits L2 - a few hundred KB, around 3-4 ns. Then L3 - measured in megabytes, around 10 ns. Outside the CPU sits main memory (RAM) - gigabytes, around 100 ns per read. The numbers vary by chip, but the ratios are stable: L1 is roughly a hundred times faster than RAM. Cache and RAM are the same kind of thing - bytes that the CPU reads - but they sit at very different distances from the arithmetic units.

When your code reads vec[17], the CPU does not pull just byte 17. It pulls a whole 64-byte chunk - a cache line - and keeps that line in L1. The next read of vec[18] is then almost free. Reading sequentially through a Vec is fast because every line that gets loaded is mostly used before it gets evicted. Reading at random is slow because every read costs a fresh trip to RAM.

A pointer is an address in memory. Following one - *ptr - is one memory read at an address the CPU does not get to predict. If the address is in cache, the read is fast; if not, you wait the full ~100 ns. A program with many objects and many pointers between them is a program with many of those waits.

That asymmetry is the dominant fact about modern CPUs. The arithmetic - adding, multiplying, branching - is virtually free; the cost is getting the data to the arithmetic. A program that respects this is fast. A program that ignores it can be a hundred times slower² than a program that does the same work, with the same number of additions, but in a layout the cache likes.

This is also what makes “complexity class” misleading on its own. An O(N log N) algorithm that hits the cache hard can outrun a “faster” O(N) algorithm that scatters reads across RAM. Big-O describes how cost grows with N; layout describes the constant factor that gets multiplied in. At the scales this book targets, the constant factor often wins.

You will measure this in the next two sections. The numbers above are nominal - the chip in front of you may be slightly faster or slightly slower, and the ratios are what matters. Once you have felt how big the gap is, the rest of the book’s reasoning about layout, SoA, locality, and parallelism follows naturally.

Measurements

Reference values for these calibrations - yours will differ by machine, and the spread is the point. Full per-machine output: code/README.md.

Exercises

These exercises are calibrations. Run them on your machine and write the numbers down - the rest of the book references them.

1. Look up your cache sizes. On Linux, lscpu | grep -i cache lists L1d, L1i, L2, L3 per core. Write them down. (On macOS: sysctl -a | grep cache.) These are the budgets node 25 will hold you to later.
2. Time a sequential sum. Build a Vec<u64> of 100,000,000 elements (use vec![1u64; 100_000_000]), then time vec.iter().sum::<u64>(). Use std::time::Instant. Note the time per element in nanoseconds.
3. Time a random-access sum. Build the same Vec<u64>, plus a Vec<usize> of 100,000,000 random indices. Time the loop let mut s = 0u64; for &i in &indices { s += vec[i]; }. Compare with exercise 2.
4. Find the cache cliffs. Repeat exercise 2 at sizes 1K, 10K, 100K, 1M, 10M, 100M. Plot time/element (or just print it). Note the size at which it jumps - that’s where you spilled out of L1, then L2, then L3.

Note - What you see depends on your CPU. On older or smaller chips (Raspberry Pi 4 Cortex-A72, 2012 i7-3610QM, 2015 i3-5010U), the L1, L2, and L3 transitions appear as a graded staircase in ns/element. On modern desktop chips, a stronger prefetcher and wider SIMD often merge L1/L2/L3 into a single visible cliff at the L3→RAM boundary. Both are correct - both teach the same point. If you see one cliff on your machine, repeat the exercise with random indices (the §1.3 pattern) to surface the others.

5. Pointer chasing. Build a linked list of 1,000,000 Box<Node> where Node { value: u64, next: Option<Box<Node>> }. Time a sum that walks the list. Compare with the same sum on a Vec<u64> of the same length. The ratio is roughly the L1-to-RAM ratio.

Note - The ratio depends on your CPU. Measured: ~60× on a Raspberry Pi 4, ~90-115× on mid-2010s Intel laptops, ~270× on a Ryzen 9 270. The wider gap on newer hardware reflects faster cores running ahead of an unchanged DRAM latency. The order of magnitude (60-270×) is robust; the exact factor is not. Note also: a list built by for i in (0..N).rev() { Box::new(...) } allocates the boxes at sequential heap addresses - the chase looks free. Shuffle the order in which you thread them to surface the real cost.

6. (stretch) Read your lscpu output to your benchmarks. With your cache sizes from exercise 1 and your timings from exercise 4, identify which level of cache each size step is leaving. The transitions are not always clean - annotate where they are noisy.

Reference notes for these exercises in 01_the_machine_model_solutions.md.

What’s next

The numbers you wrote down in exercise 1 and the cliffs you found in exercise 4 are the constants behind the whole book. §2 - Numbers and how they fit takes the next step: how big is each unit of data, and how many fit in a cache line?

Solutions: 1 - The machine model

These exercises are about measuring your machine. Numbers vary; ratios are stable. Run them and write down what you see.

Exercise 1 - Cache sizes

Linux: lscpu | grep -E 'L1|L2|L3' or getconf -a | grep CACHE.

Typical desktop x86-64 in 2026: L1d 32-48 KB per core, L2 1-2 MB per core, L3 16-128 MB shared. Apple Silicon: larger L1, very large shared L2.

Exercise 2 - Sequential sum

use std::time::Instant;

fn main() {
    // Playground-scaled. Use 100_000_000 locally for the real number.
    let n = 10_000_000;
    let v: Vec<u64> = vec![1; n];
    let start = Instant::now();
    let sum: u64 = v.iter().sum();
    let elapsed = start.elapsed();
    let ns_per_elem = elapsed.as_nanos() as f64 / v.len() as f64;
    println!("sum = {sum}, {elapsed:?}, {ns_per_elem:.2} ns/elem");
}

Expect somewhere around 0.2-1 ns per element on modern hardware. The loop is memory-bandwidth bound; the CPU is mostly waiting for RAM to deliver lines.

Exercise 3 - Random-access sum

use std::time::Instant;

fn main() {
    // Playground-scaled. Use 100_000_000 locally for the real number.
    let n: usize = 10_000_000;
    let v: Vec<u64> = vec![1; n];
    let mut state = 0xDEAD_BEEFu64;
    let indices: Vec<usize> = (0..n)
        .map(|_| {
            state = state.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
            (state as usize) % n
        })
        .collect();

    let start = Instant::now();
    let mut sum = 0u64;
    for &i in &indices {
        sum += v[i];
    }
    let elapsed = start.elapsed();
    println!("sum = {sum}, {elapsed:?}, {:.2} ns/elem",
             elapsed.as_nanos() as f64 / n as f64);
}

Expect 30-100 ns per element - close to the RAM-latency cost. Each access misses cache.

Exercise 4 - Cache cliffs

The transitions you see roughly correspond to spilling out of L1 (~32 KB), L2 (~1-2 MB), L3 (~32 MB). Below L1 you should see ~0.1-0.3 ns/elem. In L3 maybe 0.5-1.5 ns. Past L3, 0.5-3 ns (sequential, since prefetcher helps even from RAM).

For random-access cliffs (a more dramatic plot), repeat exercise 3 at sizes 1K, 10K, 100K, 1M, 10M, 100M. The transitions are sharper.

Exercise 5 - Pointer chasing

The trap in this exercise is that the obvious way to build the list does not measure what you think it measures. A list built by threading boxes as you allocate them -

#![allow(unused)]
fn main() {
// anti-pattern: bad! nodes land at sequential heap addresses
let mut head = Box::new(Node { value: 0, next: None });
for i in 1..n { head = Box::new(Node { value: i, next: Some(head) }); }
}

• hands every Box back-to-back from the allocator. The “linked list” is a Vec in disguise: traversal walks straight up consecutive cache lines and the prefetcher hides every read. You will measure ~2 ns/elem and conclude pointers are free. They are not; you measured a contiguous scan.

To surface the real cost, build in two steps: allocate all the nodes first, then shuffle the order in which you thread them. Each box keeps its original address; the chain visits them scrambled, so each next is a jump to an unpredictable line.

use std::time::Instant;

struct Node { value: u64, next: Option<Box<Node>> }

fn build_shuffled(n: usize) -> Box<Node> {
    let mut nodes: Vec<Box<Node>> = (0..n as u64)
        .map(|i| Box::new(Node { value: i, next: None }))
        .collect();

    // Fisher-Yates with an inline LCG (no deps, Playground-friendly).
    let mut s = 0x1234_5678_u64;
    let mut rng = || { s = s.wrapping_mul(6364136223846793005).wrapping_add(1); s };
    for i in (1..nodes.len()).rev() {
        let j = (rng() % (i as u64 + 1)) as usize;
        nodes.swap(i, j);
    }

    let mut head: Option<Box<Node>> = None;
    while let Some(mut node) = nodes.pop() { node.next = head; head = Some(node); }
    head.unwrap()
}

fn sum(mut head: &Node) -> u64 {
    let mut s = 0;
    loop {
        s += head.value;
        match &head.next {
            Some(next) => head = next,
            None => break s,
        }
    }
}

// Recursive Drop walks `next` on the stack and overflows at large N.
// Tear the chain down iteratively.
fn drop_list(head: Box<Node>) {
    let mut cur = Some(head);
    while let Some(mut node) = cur { cur = node.next.take(); }
}

fn main() {
    // Playground-scaled. Use 1_000_000 locally for the real number.
    let n = 100_000;
    let head = build_shuffled(n);
    let start = Instant::now();
    let s = std::hint::black_box(sum(&head));
    let elapsed = start.elapsed();
    println!("sum = {s}, {elapsed:?}, {:.2} ns/elem",
             elapsed.as_nanos() as f64 / n as f64);
    drop_list(head);
}

A Vec<u64> sum runs 0.1-2 ns/elem depending on the chip; the shuffled linked-list walk is the same scan paying full DRAM latency on every next. The measured ratio is ~60× on a Pi 4, ~90-115× on mid-2010s Intel, ~270× on a Ryzen 9 270 (see code/measurement/src/bin/pointer_chase.rs and the cross-machine table in code/README.md). Without the shuffle the ratio collapses toward 1× - that is the prefetcher, not the absence of a tax.

Three stack-overflow traps hide in this exercise, all from recursion over next:

• Building by deep recursion overflows on the way down - the loop above scales.
• A recursive sum overflows likewise - walk it iteratively.
• The implicit Drop is recursive too, and fires when head leaves scope. At N=1M it overflows even though your code never named a recursive function. drop_list tears the chain down by hand.

Exercise 6 - Reading lscpu against your benchmarks

The transitions are noisy because:

• Cache levels overlap (a hot cache line might still be in L1 after spilling to L2).
• Hardware prefetchers help sequential reads.
• The OS may evict pages between runs.

If your noise is worse than your signal, run each measurement multiple times and take the median.

2 - Numbers and how they fit

Concept node: see the DAG and glossary entry 2.

A cache line is 64 bytes on x86 and most ARM chips - the unit of memory the CPU loads at a time. (A few designs differ: some Apple Silicon cache levels use 128; §33 has the details.) This book assumes 64 throughout. Everything you do with data is, in part, a question of how many things fit in one cache line.

Rust gives you several integer widths: u8 (one byte, 0 to 255), u16 (two bytes, 0 to 65 535), u32 (four bytes, around four billion), u64 (eight bytes, around 1.8×10¹⁹). The signed versions - i8, i16, i32, i64 - use one bit for the sign and the rest for magnitude. For floating-point: f32 (four bytes, ~7 decimal digits of precision), f64 (eight bytes, ~15 decimal digits).

A Vec<u8> of length N is N bytes. A Vec<u64> is 8N bytes. So a Vec<u8> fits 64 elements per cache line; a Vec<u64> fits 8. Walk the whole vector and the u64 version pulls in 8× as many cache lines as the u8 version: the same element count, eight times the bytes¹.

This is the width budget. Picking a wider type than you need is not free; it costs cache lines, and at the scales this book targets, cache lines are the budget you spend.

The rule is simple: pick the narrowest type that holds your range, and write down why. A 52-card deck’s suits need 4 values, ranks need 13, locations need maybe 8 - all fit in u8. A creature’s pos needs about ten kilometres of grid resolved to centimetre precision; that fits in f32. A timestamp in microseconds for a year-long simulation needs something like 3×10¹³, which does not fit in u32 (4×10⁹) but fits comfortably in u64. Choose, and write the choice down.

Floats are the trickier case. They look like real numbers but are not. There are only about 4 billion f32 values; there are only about 18 quintillion f64 values; that is finite. Operations have edges: 1.0 / 0.0 = inf, 0.0 / 0.0 = NaN, and NaN != NaN - yes, equality is broken on purpose, because there is no reasonable answer. Subtracting two nearly equal floats loses most of their precision (this is catastrophic cancellation). Adding a tiny float to a large one quietly drops the tiny one (this is absorption). None of this is a problem if you know it is there; all of it is a problem if you assume floats are mathematics.

Most of this book uses u8, u16, u32, f32, and u64 for time. i* and f64 appear when the range or precision genuinely demands it. The choice is documented at every column declaration.

Measurements

Eight times the bytes is less than eight times the time - the sum is bandwidth-bound, not purely line-count-bound, and a wider type also feeds the prefetcher more to chew on. Full output: code/README.md.

Exercises

1. Sizes. Print std::mem::size_of::<u8>(), <u16>, <u32>, <u64>, <i32>, <f32>, <f64>, <usize>. Confirm usize is 8 on a 64-bit machine.
2. Cache-line packing. For each type above, compute how many fit in a 64-byte cache line. A Vec<u32> of 16 elements is exactly one line; a Vec<u64> of 8 elements is exactly one line.
3. Width and speed. Sum a Vec<u8> of 100,000,000 ones, then a Vec<u64> of the same length. Compare times. Some of the difference is memory bandwidth (8× more bytes); some is cache pressure.
4. Float weirdness. Compute 0.0_f64 / 0.0_f64, 1.0_f64 / 0.0_f64, and (0.0_f64).sqrt(). Print them. Then check let nan = 0.0_f64 / 0.0_f64; assert!(nan != nan); - confirm it does not panic.
5. Catastrophic cancellation. Compute 1e10_f32 - (1e10_f32 - 1.0_f32). The result should be 1.0; on f32 it usually is not. Repeat with f64 and observe it gets closer.
6. Choose a width. For each of these columns, write down the type you would pick and why: a creature’s age in ticks at 30 Hz over a year-long simulation; a card’s suit; the pixel count of a 4K screen; the user id in a system with up to 100 million users; an audio sample value in 16-bit PCM.
7. (stretch) The actual range of f32. Read the f32 documentation. What is f32::MAX? f32::EPSILON? What does the latter mean for a sum of small numbers?

Reference notes in 02_numbers_and_how_they_fit_solutions.md.

What’s next

§3 - The Vec is a table takes the next step: now that you know how big the elements are, what does a Vec<T> do with them?

Solutions: 2 - Numbers and how they fit

Exercise 1 - Sizes

use std::mem::size_of;

fn main() {
    println!("u8:    {}", size_of::<u8>());     // 1
    println!("u16:   {}", size_of::<u16>());    // 2
    println!("u32:   {}", size_of::<u32>());    // 4
    println!("u64:   {}", size_of::<u64>());    // 8
    println!("i32:   {}", size_of::<i32>());    // 4
    println!("f32:   {}", size_of::<f32>());    // 4
    println!("f64:   {}", size_of::<f64>());    // 8
    println!("usize: {}", size_of::<usize>());  // 8 on 64-bit
}

Exercise 2 - Cache-line packing

Exercise 3 - Width and speed

A Vec<u8> sum reads roughly 1/8 the bytes that a Vec<u64> sum does. Modern CPUs are usually memory-bandwidth bound on simple sums, so expect about 4-8× speed difference (not always 8×, because the small-element loop may not auto-vectorise as well, or because the wider type fits more arithmetic per instruction).

Exercise 4 - Float weirdness

0.0 / 0.0 = NaN
1.0 / 0.0 = inf
(-1.0).sqrt() = NaN
let nan = 0.0_f64 / 0.0_f64;
nan != nan  // true!

NaN != NaN is by IEEE 754 definition: there is no sensible value to compare with, so equality is false. assert!(nan == nan) would panic; we want assert!(nan != nan).

Exercise 5 - Catastrophic cancellation

#![allow(unused)]
fn main() {
let a: f32 = 1e10;
let b: f32 = 1e10 - 1.0;  // f32 may not even represent this distinctly
println!("{}", a - b);    // expected 1.0; you may get 0.0 or 2.0 or 1024.0

let a: f64 = 1e10;
let b: f64 = 1e10 - 1.0;
println!("{}", a - b);    // closer to 1.0
}

f32 has ~7 decimal digits; 1e10 already exhausts those. f64 has ~15.

Exercise 6 - Choose a width

Exercise 7 - f32 ranges

f32::MAX ≈ 3.4×10³⁸. f32::EPSILON ≈ 1.2×10⁻⁷. EPSILON is the smallest x for which 1.0 + x ≠ 1.0. Adding many EPSILON-scale numbers to a large value can therefore not increase it - they get absorbed. Summing 10⁹ small floats is often less accurate than summing them in pairs (a Kahan sum fixes this).

3 - The Vec is a table

Concept node: see the DAG and glossary entry 3.

A Vec<T> is three things stored on the stack: a pointer to a contiguous run of T values on the heap, the current length, and the current capacity. The values themselves live on the heap, side by side, with no padding between them. vec[i] computes ptr + i * size_of::<T>() and reads.

This is the only container the trunk of this book uses. There are no hash maps, no linked lists, no trees - not because they do not exist, but because almost every problem the book teaches is a problem of “process all the rows of a table”, and a Vec<T> is the table. Adding any other container costs cache, costs allocations, and breaks the sequential-access pattern that nodes 1 and 2 just told you to want.

vec.push(x) adds an element. If there is capacity, it writes into the next slot - O(1). If not, it allocates a larger heap region (typically twice the current capacity), copies everything across, and frees the old one. Amortised over many pushes that is O(1), but each individual push might be expensive. If you know how many elements you are going to insert, Vec::with_capacity(n) allocates once and avoids the copies.

vec.swap_remove(i) removes the element at i in O(1) by moving the last element into the freed slot. Order is sacrificed for speed. This will earn its keep at §21.

vec.iter() walks the slots in order. The compiler can usually turn this into a tight memory-bandwidth-bound loop with auto-vectorisation. vec.iter_mut() does the same, with mutation.

A &[T] is a slice - a pointer plus a length, without the capacity. It is what functions usually take when they want to read a Vec without owning it. &mut [T] is the same with mutation. Most systems in this book have signatures like fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32]) - read these, write those, no ownership taken.

That is the full vocabulary you need from Vec for the next several phases. Everything else (HashMap, BTreeMap, Box<Node>, Rc<RefCell<T>>, LinkedList) is something you will reach for only when an exercise demands it and the from-scratch test (node 40) shows it earns its weight.

Measurements

Order of magnitude (60-200×) is the durable claim; the exact factor widens with the machine because the Vec sum vectorises and prefetches and HashMap::get cannot. Full output: code/README.md.

Exercises

1. Layout. Print std::mem::size_of::<Vec<u32>>(). It should be 24 on a 64-bit machine - three pointer-sized fields. Notice that the size of the Vec value does not depend on how many elements it holds.
2. Capacity vs length. Build let mut v: Vec<u32> = Vec::new();. In a loop from 0 to 100, print v.len() and v.capacity() after each v.push(i). Observe the capacity doubling pattern: 0, 4, 8, 16, 32, 64, 128.
3. Pre-size. Build let mut v = Vec::with_capacity(100); and push 100 elements. Print len and capacity once at the end. There were no reallocations.
4. Indexing cost. Sum a 1M Vec<u32> by index: black_box((0..N).map(|i| v[i] as u64).sum::<u64>()). Time it. Now build a HashMap<usize, u32> with the same entries (key i maps to the value) and sum it by the same indices: (0..N).map(|i| map[&i] as u64).sum::<u64>(). Each map[&i] is one hash-and-probe, so this measures indexed lookup. Draining the map with into_values().sum() walks it in bucket order instead, a cheap sequential pass that measures the wrong thing. Indexed Vec reads should be ~10-100× faster¹.

Note - Measured ratios: ~65× on a Raspberry Pi 4, ~90-95× on mid-2010s Intel laptops, ~160× on a Ryzen 9 270. All use Rust’s default HashMap (SipHash). Modern hardware widens the gap because the Vec sum is auto-vectorized and well-prefetched; HashMap::get cannot be either. Order-of-magnitude (60-200×) is the durable claim.

5. swap_remove vs remove. Build a Vec<u32> of 1,000,000 elements. Time removing 100 elements from the middle with vec.remove(500_000) (in a loop, because each remove shifts roughly half the vector). Time the same with vec.swap_remove(500_000). Note the orders-of-magnitude difference.
6. Slices in function signatures. Write fn sum(xs: &[u32]) -> u64. Call it with sum(&v) where v: Vec<u32>. Note that you did not have to write &v[..] - the conversion is automatic.
7. (stretch) A from-scratch MyVec<u32>. Implement MyVec with a raw pointer, length, and capacity. Implement new, push, get, and Drop. (You will use unsafe. Read the Rustonomicon’s Vec chapter when stuck.) Convince yourself a Vec<T> is a few hundred lines of careful work, no magic.

Reference notes in 03_the_vec_is_a_table_solutions.md.

What’s next

§4 - Cost is layout, and you have a budget is where the layout reasoning from §1 and §2 meets the per-tick clock the rest of the book runs on. After that, §5 - Identity is an integer is the card game.

Solutions: 3 - The Vec is a table

Exercise 1 - Layout

use std::mem::size_of;
fn main() {
    println!("Vec<u32> = {}", size_of::<Vec<u32>>());  // 24
    println!("Vec<u64> = {}", size_of::<Vec<u64>>());  // 24
    println!("Vec<u8>  = {}", size_of::<Vec<u8>>());   // 24
}

A Vec<T> is always 24 bytes on a 64-bit machine (three 8-byte fields: ptr, len, cap), regardless of T. The element data lives elsewhere on the heap.

Exercise 2 - Capacity growth

#![allow(unused)]
fn main() {
let mut v: Vec<u32> = Vec::new();
for i in 0..100 {
    v.push(i);
    if v.len().is_power_of_two() || v.len() < 5 {
        println!("len={}, cap={}", v.len(), v.capacity());
    }
}
}

Output (Rust’s current strategy roughly doubles, but starts at 4):

len=1, cap=4
len=2, cap=4
len=4, cap=4
len=8, cap=8
len=16, cap=16
len=32, cap=32
len=64, cap=64

Each transition is a reallocation: a new heap region is allocated, all elements are memcpy’d across, the old one is freed.

Exercise 3 - Pre-size

#![allow(unused)]
fn main() {
let mut v = Vec::with_capacity(100);
for i in 0..100 { v.push(i); }
println!("len={}, cap={}", v.len(), v.capacity()); // len=100, cap=100
}

No reallocations happened. This is the right pattern when you know the upper bound - and most simulations do.

Exercise 4 - Indexing cost

Indexing a Vec<u32> runs ~1 ns/elem. A HashMap<usize, u32> lookup costs ~50-100 ns each (hash, probe, compare). Multiple orders of magnitude.

Iterate the map by index - map[&i] for each i - not into_values(). The cost being measured is the per-key hash-and-probe that an indexed lookup pays; draining the map in bucket order is a cheap sequential walk that hides exactly that cost, so it reports the wrong ratio.

Exercise 5 - swap_remove vs remove

100 calls to vec.remove(500_000) on a 1M Vec<u32> move ~50 million elements (each remove shifts ~half the vector). At ~1 ns per move that is ~50 ms total.

100 calls to vec.swap_remove(500_000) on the same vector move 100 elements total - under a microsecond.

The factor is roughly N / 2. For 1 million entries, that is half a million times faster.

Exercise 6 - Slices in function signatures

#![allow(unused)]
fn main() {
fn sum(xs: &[u32]) -> u64 {
    xs.iter().map(|&x| x as u64).sum()
}

let v: Vec<u32> = (0..1000).collect();
let total = sum(&v);  // &Vec<u32> auto-derefs to &[u32]
}

The function takes a slice; the caller passes &v. The conversion (Deref) is automatic. This is why almost every system in the book has signatures over &[T] and &mut [T], not &Vec<T>.

Exercise 7 - A from-scratch MyVec<u32>

The full implementation is in the Rustonomicon; about 200 lines including tests. The key shape:

#![allow(unused)]
fn main() {
struct MyVec<T> {
    ptr: NonNull<T>,
    len: usize,
    cap: usize,
}
}

new starts with cap = 0 and a dangling pointer. push allocates on first push (grow), then doubles capacity when full. get returns Option<&T> with bounds check. Drop frees both elements (running their destructors) and the heap allocation.

Working through this once is the cheapest way to convince yourself a Vec<T> is a small piece of careful work - and to internalise §42 - You can only fix what you wrote.

4 - Cost is layout - and you have a budget

Concept node: see the DAG and glossary entry 4.

A system is not handed a target rate; it chooses one. Work arrives as a stream - frames to draw, packets to route, sensor samples to fold in - and the only real decision is how finely to cut that stream into batches. Each batch is one tick, and the rate is the grain of the cut. Cut at one operation per tick and nothing batches: every operation carries its fixed overhead alone, and at a 1 GHz tick the budget is a few nanoseconds, too little to work in. Cut at one tick for all pending work and efficiency is maximal but nothing is answered until everything is: a tick a minute has no perceptible responsiveness. Every useful rate sits between those ends, balancing responsiveness against the efficiency of batching.

Two different things bound that band. Whether you can keep up at all is fixed by the per-item cost against the arrival rate: if work lands at rate λ and each item costs c, you survive only when λ · c ≤ 1, and c is a layout fact (§3), not a scheduling one. The rate itself is the second, separate choice: a faster tick means smaller batches and lower latency with less to amortise each fixed cost over; a slower tick means larger batches, better amortisation, and more latency. Batching only ever pays because there are fixed costs to spread - a dispatch, a cache warmup, a syscall, a kernel launch - which is the same amortisation §8 names over data, here run along time.

The responsiveness floor is set by whoever consumes the output. Roughly 24 to 30 frames a second is where discrete frames read as continuous motion for passive viewing, which is why film sits there; interactive rendering wants 60, and head-mounted VR wants 90 to 120 to stay comfortable. A control loop runs as fast as its plant needs a correction, often 1 kHz; an audio loop is pinned to its 48 kHz sample rate; an interactive shell answers as fast as a human can type. Different consumers, different floors, one calculus. The rate you choose is the coarsest batch its floor will tolerate, and it sets a budget - the time available for one tick of work. What you then spend against that budget is governed by layout, which is where the rest of this chapter goes.

Every operation the program does in one tick spends from that budget. Operations have very different costs: the arithmetic is virtually free, an L1 read is around 1 ns, an L3 read is around 10 ns, a RAM read is around 100 ns, a disk read is around 100 µs, a network round-trip is around 100 ms. A 30 Hz program spending one disk read per tick has lost a third of its budget on one operation.

Note - Three regimes are worth naming, because the rest of the book references them. A loop is compute-bound when its cost is dominated by arithmetic - typically when the data fits in L1 and the inner instructions are heavy (dot products, transcendentals, integer divides). It is bandwidth-bound when its cost is dominated by how fast the memory subsystem can deliver bytes - typically when the working set is bigger than L3 but the access pattern is sequential, so the prefetcher can fill lines ahead of demand. It is latency-bound when its cost is dominated by individual memory round-trips - typically when the access pattern is random, so the prefetcher cannot help. The three regimes have very different time budgets and very different power profiles. A sequential Vec<u64> sum on a modern desktop is bandwidth-bound at ~50 GB/s, roughly 0.15 ns per element. The same Vec accessed by random index is latency-bound at one full RAM round-trip per element, roughly 50-100 ns per element - three orders of magnitude slower, despite the same arithmetic. The lesson of node 4 is that complexity-class reasoning cannot tell these regimes apart, but they are the difference between a program that meets its tick budget and one that does not.

The unit of accounting is time - microseconds for most real-time work, nanoseconds for tight inner loops. A 30 Hz tick has 33 ms (33 000 µs) of budget; a 1 kHz tick has 1 000 µs; a 1 MHz tick has 1 µs. When a teacher asks you “what does this function cost?”, they are asking how many microseconds it takes. A function that costs 100 µs out of a 33 000 µs budget is fine - about 0.3% of the tick. The same function in a 1 000 µs budget is 10% of the tick. The same function in a 1 µs budget does not exist; there is no room for it.

Cost is also layout. The same algorithm that costs 100 µs on a sequential Vec may cost 5 ms on a hash map of the same size, because the loads scatter. Two programs with the same big-O complexity can differ by an order of magnitude on the same hardware, just because of where their data sits.

This gives you a design rule. Decide your target rate before you decide anything else. That sets the budget. Then when you choose data structures, ask whether the resulting working set fits in cache; ask how many memory loads per row your inner loop does; ask whether any single operation in the loop dominates the budget. Most decisions become forced once the budget is named.

The reverse direction is also useful. If you find yourself wanting to add something to the inner loop - a database query, a HashMap lookup, an allocation - count its cost in microseconds against the budget. Often the answer is “this single addition uses 80% of my tick”, and the right move is not to optimise it but to lift it out of the inner loop entirely.

The shape of this thinking is familiar to engineers in other domains. An electrical engineer designs a circuit by counting milliamps against a current budget. A structural engineer counts kilonewtons against a load budget. The data-oriented programmer counts memory loads and microseconds against a tick budget. Good design is measured in millivolts and microamps - and in nanoseconds and microseconds.

Note - Time is one budget. Power is another. Cache hits are energetically nearly free - the data is already next to the arithmetic units. Cache misses fire up the memory controller, the bus drivers, sometimes a DRAM refresh; that is where the watts go. A loop that fits in L2 spends most of its time on cheap arithmetic; a loop that pointer-chases through RAM spends most of its time waiting, and during the waiting the CPU drops clocks and the chip stays cool. The same SoA-and-sequential-access discipline that fits the time budget also fits a power budget. For embedded, mobile, control, and battery-powered work, power is the primary budget; time is downstream of it. The “millivolts and microamps” line above is literal, not metaphor.

The budget is a curve, not a cliff

So far the budget has been a single number: name the rate, get the time per tick. But the work in a tick is rarely fixed. It grows with the problem - more entities to step, more packets to route, more rows to fold - and if the per-item cost holds, the tick time grows with it. So the rate you can actually sustain is not a constant either; it falls as the work rises, roughly as one over the size for a loop that costs O(N). Thirty hertz is not a wall you meet at some population and shatter against. It is one point on a slope that reads thirty, twenty-five, twenty, fifteen as the work climbs.

That moves the engineering question. It is seldom “does it hit thirty hertz” and almost always “where does the curve fall, and is that fall tolerable”. A control loop specified at thirty may be well served by twenty under a heavier load; a visualisation at fifteen is still watchable. So the useful design conversation names two numbers, not one: the target rate, and the tolerance - the slowest rate the consumer will accept - then reads off the scale at which the curve crosses the tolerance. You characterise the budget around the target instead of slamming into it, and “how many can we handle” becomes a number you read off a measured curve rather than a guess you defend in a meeting.

Part II puts real numbers on this slope, measured on the simulator across two orders of magnitude of scale, where a tick that holds comfortably above the target at one size slides to a fraction of the rate at a hundred times the work - all along the same one-over-N curve.

Exercises

1. Pick your rates. For each of these systems, name a plausible target rate and the resulting per-tick budget: a card game; a real-time strategy game; a market data feed; an embedded sensor controller; a web API endpoint a user is waiting for; an offline batch job that processes a billion rows.

2. Count an operation. Time a single HashMap::get on a map of 1 000 000 entries. Note its cost in microseconds. How many can you fit in a 30 Hz tick (33 ms)? In a 1 kHz tick (1 ms)?

3. The layout difference. Sum 1 000 000 u64s in a Vec<u64>. Sum 1 000 000 u64s in a HashMap<u32, u64>. Both are O(N). What is the per-element time difference (in nanoseconds)? Where did it go?

4. The cliff. With your numbers from §1 exercise 4, pick a Vec size that just fits in L2 and one that just doesn’t. Time a sum loop at each size. The cliff is real.

5. Working backwards from the budget. You target 60 Hz; your inner loop runs over 100 000 entities; each entity touches one cache line. Estimate the cost of the loop in microseconds and compare to your 60 Hz budget (16 666 µs). Where is your headroom?

6. A bad design. Construct a design that is “obviously fast” by big-O reasoning but blows the 30 Hz budget on a million entities. (Hint: object-graph traversal with one heap allocation per node is a classic.)

7. Find your CPU’s TDP. Look up your CPU’s rated thermal design power on the manufacturer’s spec sheet, or read it locally on Linux with sudo dmidecode -t processor | grep -i 'power\|TDP'. Note the value. TDP is what the chip can dissipate sustained without thermal throttling - burst can be 1.5-2× higher for tens of seconds; sustained settles back to TDP.

8. Battery budget. A typical laptop battery holds about 50 Wh. Your simulator runs at 30 Hz and draws an average of 8 W (mostly memory bandwidth on the inner loop). How many hours of simulation does a full charge buy? If a layout change pushes more loads to RAM and raises the average draw to 14 W, how many hours then? Express the cost of the layout change as a percentage of battery life.

9. Measure delta power. A ready-made workload generator lives at code/measurement/. In one terminal: cargo run --release --bin power_loop -- sequential (then in a second run: ... -- random). In another terminal, while the loop is running: sudo perf stat -a -e power/energy-pkg/ -- sleep 30 reads the package-energy counter over 30 seconds. Run the perf command three times - idle, sequential, random - and write the joules down. Convert each to average watts. The random-access run should draw more watts than the sequential one, which should draw more than idle.

   While you are there: from power_loop’s iteration count, compute your sequential read bandwidth - iterations × 10⁷ × 8 / 45 gives bytes per second - and compare to the published peak of your DDR generation. If you get within a factor of two of peak, your inner loop is bandwidth-bound (the regime named in the prose). The random mode’s iteration count, divided into wall time, gives your effective per-element latency in nanoseconds; that is the latency-bound regime.

10. (stretch) Joules per access. Approximate energies per memory read: L1 hit ≈ 0.1 nJ, L2 ≈ 1 nJ, RAM ≈ 30 nJ (rough; published numbers vary by chip and process). Estimate the total energy of summing 10⁷ u64s sequentially (mostly prefetched, near-L1 cost) versus by random indices (mostly RAM misses). Convert both to milliwatt-hours and express as a fraction of a 50 Wh battery. The absolute numbers are tiny; the ratio is what your battery life and your data-centre electricity bill care about.

11. The budget is a curve. Take the loop from exercise 5 (100 000 entities, one cache line each, 60 Hz). Hold the per-entity cost fixed and sweep the entity count: 100 000, 300 000, 1 000 000, 3 000 000. Compute the tick time and the sustained rate at each. At what size does the sustainable rate cross 30 Hz? 15 Hz? Plot rate against size and confirm the one-over-N shape, then name the largest scale that still meets a 15 Hz tolerance. (This is the curve Part II measures on the simulator; here you derive it from the per-item cost.)

Reference notes in 04_cost_and_budget_solutions.md.

What’s next

You now have the machine model (§1), the data widths (§2), the table primitive (§3), and the budget calculus (§4). The next section is the conceptual heart of the book: §5 - Identity is an integer. The card game is waiting.

Solutions: 4 - Cost is layout, and you have a budget

Exercise 1 - Picking rates

The lesson is: every system has a rate, even ones not described as “real-time”. Naming it makes the budget visible.

Exercise 2 - Count an operation

HashMap::get on 1M entries: typically 50-150 ns. Pick the middle: 100 ns = 0.1 µs. In a 30 Hz tick (33 333 µs) you can fit 33 333 / 0.1 ≈ 330 000 lookups. In a 1 kHz tick (1 000 µs) you can fit 10 000.

If your “for each entity, look up something” loop has 1 000 000 entities, neither tick budget fits - you must restructure (a sorted index, a join, a column lookup) or accept a slower rate.

Exercise 3 - The layout difference

Vec<u64> sum: ~0.2-2 ns/elem. HashMap<u32, u64> sum: ~40-160 ns/elem.

Roughly 65-165× difference for the same total work, measured across the four reference machines (vec_vs_hashmap; per-machine numbers in code/README.md). Most of it goes to memory: hash maps have one cache line per bucket, the buckets aren’t sequential, the hash itself touches more bytes per access. Big-O is the same; constant factor decides.

Exercise 4 - The cliff

For a typical desktop with 1 MB L2:

• 100 000 u64 = 800 KB → fits in L2 → ~1 ns/elem.
• 1 000 000 u64 = 8 MB → spills L2 into L3 → ~2-4 ns/elem.

Compute the fraction of the budget by ratio. At 30 Hz (33 333 µs), 1M elements at 1 ns each is 1000 µs ≈ 3% of the budget. At 4 ns each, 4000 µs ≈ 12% of the budget. The cliff is real.

Exercise 5 - Working backwards

60 Hz tick: 16 666 µs = 16 666 000 ns. 100 000 entities × one cache line each. A cache line takes ~1-3 ns to load if sequential (L2/L3) or ~20-100 ns if random (RAM).

Sequential: 100 000 × 2 ns = 200 µs = 1.2% of the tick. Lots of headroom. Random RAM: 100 000 × 50 ns = 5 000 µs = 30% of the tick. Tight but possible. Random pointer-chase to scattered allocations: 100 000 × 100 ns = 10 000 µs = 60% of the tick. One inner loop, sixty percent. No headroom for anything else.

The lesson: ask whether the access is sequential before estimating.

Exercise 6 - A bad design

The classic: a graph of Box<Node> allocated by many small calls to Box::new, then iterated by following pointers. Each node is a separate heap allocation, scattered across the heap by the allocator. A million-node “linked structure” is a million RAM round-trips per traversal: 100 ns × 10⁶ = 100 ms - three full 30 Hz ticks for one traversal.

The same data laid out as a Vec<Node> (or, better, as SoA: Vec<u32> of values plus Vec<u32> of next-indices) traverses in 1-2 ms - fifty times faster, same algorithm, different layout. This is the whole book’s premise in one number.

Exercise 7 - Find your CPU’s TDP

Typical 2026 ranges:

• AMD Ryzen 9 mobile (e.g. 7940HS, 8945HS): cTDP 35-54 W (configurable).
• AMD Ryzen 9 desktop (e.g. 9950X): TDP 170 W; PPT (sustained) up to 230 W.
• Intel Core i9-13900H (mobile): PL1 45 W, PL2 115 W.
• Apple M3 Pro: roughly 25-35 W under sustained load.

A “TDP” number is the sustained envelope; burst (PL2 on Intel, PPT on AMD) can run 1.5-3× higher for tens of seconds before thermal/PPT limits clamp it back. For a sim that runs continuously, the sustained number is the budget that matters.

Exercise 8 - Battery budget

50 Wh ÷ 8 W = 6.25 hours. 50 Wh ÷ 14 W = ~3.57 hours.

The 75% rise in average draw (8→14 W) cuts battery life by 43%. A layout change that pushes loads to RAM is not a footnote on the time budget; it is a roughly halving of how long the laptop runs on one charge.

Exercise 9 - Measure delta power

The workload generator at code/measurement/src/bin/power_loop.rs takes one argument (sequential or random) and runs the chosen workload in a tight loop for 45 s - long enough to outlast a 30 s perf stat window comfortably. Build once with cargo build --release --bin power_loop inside code/measurement/, then run from a terminal:

# Terminal 1: pick a mode
cargo run --release --bin power_loop -- sequential
# (or: cargo run --release --bin power_loop -- random)

# Terminal 2: measure during a fresh start of the workload
sudo perf stat -a -e power/energy-pkg/ -- sleep 30

For the idle reading, run the perf command with no workload running. Permissions: sudo is usually required; sudo sysctl kernel.perf_event_paranoid=0 is the alternative if you don’t want to keep typing the password.

Reading the power_loop output

The binary prints something like:

sequential: summing 10000000 u64 elements in order for 45s
done: 29131 iterations in 45.000658338s - sum = 291310000000 ...

From the iteration count you can compute throughput:

elements per second = iterations × N / wall time
                    = 29131 × 10_000_000 / 45 s
                    = 6.47 × 10⁹ elements/s

read bandwidth      = elements/s × 8 bytes
                    = 52 GB/s

ns per element      = wall time / total elements
                    = 45 × 10⁹ ns / (29131 × 10⁷)
                    = 0.15 ns/element

52 GB/s is close to the practical peak of dual-channel DDR5-5600 (~60 GB/s sustained from a typical workload). The loop is bandwidth-bound: the CPU can consume bytes faster than the memory subsystem can deliver them, so the prefetcher and SIMD are saturating the channel. There is essentially no slack left in the sequential read path on this hardware.

The random-access run will show a very different number. Expect the iteration count to drop by a factor of 50-300, because each element costs a full RAM round-trip (~50-100 ns) instead of being delivered in a sequential stream (~0.15 ns).

Reading the perf stat output

perf stat -a -e power/energy-pkg/ -- sleep 30 prints something like:

       56.58 Joules power/energy-pkg/
   30.001766291 seconds time elapsed

Compute average watts:

average watts = joules / seconds = 56.58 / 30 = 1.89 W

That is the package power for the whole 30-second window, including all process activity on the machine. The per-process number is rarely what you want; the package-level number is what your battery, cooling fan, and electricity bill see.

Typical numbers

• Idle (trimmed Linux install): 2-8 W. A Ryzen 9 mobile on Arch Linux with no background work reports 1.89 W in one of this book’s draft sessions - exceptional but real. The cores are spending most of the window in deep C-states.
• Sequential power_loop: idle + 5-10 W. Bandwidth-bound: memory controller and SIMD units are working hard but the CPU also drops clocks during the prefetcher’s brief refills. High utilisation but not thermally maxed.
• Random power_loop: idle + 10-25 W. Memory subsystem is working similarly hard, but now CPU stalls on every access cannot be filled by the prefetcher. Stalls do not save power if the rest of the chip stays active - clocks remain elevated while waiting for lines.

For perspective, the chip’s cTDP is 35-54 W. That is the sustained envelope under heavy load, not a soft cap; bursts can briefly exceed it.

The lesson

Sequential vs random over the same data, the same size, the same arithmetic:

• ~300× difference in time per element (0.15 ns vs ~50 ns)
• ~2-3× difference in instantaneous watts
• ~600-1000× difference in joules per element processed

Notice that the energy ratio is the product of the time ratio and the power ratio. Energy is power × time; when a layout choice slows things down and draws more watts, the two effects compound multiplicatively. A workload that is 300× slower and 3× more power-hungry is 900× more energy-expensive - not 303×. Slow workloads pay twice: once in elapsed seconds, once in watts per second.

Layout-aware programming is power-aware programming. The bandwidth-bound path keeps the chip cool and fast; the latency-bound path keeps it hot and slow. Two paths, one chip, one decision: where does the data sit?

Exercise 10 - Joules per access

10⁷ sequential u64 reads: each line carries 8 elements, so 1.25 × 10⁶ line loads. Most are served from the prefetcher’s pipeline (effectively L1-priced after the first miss in a stream), so use ~0.5 nJ per element on average. Total: 10⁷ × 0.5 nJ = 5 mJ.

10⁷ random u64 reads: every access is a fresh L3-or-RAM miss. Use ~30 nJ per element. Total: 10⁷ × 30 nJ = 300 mJ.

Ratio: 60×.

As a fraction of a 50 Wh battery (50 × 3600 = 180,000 J):

• Sequential: 5 × 10⁻³ J / 180,000 J ≈ 2.8 × 10⁻⁸ of a charge - negligible per run.
• Random: 300 × 10⁻³ J / 180,000 J ≈ 1.7 × 10⁻⁶ of a charge - still small per run.

The absolute numbers are tiny; the ratio is what scales. Run that loop ten million times a day across a fleet of devices and the choice of layout becomes the difference between a noticeable cooling-fan hum and a silent machine - and, at data-centre scale, a measurable line item on the electricity bill.

Exercise 11 - The budget is a curve

From exercise 5, the per-tick cost is one cache line per entity. Hold that fixed and the tick time is linear in the count: if 100 000 entities cost c per tick, then N entities cost c · N / 100 000, and the sustainable rate is 1 / tick_time, which falls as one over N.

Take a round example: say the 100 000-entity loop runs in 2 ms (well inside the 16.7 ms / 60 Hz budget). Then:

The rate crosses 30 Hz between 1M and 1.7M entities (33.3 ms / (2 ms / 100 000) ≈ 1.67M); it crosses 15 Hz at about 3.3M (66.7 ms ÷ 20 ns per entity). Plotted, rate against size is a 1/N hyperbola - halve nothing, double the work, halve the rate.

The point is the shape, not the exact numbers (your c depends on your machine): a budget is a curve, and “the largest scale that still meets 15 Hz” is a number you read off it. This is the same curve Part II measures directly on the simulator, where the per-entity cost is whatever the tick actually does rather than an assumed cache line; the slope is the same.

5 - Identity is an integer

Concept node: see the DAG and glossary entry 5.

Hand a programmer fifty-two cards and tell them to write code that shuffles, sorts, and deals. Ask how long.

Most will start drawing classes - Card, Deck, Hand, Player, maybe a Game - and quote you four hours. They are being honest. The class hierarchy is real work. There will be constructors, copy semantics, and a vague unease about whether Hand should hold pointers or values, whether Deck owns its cards or borrows them, whether shuffling should mutate the deck or return a new one.

The whole problem fits in three lines. The way it fits is the lesson of this section.

A deck of cards has three pieces of information per card: its suit (♠ ♥ ♦ ♣), its rank (A, 2, …, K), and its current location (in the deck, in someone’s hand, in the discard pile). That is three columns. The deck itself is fifty-two rows.

In Rust:

#![allow(unused)]
fn main() {
let suits:     Vec<u8> = vec![ /* 52 entries: 0..4 */ ];
let ranks:     Vec<u8> = vec![ /* 52 entries: 0..13 */ ];
let locations: Vec<u8> = vec![ /* 52 entries: 0=deck, 1=hand1, ... */ ];
}

That is the deck. There is no Card struct. There is no Deck class. The card at index 17 has its suit at suits[17], its rank at ranks[17], and its current location at locations[17]. The card is the index.

Dealing a card from the deck to player 1 is one line:

#![allow(unused)]
fn main() {
locations[17] = 1; // card 17 is now in player 1's hand
}

Asking what’s in player 1’s hand is one loop:

#![allow(unused)]
fn main() {
let mut hand: Vec<usize> = Vec::new();
for i in 0..52 {
    if locations[i] == 1 {
        hand.push(i);
    }
}
}

Asking how many cards are left in the deck is one counter:

#![allow(unused)]
fn main() {
let mut count = 0u32;
for i in 0..52 {
    if locations[i] == 0 { count += 1; }
}
}

Shuffling - the move students expect to be hard - is shuffling the order of indices. 0..52 becomes [7, 32, 1, 19, ...], and you read your way through the cards in that order:

#![allow(unused)]
fn main() {
let mut order: Vec<usize> = (0..52).collect();
fisher_yates(&mut order, &mut rng); // 5 lines, written below
}

Look at what just happened. Nothing about the cards changed. suits[17], ranks[17], and locations[17] are exactly the values they were before. The shuffle moved indices, not data.

Sorting works the same way. To sort by suit then rank, you sort the indices by (suits[i], ranks[i]):

#![allow(unused)]
fn main() {
order.sort_by_key(|&i| (suits[i], ranks[i]));
}

The cards do not move. Their identifiers are reordered.

That’s the deck of cards in maybe twenty lines of Rust. It includes shuffle, sort, deal, and several queries. It is not a stylistic shortcut; it is what a deck of cards is. The OOP version’s four hours of work was the cost of pretending a card was an object that owned its suit and rank, when actually a card is one number - an index - and its suit and rank are values stored in arrays at that index.

We call this identity-is-an-integer, and it is the precondition for every economy the rest of this book buys you. Persistence will work because tables are easy to serialise. Parallelism will work because indices are cheap to partition. Replay will work because a deck is just three arrays in a state. None of it works if you reach for class Card.

Note - The strong form, which we will return to later: sometimes you do not even need the index. The pair (suit, rank) already uniquely identifies a playing card - there are only fifty-two such pairs. The index is a surrogate key; the pair is a natural key. For variable-quantity tables (creatures that come and go) you usually need a surrogate, because two creatures can be identical. For a constant-quantity 52-card deck, you do not.

Exercises

The first time through, write everything from scratch in src/main.rs. Resist the urge to add a Card struct or helper methods. Three Vecs.

1. Build the deck. Write fn new_deck() -> (Vec<u8>, Vec<u8>, Vec<u8>) that returns the suits, ranks, and locations for a fresh, ordered deck (all 52 in location 0 = deck).
2. Print a card. Write fn card_to_string(suit: u8, rank: u8) -> String that returns strings like "A♠", "10♥", "K♦". Use it to print the whole deck.
3. Shuffle. Write a tiny LCG random function (one-liner) and use it to implement Fisher-Yates on a Vec<usize>. Print the deck in shuffled order. Confirm by inspection that the suits, ranks, and locations arrays are unchanged.
4. Sort by suit then rank. Sort the order vector so suits come out grouped, ranks ascending within each suit. Print again. Once again, the deck arrays are unchanged.
5. Deal a hand. Move the first 5 cards from the deck (location 0) to player 1 (location 1). Print player 1’s hand using card_to_string.
6. Hand query. Write fn cards_held_by(locations: &[u8], player: u8) -> Vec<usize> returning all card indices currently held by a given player.
7. Count by location. Write a function that returns counts grouped by location: how many in the deck, in each hand, in discard.
8. Deal four hands. Deal 5 cards to each of players 1, 2, 3, 4. Print all four hands.
9. (stretch) Drop the index. Rewrite cards_held_by to return Vec<(u8, u8)> of (suit, rank) pairs directly - no indices. What does this make easier? What does it make harder? (Hint: you cannot move the cards back to the deck without knowing which i they were.)
10. (stretch) The sort hazard. While player 1 is holding indices [3, 17, 21, 28, 41], sort the deck arrays themselves (not just the order) by suit. What does player 1 think they hold now? This is the bug node 9 (“sort breaks indices”) was written for. Don’t fix it yet - observe it.

Reference solutions for exercises 1-3 in 05_identity_is_an_integer_solutions.md. Solutions for the rest follow the same shape.

What’s next

Exercise 10 leaves you with a bug. The next section (§9 - Sort breaks indices) is the fix; it teaches you to keep a stable id alongside the position so external references survive reordering.

Solutions: 5 - Identity is an integer

Reference solutions for the exercises in 05_identity_is_an_integer.md. Try the exercises first.

Exercise 1 - Build the deck

#![allow(unused)]
fn main() {
fn new_deck() -> (Vec<u8>, Vec<u8>, Vec<u8>) {
    let mut suits = Vec::with_capacity(52);
    let mut ranks = Vec::with_capacity(52);
    let mut locations = Vec::with_capacity(52);
    for s in 0..4u8 {
        for r in 0..13u8 {
            suits.push(s);
            ranks.push(r);
            locations.push(0); // 0 = in the deck
        }
    }
    (suits, ranks, locations)
}
}

The order of insertion sets the index-to-card mapping. Spades fill indices 0-12, hearts 13-25, and so on. The Ace of Spades is at index 0; the King of Clubs at index 51.

Vec::with_capacity(52) is a small but honest gesture: the size is known up front, so we ask for exactly that much memory. No reallocation, no surprise. This is constant-quantity behaviour - node 27 will explain why it matters at a million.

Exercise 2 - Print a card

#![allow(unused)]
fn main() {
const SUIT_CHARS: [&str; 4] = ["♠", "♥", "♦", "♣"];
const RANK_CHARS: [&str; 13] = [
    "A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K",
];

fn card_to_string(suit: u8, rank: u8) -> String {
    format!("{}{}", RANK_CHARS[rank as usize], SUIT_CHARS[suit as usize])
}

fn print_deck(suits: &[u8], ranks: &[u8]) {
    for i in 0..suits.len() {
        println!("{:>2}: {}", i, card_to_string(suits[i], ranks[i]));
    }
}
}

The as usize casts are because Vec and array indexing want usize. Choose u8 for the columns because we’re never going to have 256 suits or ranks; the smaller width saves memory and keeps more of the deck in L1.

Exercise 3 - Shuffle

A tiny LCG, then Fisher-Yates over the index order:

#![allow(unused)]
fn main() {
// A Linear Congruential Generator. Not cryptographic. Fine for shuffling cards.
fn rand(state: &mut u64) -> u64 {
    *state = state
        .wrapping_mul(6364136223846793005)
        .wrapping_add(1442695040888963407);
    *state >> 32
}

fn shuffle(n: usize, seed: u64) -> Vec<usize> {
    let mut order: Vec<usize> = (0..n).collect();
    let mut state = seed;
    for i in (1..n).rev() {
        let j = (rand(&mut state) as usize) % (i + 1);
        order.swap(i, j);
    }
    order
}

fn print_deck_shuffled(suits: &[u8], ranks: &[u8], order: &[usize]) {
    for &i in order {
        println!("{}", card_to_string(suits[i], ranks[i]));
    }
}
}

The print function takes suits, ranks, and order as &[u8] and &[usize] slices - none of them are mutated. The cards stay where they are. Only the traversal changes.

Note - A real shuffle wants a stronger RNG; for fifty-two cards an LCG is fine. The exercise is about the indices, not the entropy. When you have an excuse to use a real RNG, the from-scratch test (node 40) applies: write the LCG version first, then read whatever crate you might reach for, and pick consciously.

Exercises 4-8

Same shape. The pattern is:

• Whatever query you want is a for loop over an index range, asking the columns at each index.
• Whatever rearrangement you want is a permutation of the order vector, leaving the columns unchanged.
• A “move” - dealing, discarding - is a write to locations[i], never a copy of the card.

If you find yourself constructing a Card struct to make exercise 8 cleaner, stop. The four hands together are simply a Vec<u8> of length 52 (the existing locations array) with values 0..5. Printing each hand is cards_held_by(&locations, p) for p in 1..=4.

Exercises 9-10

Both are bridges into the next sections.

• Exercise 9 (drop the index) is a preview of nodes 6 (row is a tuple) and the natural-key idea named in the strong-form note above. The (suit, rank) pair is the card; you don’t need an integer to refer to it. But moving it back to the deck is now harder, because you’ve lost the slot reference.
• Exercise 10 (the sort hazard) is the bug that motivates §9 - Sort breaks indices, which in turn motivates §10 - Stable IDs and generations. The bug shows up the moment you sort the data arrays themselves rather than the order vector. You need a stable name for a card that survives reordering - and that name is what node 10 introduces.

6 - A row is a tuple

Concept node: see the DAG and glossary entry 6.

In §5 you built a deck of 52 cards as three Vecs. The card at index 17 is the triple (suits[17], ranks[17], locations[17]). Together those three values are the row. There is no Card struct. The row exists implicitly in the alignment: the same index, used in every column, recovers all the data about one card.

This is what we call a row throughout the rest of the book - a coherent set of values that belong to the same entity. In a creature table the row is (pos[i], vel[i], energy[i], birth_t[i], id[i], generation[i]). In a food table it is (pos[i], value[i], id[i]). The fields belong to the same entity by virtue of all sharing index i. There is no struct holding them; there is only the discipline that whatever index i you used to read one column, you also use to read every other column of the same table.

The cost of implicit binding is that you must keep the indices aligned. If you sort ranks without also sorting suits and locations, you corrupt the column: the rank stored at a slot is no longer the rank of the card whose suit and location sit beside it. The deck still has 52 entries in 52 slots, but the slots no longer describe coherent cards. This is not a hypothetical bug; §9 will produce it deliberately so you can feel the consequences. The structural fix in this book is simple: every operation that reorders any column of a table must reorder all columns of that table together.

The discipline that makes alignment maintainable is single-writer-per-column. If only one system writes to locations, and that system writes consistently, alignment is never violated. Multiple writers to the same column race against each other and produce inconsistent rows. This is what node 25 (one writer, many readers) enforces: each table is written by exactly one system, and a row is a tuple precisely because that one writer kept all its columns in step.

A row is a tuple - assembled from columns indexed by the same entity, kept aligned by discipline rather than by any container holding it together.

Exercises

These extend your src/main.rs from §5.

1. Print row 17. Write fn row(suits: &[u8], ranks: &[u8], locations: &[u8], i: usize) -> (u8, u8, u8). Use it to print the suit, rank, and location of card 17.
2. Mishandle the alignment. Sorting suits here would do nothing - the deck was built with suits already in order, so suits.sort() is a no-op that quietly hides the trap. Sort only ranks instead (ranks.sort(), no order vector), leaving suits and locations untouched. Print rows 5 and 17. Row 5 is plainly broken: its suit and location still belong to card 5, but its rank was overwritten by another card’s. Row 17 looks fine, because slot 17 happens to hold rank 4 both before and after the sort. The column is corrupted either way; the surviving slot only proves that one spot-check is not a test.
3. Lockstep sort. Reset the deck. Now sort all three columns together by rank, using an order vector (the technique from §10): build order, sort it by ranks[i], then rebuild every column through that one order. Print rows 5 and 17 again. Whatever card now sits in a slot, its suit, rank, and location moved there together, so the row is coherent again.
4. Add a fourth column. Add let mut dealt_at: Vec<u32> = vec![u32::MAX; 52]; (when a card is dealt, write the current tick number into dealt_at[i]). Modify your lockstep sort to also reorder this column. Verify by spot-check that a row is still consistent after a sort.
5. The single-writer rule. Write fn reorder_deck(suits: &mut Vec<u8>, ranks: &mut Vec<u8>, locations: &mut Vec<u8>, dealt_at: &mut Vec<u32>, order: &[usize]). This function is the only one that should ever reorder any column of the deck. Document that contract in a comment above the function.
6. (stretch) When alignment is moot. A query that uses only (suits[i], ranks[i]) to identify a card - for instance, “is this the Ace of Spades?” - does not depend on locations or dealt_at. Write such a query. The natural-key view from §5’s strong form means this query survives reorderings of unrelated columns; only suits and ranks need to be aligned with each other.

Reference notes in 06_a_row_is_a_tuple_solutions.md.

What’s next

§7 - Structure of arrays (SoA) names the layout choice you have been making implicitly: each field its own column. The next section defends that choice against its alternative.

Solutions: 6 - A row is a tuple

Exercise 1 - Print row 17

#![allow(unused)]
fn main() {
fn row(suits: &[u8], ranks: &[u8], locations: &[u8], i: usize) -> (u8, u8, u8) {
    (suits[i], ranks[i], locations[i])
}

let (suits, ranks, locations) = new_deck();
let (s, r, l) = row(&suits, &ranks, &locations, 17);
println!("row 17: suit={s} rank={r} location={l}");
}

The function does not look up by id; it looks up by slot. With a fresh deck the slot 17 holds a stable card, but as soon as the deck is sorted or rearranged, the same call returns a different card. That is the §9 lesson; here we only ask the slot what it holds right now.

Exercise 2 - Mishandle the alignment

#![allow(unused)]
fn main() {
// `suits` is already sorted by construction, so `suits.sort()` would change nothing.
// Sort `ranks` in isolation to actually reorder a column:
ranks.sort();
for i in [5, 17] {
    let (s, r, l) = row(&suits, &ranks, &locations, i);
    println!("row {i}: suit={s} rank={r} location={l}");
}
}

After ranks.sort(), slot 5 still reads suit 0 and location 0 - those columns were untouched, so they are card 5’s - but its rank is now 1, dragged in from a different card. The row no longer describes any real card. Slot 17 reads suit 1, rank 4, location 0, exactly as before: the four rank-4 cards occupy slots 16 through 19 both before and after the sort, so this one slot survives by coincidence. Note that only one column moved, so a broken row mixes two cards, not three; the real danger is that inspecting a single lucky slot like 17 would tell you nothing is wrong.

Exercise 3 - Lockstep sort

#![allow(unused)]
fn main() {
let mut order: Vec<usize> = (0..52).collect();
order.sort_by_key(|&i| ranks[i]);
suits     = order.iter().map(|&i| suits[i]).collect();
ranks     = order.iter().map(|&i| ranks[i]).collect();
locations = order.iter().map(|&i| locations[i]).collect();
}

Every column is rebuilt through the same order, so whichever card lands in slot 5 or 17 brings its suit, rank, and location with it. The slot’s three fields agree on one card again. The fix is structural: one order vector, applied to every column, can never produce the mismatch from exercise 2.

Exercises 4-6 - Sketches

Exercise 4. dealt_at is just a fourth column. Add it to the lockstep sort. Spot-check by setting dealt_at[7] = 42 before the sort, then verifying that after the sort dealt_at[new_slot] is still 42 for that same card (find it via the id column from §10).

Exercise 5. The reorder_deck function takes &mut references to all four columns plus &[usize] order. Inside, it does the four iter().map(...).collect() lines. The contract in the comment: “any reorder of the deck must use this function. Direct calls to Vec::sort() or Vec::swap() on individual columns are forbidden, even if they happen to compile.”

Exercise 6. A natural-key query like is_ace_of_spades(s, r) reads only suits[i] and ranks[i] without caring what locations[i] says. The locations column can be reordered independently and the query remains correct - provided suits and ranks stay aligned with each other. Two-of-three alignment is sometimes acceptable; full alignment is the only state in which all queries are valid. Reasoning about partial alignment is fragile and rarely worth the complexity.

7 - Structure of arrays (SoA)

Concept node: see the DAG and glossary entry 7.

Your deck has three Vecs: suits, ranks, locations. Each field lives in its own array, indexed by entity. This layout is called Structure of Arrays - SoA. The opposite layout - a single Vec<Card> where each element is a struct holding all three fields - is called Array of Structs - AoS. They are different choices about where the same data lives.

#![allow(unused)]
fn main() {
// SoA: three columns, indexed in lockstep
let suits:     Vec<u8> = vec![/* 52 */];
let ranks:     Vec<u8> = vec![/* 52 */];
let locations: Vec<u8> = vec![/* 52 */];

// AoS: one column of structs
struct Card { suit: u8, rank: u8, location: u8 }
let cards: Vec<Card> = vec![/* 52 */];
}

Most programmers reach for AoS by default because it groups “related” data together. The trouble is that in a real loop “related” is whatever the inner loop reads, not whatever the data model says belongs together. A system that counts cards in player 1’s hand reads only locations - it does not need suits or ranks at all. With SoA, that loop reads exactly 52 bytes from locations. With AoS, the loop reads all three bytes of each Card (because they live next to each other in memory and arrive on the same cache line) and ignores two of them - three times the memory traffic for the same answer.

At 52 cards the difference is invisible. At one million creatures with six fields each, the difference is the difference between a 30 Hz simulation and a 5 Hz one¹. The motion system in §1’s simulator reads only pos, vel, and energy - three of six creature fields. With SoA it reads three sequential streams of exactly the bytes it needs. With AoS it reads all six fields of every creature, paying twice the memory bandwidth for half the data it actually wants.

This is the bandwidth-bound regime named in §4. SoA keeps the inner loop’s working set small; AoS bloats it with fields the loop ignores. At cache-spilling sizes (any working set bigger than L3) the bloat becomes the dominant cost.

SoA is therefore the default in this book. AoS is sometimes the right choice - for example when every system reads every field, or when N is so small the cache line is dominated by per-row overhead either way. But this is a tradeoff to earn by measurement, not to assume by habit. Write SoA first; switch to AoS only when a benchmark forces you to.

Measurements

How far SoA beats a padded array-of-structs grows with the cache budget: the small-cache Pi pays for every wasted byte (5.7x), a modern desktop with generous L3 mutes it (1.6x). The principle holds on every machine. Full output: code/README.md.

Exercises

You will need a stopwatch (std::time::Instant) for some of these.

1. Build both layouts. Take your §5 deck and add an AoS twin: a Vec<Card> of 52 entries, where Card { suit: u8, rank: u8, location: u8 }. Build both and verify they hold the same logical content.
2. Count cards in a player’s hand, both ways. Write fn count_held_soa(locations: &[u8], player: u8) -> usize and fn count_held_aos(cards: &[Card], player: u8) -> usize. Confirm they return the same number on the same deck.
3. Time the count at 10,000 entries. Make Vec<u8> and Vec<Card> of length 10,000 (replicate the deck 192-fold, or fill arbitrarily). Time each count_held_* function. Note the ratio.
4. Scale to 1,000,000 entries. Repeat at length 1,000,000. The SoA version reads 1 MB; the AoS version reads 3 MB (assuming size_of::<Card>() == 3 plus padding). On most chips L2 fits one but not the other. Note where the cliff appears.
5. The wide-field case. Extend the row with a 16-byte nickname: [u8; 16]. Rebuild both. Now AoS reads 19+ bytes per element while SoA still reads 1. Time the count again. The gap should widen sharply.

Note - How sharp depends on your memory hierarchy. Measured ratios at N=10M: ~2× on machines with generous L3 (modern desktops, mid-2010s Intel laptops), ~6× on a Raspberry Pi 4 (no L3, narrow LPDDR4 channel). The principle is the same; the slope of the cliff scales with how badly the AoS row blows the cache budget.

6. A case where AoS wins. Write a function that updates every field of one specific card. SoA writes to three different lines; AoS writes to one. For the case “update every field of every card” (rare in practice), AoS may even tie or win. Time it and discuss.
7. (stretch) A from-scratch SoaDeck struct. Wrap the three (or four) columns in one struct that owns them all. Provide fn reorder(&mut self, order: &[usize]) as the only public mutator. What do you gain in correctness? What do you lose in flexibility?

Reference notes in 07_structure_of_arrays_solutions.md.

What’s next

§8 - Where there’s one, there’s many is the universalising principle. The deck taught it implicitly; the next section names it.

Solutions: 7 - Structure of arrays (SoA)

Exercise 1 - Build both layouts

#![allow(unused)]
fn main() {
struct Card { suit: u8, rank: u8, location: u8 }

// SoA
let (suits, ranks, locations) = new_deck();

// AoS
let cards: Vec<Card> = (0..52)
    .map(|i| Card {
        suit: suits[i],
        rank: ranks[i],
        location: locations[i],
    })
    .collect();
}

Note std::mem::size_of::<Card>() is 3 in theory but Rust may pad to 4 for alignment unless you #[repr(packed)] (which has its own hazards). For the timing exercises below, that 3-vs-4 detail does not change the qualitative result.

Exercises 2-3 - Counting and timing

#![allow(unused)]
fn main() {
fn count_held_soa(locations: &[u8], player: u8) -> usize {
    let mut n = 0;
    for &l in locations { if l == player { n += 1; } }
    n
}

fn count_held_aos(cards: &[Card], player: u8) -> usize {
    let mut n = 0;
    for c in cards { if c.location == player { n += 1; } }
    n
}
}

Both compile to tight loops. The SoA loop reads one byte per iteration; the AoS loop reads sizeof(Card) bytes per iteration even though it only inspects one field. For 10,000 entries the AoS loop is roughly 3-4× slower; for 1,000,000 the gap widens further as the AoS working set spills more cache levels.

Exercise 4 - The cliff

At 1,000,000 entries: SoA is 1 MB (fits L2 on most chips); AoS at 4 bytes/Card is 4 MB (out of L2, into L3). At 10,000,000 SoA still fits L3; AoS does not. Each cache transition is a sharp slowdown for AoS while SoA continues at near-L2 speed.

Exercise 5 - The wide-field case

Add nickname: [u8; 16] to Card. Now size_of::<Card>() ≥ 19 (will pad to 20 or 24 for alignment). The AoS count loop reads 24 bytes per element while SoA still reads 1. The ratio is no longer ~3-4× - it is ~20×. This is the whole point of SoA: a field a loop does not read sits in its own column, untouched, costing no bandwidth. There is no row to drag it along with - which is exactly why a structure-of-arrays world never needs a separate hot/cold field split.

Exercise 6 - When AoS wins

#![allow(unused)]
fn main() {
fn touch_all_fields_aos(cards: &mut [Card], i: usize) {
    cards[i].suit = (cards[i].suit + 1) % 4;
    cards[i].rank = (cards[i].rank + 1) % 13;
    cards[i].location = 0;
}

fn touch_all_fields_soa(suits: &mut [u8], ranks: &mut [u8], locations: &mut [u8], i: usize) {
    suits[i]     = (suits[i] + 1) % 4;
    ranks[i]     = (ranks[i] + 1) % 13;
    locations[i] = 0;
}
}

For a single card, AoS touches one cache line; SoA touches three (one per column). For a million cards iterated in order, both layouts stream their data through cache and the difference is small - but the SoA version uses 3× the cache lines per iteration. This is the reverse of exercise 2 and is the case where AoS may win.

In practice this case is rarer than it sounds. Most systems read or write only a subset of fields per pass; the situations where every field is touched together usually live at the boundary (deserialise the row, write it to disk) rather than in the inner loop.

Exercise 7 - SoaDeck sketch

#![allow(unused)]
fn main() {
struct SoaDeck {
    suits:     Vec<u8>,
    ranks:     Vec<u8>,
    locations: Vec<u8>,
}

impl SoaDeck {
    fn reorder(&mut self, order: &[usize]) {
        self.suits     = order.iter().map(|&i| self.suits[i]).collect();
        self.ranks     = order.iter().map(|&i| self.ranks[i]).collect();
        self.locations = order.iter().map(|&i| self.locations[i]).collect();
    }
    // Read accessors; no per-column mutators exposed.
}
}

What you gain: the only way to reorder is through reorder, which takes all columns at once - alignment is enforced by the type system. What you lose: a system that wants to mutate just locations cannot do so without going through the wrapping struct or breaking the encapsulation. You have moved the alignment discipline from a code-review concern to a type-system concern, at the cost of some flexibility. This is mechanism-vs-policy (§40) - choose where the rule lives.

8 - Where there’s one, there’s many

Concept node: see the DAG and glossary entry 8.

Code is written for the array. A function that operates on one entity is just the special case of N = 1; it does not need its own abstraction. A card game with 52 cards is three arrays - suit, rank, location - not 52 objects. A simulation with 100 creatures is six arrays of length 100, not 100 instances of Creature. The plural is the primary unit; the singular is the trivial case.

The pattern is simple. Write the array version first. The singleton drops out as a one-element slice. To shuffle one card you swap two indices in the order vector - same as shuffling the whole deck. To find the highest-rank card in player 1’s hand you scan the (small) hand vector - same shape as scanning all 52. To deal one card you write one cell in locations - same shape as dealing many cells.

This stands against an instinct most programmers acquire from OOP: the urge to write card.shuffle() or creature.update() and then puzzle over how to do it for many. The puzzle does not exist when you write for arrays from the start. shuffle(&mut deck) is one function that works for any deck, including a deck of one. update(&mut creatures) is one function that works for any population, including a population of one.

A useful test: when you find yourself writing a method on a struct, ask what does this look like over an array? If the array version is shorter, drop the method. If the array version is the same length, keep the method as a function over a slice - fn shuffle(deck: &mut Deck), not impl Deck { fn shuffle(&mut self) }. Either way, the singleton was never the right unit of code.

There is also a cost reason, though it does not bite at 52 cards. A method that runs on one entity at a time forces its caller to invoke it N times: N opaque calls the optimiser cannot fuse into a loop. A function over a slice is one call - the compiler sees the whole loop and can lift invariants, reorder, and vectorise it. Writing for the array keeps the work visible to the optimiser; writing for the singleton hides it. The bill for that hiding does not arrive until the simulator is walking a million rows a tick, and §19 measures it there. At deck scale this is a reason to prefer the array form, not yet a speed you can feel.

“Where there’s one, there’s many” is therefore not an architectural slogan but a daily practice. It costs nothing the first time. It costs everything the first time you forget.

Exercises

These extend the deck again. The aim is to feel the array-first pattern in your fingertips before §5 turns into the rest of the book.

1. The function over a slice. Write fn highest_rank_in_hand(hand: &[u32], ranks: &[u8]) -> Option<u8> returning the highest rank held in the supplied set of card ids. Use it on a 5-card hand. Then use it on a 1-card hand. Then use it on an empty hand. Same function, three N values.
2. Reverse the urge. Given an OOP-style Card::is_face_card(&self) -> bool, rewrite it as fn face_cards(ranks: &[u8]) -> Vec<bool> - a function over the whole ranks array returning a parallel mask. Apply it to all 52 cards in one call.
3. The N = 0 case. What does highest_rank_in_hand do for an empty hand? Should it panic, return None, or return some sentinel? Pick one and justify.
4. Predicate over a single value. Suppose you want is_red(suit: u8) -> bool for a single card (suits 0 and 1 are hearts/diamonds). Write the array version fn red_mask(suits: &[u8]) -> Vec<bool> first. Then convince yourself the singleton case is red_mask(&[suit])[0] - the array version covers it.
5. (stretch) From a tutorial. Find any Rust tutorial that uses a struct Card with methods (new, is_face, display, etc.). Rewrite their full card game as three (or four) Vecs plus free functions. Compare line counts. Compare clarity. Compare what happens when you want to query “all face cards across the table” - one function call versus a loop over per-card method calls.

Reference notes in 08_where_theres_one_theres_many_solutions.md.

What’s next

You have closed Identity & structure. Cards behave; rows align; layouts are SoA; the singleton drops out. The next phase is Time & passes, starting with §11 - The tick. The ecosystem simulator from code/sim/SPEC.md is about to start running.

Solutions: 8 - Where there’s one, there’s many

Exercise 1 - The function over a slice

#![allow(unused)]
fn main() {
fn highest_rank_in_hand(hand: &[u32], ranks: &[u8]) -> Option<u8> {
    let mut best: Option<u8> = None;
    for &id in hand {
        let r = ranks[id as usize];
        best = match best {
            None => Some(r),
            Some(b) => Some(b.max(r)),
        };
    }
    best
}

let hand5: Vec<u32>   = vec![3, 17, 21, 28, 41];
let hand1: Vec<u32>   = vec![41];
let hand0: Vec<u32>   = vec![];

println!("{:?}", highest_rank_in_hand(&hand5, &ranks)); // Some(some rank)
println!("{:?}", highest_rank_in_hand(&hand1, &ranks)); // Some(ranks[41])
println!("{:?}", highest_rank_in_hand(&hand0, &ranks)); // None
}

One function. Three N values. The N = 1 and N = 0 cases are not special-cased; they fall out.

Exercise 2 - Reverse the urge

#![allow(unused)]
fn main() {
fn face_cards(ranks: &[u8]) -> Vec<bool> {
    ranks.iter().map(|&r| r >= 10).collect() // 10 = J, 11 = Q, 12 = K (0-indexed)
}
}

Compared to a per-card Card::is_face_card(&self) -> bool plus a loop, the array version is shorter, more cache-friendly, and trivially vectorisable.

Exercise 3 - The N = 0 case

Option<u8> returning None is the cleanest answer. A panic is hostile to callers (the empty case is a valid state of the world - a player just played their last card). A sentinel like 255 confuses the type. Option makes the absence visible at the type level.

Exercise 4 - Singleton as trivial array

#![allow(unused)]
fn main() {
fn red_mask(suits: &[u8]) -> Vec<bool> {
    suits.iter().map(|&s| s < 2).collect() // suits 0,1 = hearts, diamonds
}

let one_suit = 0u8;
let is_red_one = red_mask(&[one_suit])[0]; // true
}

The singleton drops out as a one-element call. There is no separate is_red(suit: u8) -> bool function. If the call site is ergonomic enough you may write a thin wrapper for clarity, but the array version is the canonical implementation.

Exercise 5 - From a tutorial

This is open-ended. The expected outcome: the rewritten array-first version is shorter, has fewer indirections, and answers cross-cutting queries (all face cards on the table; all spades in any hand) in one function call rather than a loop over methods.

A typical OOP card game tutorial weighs around 200-400 lines. The array-first rewrite of the same functionality usually lands at 80-150 lines, with the bulk of the savings coming from not writing getters, setters, copy semantics, and the various small accessors that an OOP Card accumulates.

9 - Sort breaks indices

Concept node: see the DAG and glossary entry 9.

In §5 - Identity is an integer, exercise 10 left you with a bug. Player 1 was holding the index list [3, 17, 21, 28, 41]. The dealer sorted the deck columns by suit. Player 1’s hand was now wrong - the same indices, the same slots, but different cards.

That bug is the structural fact this section names. Sorting did not damage anything; the player’s reference was never robust to begin with. An index points at a slot, not at a thing. When the slot’s contents change, the index quietly changes meaning.

It is not only sorting. Any rearrangement does it: swap_remove (a O(1) deletion that moves the last row into the freed slot, coming in §21), reshuffling for locality (§28), compacting after a batch of deletions. The same index, the same array, the same line of code, now means a different card.

This is uncomfortable. In OOP you held a Card reference and the card stayed put because Card was a thing. In data-oriented code the card is the slot, and the slot does not have permanent meaning. The card you saved a reference to yesterday may be a different card today, if the deck has been touched.

There are two ways forward. The lazy one is to never rearrange the deck. That works for fifty-two cards, fails for ten thousand creatures, and becomes catastrophic for a million. The book is going to need rearrangements - sorting, deletion, compaction - at every scale beyond §0. So we need the other fix: a stable name that survives the slot it currently occupies.

That is what §10 - Stable IDs and generations does. This section’s only job is to make the slot vs name distinction concrete enough that §10’s solution feels inevitable rather than ceremonial.

Note - Why feel the pain first? Because the fix in §10 is small - one extra column - and small fixes only stick if the student knows what they fix. Reading “always store an id” without first feeling the bug produces students who add ids cargo-culted, then drop them when the codebase looks too cluttered. Reading it after watching player 1 lose their hand produces students who never drop them.

Exercises

You should still have your src/main.rs from §5. These exercises extend it.

1. Reproduce the bug. With player 1 holding [3, 17, 21, 28, 41], sort the deck columns themselves (suits, ranks, and locations in lockstep) by suit. Print player 1’s hand using card_to_string. Confirm the cards have changed.
2. A second rearrangement. Instead of sorting, swap two cards’ positions:

   #![allow(unused)]
   fn main() {
   suits.swap(3, 17);
   ranks.swap(3, 17);
   locations.swap(3, 17);
   }

   Print player 1’s hand again. Same bug shape, different cause.
3. A third rearrangement. Remove the card at slot 3 with swap_remove(3) on each column. Print player 1’s hand. Note that the cards at slots [17, 21, 28, 41] are unchanged but slot 3 may now hold what was previously the last card; meanwhile slot 51 has silently been deleted.
4. Quantify the breakage. Write a function that takes the original [3, 17, 21, 28, 41] plus a freshly built deck, applies a Fisher-Yates shuffle to the deck columns themselves, and counts how many of the five references still point at the same (suit, rank) value. Run it 100 times. Roughly what fraction of references survive a random shuffle of the deck?
5. A reference that can survive. Without writing any new code - on paper - describe what kind of reference would survive a shuffle. (Hint: you already know. The card’s (suit, rank) is unique to that card. The reference that survives is the one that does not depend on the slot.)
6. (stretch) The cost of never rearranging. Suppose you decide to never sort, swap, or remove from the deck columns, to avoid this bug forever. How would shuffling work? How would discarding a card work? Why does this not scale to ten thousand creatures?

Reference notes for these exercises in 09_sort_breaks_indices_solutions.md.

What’s next

Exercise 5 points at the answer; exercise 6 makes the never-rearrange option look bad. The real fix is to store identity separately from position - an id column that travels with the row across rearrangements, with a generation counter on top for variable-quantity tables. §10 - Stable IDs and generations builds it.

Solutions: 9 - Sort breaks indices

Reference notes for the exercises in 09_sort_breaks_indices.md. These exercises are mostly about observing the bug; there is little new code. Work them first - the lesson is felt, not read.

Exercise 1 - Reproduce the bug

#![allow(unused)]
fn main() {
let (mut suits, mut ranks, mut locations) = new_deck();
let player_1_hand = vec![3, 17, 21, 28, 41];

// Sort the deck columns themselves (in lockstep) by suit.
let mut order: Vec<usize> = (0..52).collect();
order.sort_by_key(|&i| suits[i]);
suits     = order.iter().map(|&i| suits[i]).collect();
ranks     = order.iter().map(|&i| ranks[i]).collect();
locations = order.iter().map(|&i| locations[i]).collect();

for &i in &player_1_hand {
    println!("{}", card_to_string(suits[i], ranks[i]));
}
}

Compare what prints now with what printed before the sort. The slots [3, 17, 21, 28, 41] still exist; their contents moved.

Exercise 4 - Quantify the breakage

After a uniform random shuffle, a fixed slot has a 1-in-52 chance of holding the same card it held before. Five references, five independent draws - expected survivors per shuffle is roughly 5/52 ≈ 0.10. Most runs have zero references survive. The bug is total, not partial.

Exercise 5 - A reference that can survive

The pair (suit, rank) is unique to each card. A reference that stores (suit, rank) rather than a slot index survives any rearrangement of the deck columns, because the pair is natural - it lives in the data itself, not in the slot. This is the strong form of §5 - Identity is an integer.

The price of natural keys is paid in §10: returning Vec<(u8, u8)> from a query is fine for reading, but to move a card (deal, discard) you still need to know which slot to write to. Surrogate ids generalise to cases where the data has no natural unique tuple.

Exercise 6 - The cost of never rearranging

Without rearranging:

• Shuffling must be implemented as a permutation in order: Vec<usize> while the deck columns themselves stay in their original layout. This is what §5 actually did. It works.
• Discarding means writing locations[i] = DISCARD rather than removing the row. Logical removal via a flag, not structural removal. The deck never shrinks.

For 52 cards the deck-never-shrinks rule is fine. For 10,000 creatures with steady birth and death the table grows without bound; every system scans past dead rows; cache traffic doubles, then quadruples. The fix in §21 - swap_remove needs the stable references that §10 introduces in order to be safe to call.

10 - Stable IDs and generations

Concept node: see the DAG and glossary entry 10.

In §9 you watched a player’s reference go stale because they were holding slots, not names. The fix is to give each row a name - a stable identifier - that travels with the row when it moves.

A stable id is one extra column. For the deck:

#![allow(unused)]
fn main() {
let mut ids: Vec<u32> = (0..52).collect();
}

Now every card has both a slot (its current index in the columns) and an id (its name). When you sort the columns, you reorder ids in lockstep:

#![allow(unused)]
fn main() {
// sort by suit, taking ids along
let mut order: Vec<usize> = (0..52).collect();
order.sort_by_key(|&i| suits[i]);

let new_suits:     Vec<u8>  = order.iter().map(|&i| suits[i]).collect();
let new_ranks:     Vec<u8>  = order.iter().map(|&i| ranks[i]).collect();
let new_locations: Vec<u8>  = order.iter().map(|&i| locations[i]).collect();
let new_ids:       Vec<u32> = order.iter().map(|&i| ids[i]).collect();
}

The card with id = 17 is still the same card - its suit, rank, and location are unchanged. It is just at a different slot.

To find a card by id, scan the ids column:

#![allow(unused)]
fn main() {
fn slot_of(ids: &[u32], target: u32) -> Option<usize> {
    for i in 0..ids.len() {
        if ids[i] == target {
            return Some(i);
        }
    }
    None
}
}

That is O(N), which is fine for a 52-card deck and slow for a million creatures. The fix - an id_to_slot map maintained on every rearrangement - is §23 - Index maps. For now the linear scan is honest pedagogy.

Generations: when slots are reused

The deck is constant-quantity. Always 52 cards, never more, never less. The simple id column is enough.

For variable-quantity tables - creatures that are born and die, packets that arrive and are processed, sessions that come and go - slots get reused. A new creature is born in the slot that just held a dead one. Now imagine a player who held a reference to the dead creature: their reference points at the same slot with the same id, but the row at that location is a different creature.

One more column fixes it: a generation counter that increments every time a slot is recycled. A reference is now a pair (id, generation). To dereference it, you check that the row’s stored generation still matches the reference’s generation. If it does, the reference is live. If it does not, the slot has been recycled since the reference was taken, and the dereference returns None.

#![allow(unused)]
fn main() {
struct CreatureRef {
    id:  u32,
    generation: u32,
}

fn get(creatures: &Creatures, r: CreatureRef) -> Option<usize> {
    let slot = creatures.id_to_slot.get(r.id as usize).copied()?;
    if creatures.generation[slot] == r.generation {
        Some(slot)
    } else {
        None
    }
}
}

This is the pattern called a generational arena. It is the single mechanism behind every “handle” type in every ECS engine: Bevy’s Entity, slotmap::SlotMap, C++’s entt::registry. They differ in details - width of the id, packing into a u64, generation overflow handling - but the structural idea is the same: one column for identity, one for generation, a checked dereference.

That is enough machinery for the rest of the book to lean on. Sorting now works because the id column travels with the row. Deletion now works because the generation counter rejects stale references. Append-only and recycling tables (§24) are two policies on the same machinery.

Note - The strong form of §5 still applies. If your row has a natural key - (suit, rank), (date, ticker), (species, position) - you do not need a surrogate id. The card-game deck can be played without ids; the reference that survives is the (suit, rank) pair, because the data is unique by construction. Surrogate ids and generations earn their keep when the data has no natural unique tuple - which is most of the time once you start producing rows at runtime.

Exercises

These extend the §5 deck once more, then take a step toward the simulator’s variable-quantity case.

1. Add the id column. Add let ids: Vec<u32> = (0..52).collect(); to your deck. Modify your sort so it reorders ids along with the other columns. Verify the original ids are still there, just in a new order.
2. Find a card by id. Implement slot_of(ids: &[u32], target: u32) -> Option<usize> as in the prose. Use it to look up the card with id = 17 after a sort.
3. Resolve the §9 bug. With player 1 holding ids [3, 17, 21, 28, 41] (not slots), sort the deck. Use slot_of to translate ids to slots and print the hand. Confirm the cards are unchanged.
4. Permutation-friendly hand query. Rewrite cards_held_by(locations, ids, player) -> Vec<u32> to return ids, not slots. The player now holds names. Test by sorting the deck after a deal and confirming cards_held_by still returns the same five cards.
5. A first generation counter. Add let mut generation: Vec<u32> = vec![0; 52];. The 52-card deck does not actually recycle, but extend a small swap_remove-like operation: pop the last card from the deck (location 0), insert a “fresh” card at the freed slot, and bump that slot’s generation by one. Take a CreatureRef-style (id, generation) reference before the operation. After the operation, look up the slot by id; check generation[slot] against the reference’s generation. Confirm the dereference correctly reports stale.
6. (stretch) A tiny generational arena. Outside the deck, build a Creatures struct with pos: Vec<f32>, generation: Vec<u32>, plus free: Vec<u32> of slots awaiting reuse. Implement insert(pos) -> (slot, generation), remove(slot), and get(slot, generation) -> Option<f32>. Convince yourself by example that stale references cannot read a fresh creature’s data.
7. (stretch) Compare with slotmap. Read slotmap::SlotMap::insert and get. Identify which of your fields and operations correspond. What does slotmap add that you didn’t need for the simulator? Decide consciously whether to adopt it. (This is the from-scratch-then-price-the-crate move from §41 - Deferred abstraction and §42 - You can only fix what you wrote.)

Reference solutions for the deck exercises (1-5) in 10_stable_ids_and_generations_solutions.md. The arena and slotmap exercises follow the same shape and are worth working without reference.

What’s next

You now have stable references. The next thing the simulator will need is to look up a row by id in O(1) rather than O(N) - an id_to_slot map maintained on every reordering. That is §23 - Index maps. It is one extra Vec<u32>, updated whenever the columns move.

Solutions: 10 - Stable IDs and generations

Reference solutions for the exercises in 10_stable_ids_and_generations.md. The deck exercises (1-3) are full; the rest are sketches.

Exercise 1 - Add the id column

#![allow(unused)]
fn main() {
fn new_deck_with_ids() -> (Vec<u8>, Vec<u8>, Vec<u8>, Vec<u32>) {
    let mut suits     = Vec::with_capacity(52);
    let mut ranks     = Vec::with_capacity(52);
    let mut locations = Vec::with_capacity(52);
    for s in 0..4u8 {
        for r in 0..13u8 {
            suits.push(s);
            ranks.push(r);
            locations.push(0);
        }
    }
    let ids: Vec<u32> = (0..52).collect();
    (suits, ranks, locations, ids)
}

fn sort_deck_by_suit(
    suits: &mut Vec<u8>,
    ranks: &mut Vec<u8>,
    locations: &mut Vec<u8>,
    ids: &mut Vec<u32>,
) {
    let mut order: Vec<usize> = (0..suits.len()).collect();
    order.sort_by_key(|&i| suits[i]);
    *suits     = order.iter().map(|&i| suits[i]).collect();
    *ranks     = order.iter().map(|&i| ranks[i]).collect();
    *locations = order.iter().map(|&i| locations[i]).collect();
    *ids       = order.iter().map(|&i| ids[i]).collect();
}
}

The four columns are reordered in lockstep. Failing to reorder one of them is the bug.

Exercise 2 - Find a card by id

#![allow(unused)]
fn main() {
fn slot_of(ids: &[u32], target: u32) -> Option<usize> {
    for i in 0..ids.len() {
        if ids[i] == target {
            return Some(i);
        }
    }
    None
}
}

The for-loop is intentional. Iterators with position do the same thing in one line; the loop version is the one to read first.

Exercise 3 - Resolve the §9 bug

#![allow(unused)]
fn main() {
let (mut suits, mut ranks, mut locations, mut ids) = new_deck_with_ids();

// Player 1 holds *ids*, not slots
let player_1_hand: Vec<u32> = vec![3, 17, 21, 28, 41];

sort_deck_by_suit(&mut suits, &mut ranks, &mut locations, &mut ids);

for id in &player_1_hand {
    let slot = slot_of(&ids, *id).expect("card vanished");
    println!("{}", card_to_string(suits[slot], ranks[slot]));
}
}

The output is the same five cards as before the sort - different slots, same cards. The §9 bug is gone.

Exercises 4-5 - Sketches

Exercise 4. cards_held_by(locations, ids, player) -> Vec<u32> walks the rows in lockstep and pushes ids[i] (not i) when locations[i] == player. Apply any rearrangement; the function still returns the same five ids. Using slot_of afterwards finds the cards.

Exercise 5. Take a (id, generation) reference before the swap-and-bump. After the operation, find the slot by id and read generation[slot]. The slot is the same; generation[slot] is 1 instead of 0; the reference’s generation is 0; the dereference reports stale. The 52-card deck does not feel motivated yet - the simulator’s creature table in §1 is where this stops feeling ceremonial.

Exercise 6 - A tiny generational arena

The shape:

#![allow(unused)]
fn main() {
struct Creatures {
    pos:        Vec<f32>,
    generation:        Vec<u32>,
    id_to_slot: Vec<u32>, // id i -> current slot, or u32::MAX when removed
    free:       Vec<u32>, // slots awaiting reuse
    next_id:    u32,
}
}

insert(pos) either pops a slot from free (bumping generation[slot]) or pushes a new slot. remove(slot) pushes the slot into free and bumps generation[slot]. get(slot, generation) returns Some(pos[slot]) only if self.generation[slot] == generation. The exercise is worth coding; the shape above is enough scaffolding.

Exercise 7 - Comparing with slotmap

slotmap::SlotMap does the same thing with prettier ergonomics: keys pack (slot, generation) into a u64, the API uses Index/IndexMut, removals return the removed value, iterators are provided. None of these are required for the simulator; they are nice. Whether to adopt depends on whether you trust the crate to keep working (§42). The from-scratch version above is small enough that you can fix it yourself if it ever breaks - which is the only reason to choose it over slotmap.

11 - The tick

Concept node: see the DAG and glossary entry 11.

A program’s life has a shape:

• Start-up - initialisation. Tables are allocated, inputs are opened, the RNG is seeded, the world reaches a known state.
• Steps - ticks of the clock in a simulation, turns in a card game, event handlers in a server. The repeating unit of forward motion.
• Save and load - the in-memory state is preserved to disk so a future run can resume from where this one left off. Optional, but if you want it, it lives here.
• Exit - resources are returned to the kernel. Memory, file handles, sockets, lockfiles. Failure to do this cleanly is called a memory leak (or a stale lock, or a broken socket).

This section is about the step. The step is where the time budget binds, where the systems - the functions that read and write the world’s tables - run in the order set by their DAG, where determinism either holds or breaks. The other phases are real and important - the book returns to save and load when persistence is named at §36, and exit is mostly the operating system’s job - but the inner step is what makes or breaks every other property the book builds on.

Each step is a tick. State at the start of a tick is read; state at the end is written; nothing is half-updated mid-tick. Even an interactive program - a card game waiting for the next move, a text editor waiting for a keystroke - is a tick loop, just with an external trigger driving it. A program that does a single pass over a file and exits is a degenerate tick loop with one tick: it has the same start-of-tick / end-of-tick contract, just with N=1.

Ticks come in two natural shapes.

A time-driven tick fires at a fixed rate. The simulator from code/sim/SPEC.md runs at 30 Hz: one tick every 33 ms. The loop wakes up, advances every system by one step, sleeps until the next tick. Most simulations, games, control loops, audio engines, and animation systems are time-driven. The rate is a contract with the rest of the world: at this rate, output appears.

A turn-based tick fires when an event arrives. A card game ticks when a player makes a move. A chess engine ticks when its opponent moves. A discrete-event simulator ticks at the timestamp of the next pending event, however far in the future that is. The clock advances with the events, not under them. Turn-based ticks have no fixed rate; their pace is set by the input stream.

Both are ticks. The difference is what triggers the next pass:

#![allow(unused)]
fn main() {
// time-driven
use std::time::{Duration, Instant};
const TICK: Duration = Duration::from_millis(33);

loop {
    let start = Instant::now();
    run_all_systems(&mut world);
    let elapsed = start.elapsed();
    if elapsed < TICK {
        std::thread::sleep(TICK - elapsed);
    }
}
}

#![allow(unused)]
fn main() {
// turn-based
loop {
    let event = wait_for_next_event();
    apply_event(&mut world, event);
}
}

The §0 simulator runs time-driven. The card game from §5 ran turn-based - every card you dealt was one tick. Both are valid; both fit the same framework.

Within each tick, the systems run in an order specified by the system DAG (§14’s topic). Each tick has a budget: 33 ms at 30 Hz, the ms-per-move in a card game played at human speed. The budget binds the design: at 30 Hz with 1 000 000 creatures, each motion update has 33 nanoseconds, which only fits if the data layout cooperates (§4 made this precise).

When a tick runs long, the budget has a visible failure mode: the frame is dropped. The loop wakes late, the rate sags below its contract, and downstream something stutters. That visibility is worth instrumenting from the first day, because it is the cheapest operations tool you will ever have. Time each tick, and when it overruns the budget, raise in development so a regression stops the build, and warn and count in production so a degraded run is recorded rather than silent. The count of late ticks is the first number an operator reads when asked whether the thing is keeping up. The book returns to this in Part II as the front of the operations toolkit; here it is just a habit, that a tick which can blow its budget should be able to say so.

A subtle pitfall worth naming. Mixing turn-based and time-driven thinking in the same loop produces drift: the turn-based subsystem’s pace bleeds into the time-driven subsystem’s budget. The fix is to keep the two cleanly separated - typically, one outer loop and the other as an event source feeding it.

A tick is the unit of forward motion in any program that has forward motion. The next sections name what fits in one tick, in what order, and what does not.

Exercises

You will need a minimal Rust project for these. cargo new tick_lab is enough.

1. A 30 Hz time-driven loop. Write a main that loops at 30 Hz. Each iteration, print the elapsed time since program start. Sleep between ticks to maintain the rate. Run it for 10 seconds. Did you actually get 300 iterations?
2. The naive sleep mistake. Replace your sleep logic with std::thread::sleep(Duration::from_millis(33)) (no measurement). Run for 30 seconds. Does the program drift over time? Why?
3. Dropped frames. Inside the loop, sleep for 50 ms - longer than the budget. The loop is now running at 20 Hz; it has missed frames. Print a warning when this happens, and keep a running count of late ticks - that count is the first number an operator reads when asked whether the loop is keeping up.
4. A turn-based loop. Write a tiny REPL: print > , read a line, print you said: <line>. Each line is one tick. Run it. Note that the loop has no fixed rate - its pace is your typing.
5. Mixing the two. Modify exercise 4 so that, while waiting for input, the program also prints the current second once per second. (Hint: spawn a thread, use a non-blocking read, or interleave with timeouts.) Note how mixing the two patterns adds complexity quickly.
6. (stretch) A discrete-event tick loop. Maintain a Vec<(f64, String)> of (timestamp, message) events. Pop the smallest-timestamp event, advance a “simulation clock” to that timestamp, print the message, repeat until the queue is empty. This is the structure of a discrete-event simulator and a preview of §12.

Reference notes in 11_the_tick_solutions.md.

What’s next

Exercise 6 hints at the next section. The clock can live on the events themselves, independent of how often the loop fires. §12 - Event time is separate from tick time names that separation.

Solutions: 11 - The tick

Exercise 1 - A 30 Hz time-driven loop

use std::time::{Duration, Instant};

const TICK: Duration = Duration::from_millis(33); // ~30 Hz

fn main() {
    let program_start = Instant::now();
    let mut tick = 0u64;
    loop {
        let tick_start = Instant::now();
        println!("tick {tick} at {:?}", program_start.elapsed());
        tick += 1;
        if tick >= 300 { break; }

        let elapsed = tick_start.elapsed();
        if elapsed < TICK {
            std::thread::sleep(TICK - elapsed);
        }
    }
}

Expected: 300 iterations, total wall time ≈ 10 s. Sleeping the remainder of each tick keeps the rate stable; sleeping a fixed 33 ms regardless of work would let drift accumulate.

Exercise 2 - The naive sleep mistake

thread::sleep(Duration::from_millis(33)) sleeps 33 ms in addition to the work the loop did. If each iteration’s work takes 5 ms, the total period is 38 ms = 26 Hz, not 30 Hz. Over 30 seconds the program runs ~780 iterations instead of 900. The drift is the work-per-tick, multiplied by the number of ticks.

Exercise 3 - Dropped frames

Compare tick_start.elapsed() against TICK. If it exceeded TICK, the loop has missed a frame:

#![allow(unused)]
fn main() {
let work = tick_start.elapsed();
if work > TICK {
    eprintln!("missed frame: work {work:?} > tick {TICK:?}");
} else {
    std::thread::sleep(TICK - work);
}
}

A 50 ms sleep blows the 33 ms budget by 17 ms - every tick logs a missed-frame warning. Real simulators count and surface this metric: it is the most direct sign you have left your real-time budget.

Exercise 4 - A turn-based loop

use std::io::{self, BufRead, Write};

fn main() {
    let stdin = io::stdin();
    let mut stdout = io::stdout();
    loop {
        write!(stdout, "> ").unwrap();
        stdout.flush().unwrap();
        let mut line = String::new();
        if stdin.lock().read_line(&mut line).unwrap() == 0 { break; }
        println!("you said: {}", line.trim());
    }
}

The loop’s pace is your typing speed, not the system clock. Each line is exactly one tick.

Exercises 5-6 - Sketches

Exercise 5. Two clean approaches: spawn a thread that prints once per second using mpsc::channel with a one-second timeout, or use mio/tokio for non-blocking stdin. Either works; both add code that has nothing to do with the original logic. The lesson is in the friction.

Exercise 6. Maintain events: Vec<(f64, String)> sorted by timestamp (or use a BinaryHeap). Pop the smallest, advance sim_time to that timestamp, print, repeat. Wall-clock time and sim_time decouple completely - the loop processes events as fast as it can; sim_time advances in jumps determined by the data.

12 - Event time is separate from tick time

Concept node: see the DAG and glossary entry 12.

Most beginners assume the loop’s frequency sets the model’s time resolution. If the loop runs at 30 Hz, surely the model can only resolve events at 1/30 s = 33 ms? This is wrong, and the confusion costs many simulations their precision.

The tick rate is how often the loop runs. It says nothing about what the loop does inside one tick. Inside one tick, the loop can process events at arbitrary timestamps - microsecond, picosecond, whatever the data carries. The clock lives on the events, not on the loop.

Concretely: a 30 Hz loop receiving 1 000 events per tick, each with microsecond-precision timestamps, processes them in timestamp order - applying each event’s effect with the precision the timestamp implies. Output to the rest of the world (rendering, logging, network) happens at 30 Hz, but the physics inside runs at microsecond resolution. The tick is a sampling rate; the events are the actual phenomena.

This is the model used by:

• Discrete-event simulators (queueing networks, traffic, supply chains): events fired at exact times.
• Game replay systems (rollback netcode, multiplayer): events arrive late but with their original timestamps.
• Trade execution engines: orders carry nanosecond timestamps; the loop processes them in order.
• Logic simulators in chip design: gate transitions at picosecond resolution; the simulator advances one transition at a time.

In each case, the tick rate of the host loop is irrelevant to the simulation’s resolution. The data carries the time.

This separation is what makes the simulator’s pending_event table possible. Each tick, the loop builds a list of events that should fire - collisions, eats, reproductions - each tagged with its predicted timestamp. The events fire in timestamp order regardless of which tick they were predicted in. A creature that “would have eaten 2 µs into the tick” has its eat applied at that exact moment, not at the start or end of the tick.

The pitfall is hard-coding the tick interval as the simulation’s clock granularity. Code that says

creature.energy -= 1.0 / 30.0; // "one tick worth of fuel"

is conflating the two clocks. The right shape is

creature.energy -= elapsed_event_seconds * burn_rate;

using the actual elapsed event-time, not the tick interval.

Event time and tick time are decoupled because they answer different questions. Event time answers when did this thing happen. Tick time answers when does the loop wake up. The same model can be sampled at any tick rate the application needs - visualisation at 30 Hz, recording at 60 Hz, fast-forward replay at 1 kHz - without changing what the model means.

Exercises

These extend the discrete-event loop from §11 exercise 6.

1. A tiny event queue. Use Vec<(f64, String)> and Vec::sort_by. Push 10 events with random timestamps in [0, 10] seconds. Pop them in order; print each as [t=<sec>] <message>. Verify the output is timestamp-sorted.
2. The wrong way: tick-rate clock. Run a 30 Hz loop. In each tick, advance a counter by 1.0 / 30.0. Use this counter as your “simulation time”. Try to fire an event at t = 0.005 s (5 ms). What happens? When does the event fire?
3. The right way: timestamp on events. Run the same 30 Hz loop, but each tick pop all events with timestamp ≤ current real time, applied in timestamp order. Fire an event at t = 0.005 s. Show that the event applies at exactly that time, not at the next tick boundary.
4. Sampling at different rates. Run the same model under a 30 Hz loop, then a 60 Hz loop, then a 1 Hz loop. The events should fire at the same simulation times in all three runs (down to whatever precision the loop allows).
5. Float and time. What’s the smallest time step f32 can represent for events at t ≈ 1 hour? At t ≈ 1 day? At t ≈ 1 year? When do you need f64? (See §2.)
6. (stretch) A budget-aware loop. Modify your 30 Hz loop: at the start of each tick, pop events until either (a) the queue is empty or (b) you have used 25 ms of the 33 ms budget. Defer remaining events to the next tick. This is the soft-real-time pattern used in interactive simulators.

Reference notes in 12_event_time_vs_tick_time_solutions.md.

What’s next

§13 - A system is a function over tables introduces the building block of every tick: the system. Read-set in, write-set out, no hidden state, no surprises.

Solutions: 12 - Event time vs tick time

Exercise 1 - A tiny event queue

fn main() {
    let mut state: u64 = 0xC0FFEE;
    let rand = |s: &mut u64| { *s = s.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); *s };

    let mut events: Vec<(f64, String)> = Vec::new();
    for i in 0..10 {
        let t = (rand(&mut state) >> 32) as f64 / u32::MAX as f64 * 10.0;
        events.push((t, format!("event #{i}")));
    }
    events.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());

    for (t, msg) in events {
        println!("[t={t:.4}] {msg}");
    }
}

The events come out timestamp-sorted, even though they were generated in arbitrary order. The sort is the entire trick.

Exercise 2 - The wrong way

A 30 Hz counter advances by 1.0 / 30.0 ≈ 0.0333. Asking it to fire an event at t = 0.005 either fires the event at t = 0.0333 (the first tick boundary that crosses 0.005) or skips it entirely. Either way, the model has lost 28 ms of resolution.

Exercise 3 - The right way

Inside the 30 Hz loop:

#![allow(unused)]
fn main() {
let real_now = program_start.elapsed().as_secs_f64();
while events.first().is_some_and(|e| e.0 <= real_now) {
    let (t, msg) = events.remove(0);
    println!("[event t={t:.6}] {msg}"); // applies at t, not at real_now
}
}

The event at t = 0.005 fires inside whichever tick has real_now >= 0.005 - the first one - and the printed t is 0.005, not the tick boundary. The simulation time is what the data says.

Exercise 4 - Sampling at different rates

Run the same event list through three loops at 30 Hz, 60 Hz, 1 Hz. The events fire at the same t values in all three runs. Only the wall-clock time at which they fire differs. The model is invariant under tick rate.

Exercise 5 - Float and time

f32 has ~7 significant decimal digits. At t ≈ 1 hour = 3600 s, the smallest distinguishable step is roughly 3600 / 10^7 = 0.00036 s = 360 µs. At t ≈ 1 day = 86400 s, ~8.6 ms. At t ≈ 1 year ≈ 3.15 × 10^7 s, ~3.2 s - f32 cannot represent millisecond resolution at year-scale.

f64 has ~15-16 significant digits. At one year, the smallest step is microseconds. For any simulation longer than a few hours of real time at sub-millisecond resolution, use f64 for timestamps.

Exercise 6 - Budget-aware loop

#![allow(unused)]
fn main() {
let tick_start = Instant::now();
let budget = Duration::from_millis(25);
while events.first().is_some_and(|e| e.0 <= sim_now) {
    if tick_start.elapsed() > budget { break; }
    let (t, msg) = events.remove(0);
    apply(msg);
}
}

Events that did not fit are still in the queue; they fire in the next tick. The model degrades gracefully under load instead of stalling the loop.

13 - A system is a function over tables

Concept node: see the DAG and glossary entry 13.

A system is a function that reads from one or more tables and writes to one or more tables. It declares its inputs (the read-set) and its outputs (the write-set). It has no hidden state, no global side effects, no interaction with the outside world during a tick. The signature is the contract.

#![allow(unused)]
fn main() {
fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], dt: f32) {
    for i in 0..px.len() {
        px[i] += vx[i] * dt;
        py[i] += vy[i] * dt;
    }
}
}

Read-set: vx, vy, dt. Write-set: px, py. That is the entire contract. This system can run any time the velocity columns and dt are available and nothing else is writing the position columns.

Every system takes one of three shapes.

An operation is 1→1: every input row produces exactly one output row. motion is an operation: each creature’s position is updated to its new position. Most update functions are operations.

A filter is 1→{0, 1}: every input row produces zero or one output rows. apply_starve (from code/sim/SPEC.md) is a filter: each creature with energy ≤ 0 produces an entry in to_remove; creatures with energy > 0 produce nothing.

An emission is 1→N: every input row produces zero or more output rows. apply_reproduce is an emission: a parent above the energy threshold produces two offspring (a 1→2 emission).

These three shapes are the same shapes a database query takes. SELECT * FROM t WHERE p is a filter, SELECT a + b FROM t is an operation, SELECT explode(arr) FROM t is an emission. A system is a database operation written in Rust against Vecs instead of SQL against tables.

The contract that the system has no hidden state is what makes systems compose. Two systems with disjoint write-sets can run in parallel without coordination (§31). Two systems whose read-set and write-set form a chain must run in order (§14). The contract is the basis for all of this.

Even observability is a system. A debug inspector is a system whose read-set is “all tables” and whose write-set is “nothing”. It runs alongside the others, gathers data for inspection, and produces no side effects on the world. In production it is absent, not gated by a flag - the binary simply does not contain it.

A few patterns to watch for. A function that reads a table, writes to it, and reads it again in the same call is not a system - it has implicit ordering inside the body. Either split it into two systems with explicit ordering, or buffer the writes until the function exits. A function that takes &mut World and mutates whatever it likes is not a system - it has no declared write-set, and you cannot reason about it from its signature.

A system declares its inputs, declares its outputs, and does no more. That is the shape that lets every other discipline in the book work.

Exercises

Use the deck from §5 or the §0 simulator skeleton; either provides enough tables.

1. Identify the shape. Classify each as operation, filter, or emission:
   • Squaring every entry in a Vec<f32>.
   • Filtering even integers from a Vec<u32>.
   • Splitting each string in Vec<String> into words, returning all words.
   • Computing the sum of a Vec<u32>.
2. Write motion as a system. With position columns px, py: Vec<f32> and velocity columns vx, vy: Vec<f32>, write fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], dt: f32). Apply it to 100 creatures with random initial positions and velocities. Print the position of one creature across 10 ticks.
3. Declare the contract. Add doc comments to motion listing its read-set and write-set explicitly. The signature plus the doc comment is the system’s contract.
4. Write a filter. With energy: &[f32], write fn starving(energy: &[f32]) -> Vec<usize> returning the indices where energy[i] <= 0. This is the read-only first half of apply_starve.
5. Write an emission. With parent_energy: &[f32], threshold threshold: f32, write fn reproduce(parent_energy: &[f32], threshold: f32) -> Vec<(usize, f32)> returning, for each parent above threshold, two (parent_index, offspring_energy) entries. This is a 1→2 emission.
6. Observe non-systems. Find a function in your previous work (or any tutorial) that mutates global state, writes to stdout in its body, or takes &mut World. Note what makes it not a system.
7. (stretch) A test as a system. Write fn no_creature_moved_too_far(prev_px: &[f32], prev_py: &[f32], cur_px: &[f32], cur_py: &[f32]) -> Vec<usize>, returning indices where the move was implausibly large. The “test” is just an inspection system reading the world.

Reference notes in 13_system_as_function_solutions.md.

What’s next

§14 - Systems compose into a DAG takes the next step: when many systems run together, how do they fit?

Solutions: 13 - A system is a function over tables

Exercise 1 - Identify the shape

The aggregate case is real but rare in this book - most systems run row-by-row. When you do need an aggregate, treat it as a system whose output is a single-row table.

Exercise 2 - Motion as a system

#![allow(unused)]
fn main() {
/// motion: advance each creature's position by its velocity over `dt` seconds.
///
/// Read-set:  vx, vy, dt
/// Write-set: px, py
fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], dt: f32) {
    assert_eq!(px.len(), vx.len());
    for i in 0..px.len() {
        px[i] += vx[i] * dt;
        py[i] += vy[i] * dt;
    }
}
}

The assert_eq! enforces alignment (§6). Without it, a mismatched-length call silently iterates over the shorter array.

Exercise 4 - Filter

#![allow(unused)]
fn main() {
fn starving(energy: &[f32]) -> Vec<usize> {
    let mut out = Vec::new();
    for i in 0..energy.len() {
        if energy[i] <= 0.0 {
            out.push(i);
        }
    }
    out
}
}

This is the read-only first half of apply_starve. The actual apply_starve would push these indices into to_remove. Splitting the query from the mutation lets you test the query in isolation.

Exercise 5 - Emission

#![allow(unused)]
fn main() {
fn reproduce(parent_energy: &[f32], threshold: f32) -> Vec<(usize, f32)> {
    let mut out = Vec::new();
    for i in 0..parent_energy.len() {
        if parent_energy[i] >= threshold {
            let half = parent_energy[i] / 2.0;
            out.push((i, half));
            out.push((i, half));
        }
    }
    out
}
}

For each parent above threshold, two output rows. A 1→2 emission. The pattern is clear: the output Vec has a variable length depending on how many parents qualified.

Exercises 3, 6, 7 - Sketches

Exercise 3. Doc comments listing read-set and write-set are the system’s contract in machine-readable form. A reader of the function knows exactly what can change.

Exercise 6. Anti-system patterns: fn update(world: &mut World) (no declared write-set), fn step() that touches a static mut (hidden state), fn motion(px: &mut [f32], py: &mut [f32], ...) with an eprintln! inside (side effect - reduces parallelism, harms determinism, makes testing harder).

Exercise 7. A “test” is a system whose write-set is empty (or a small report table). Read pos/vel, output a list of suspicious creatures. Same code path as a debug inspector.

14 - Systems compose into a DAG

Concept node: see the DAG and glossary entry 14.

A program with one system is uninteresting; a program with many systems must say what runs in what order. The order is given by data dependencies: a system that reads a table must run after every system that writes that table within the same tick. No ordering is fixed by intuition; everything is given by the read-sets and write-sets.

Draw the dependency graph. Each system is a node. For every system that reads table T and every system that writes T, draw an edge writer → reader. The result is a directed acyclic graph (the DAG). A topological sort gives a valid execution order: any sort that respects the edges is correct. The program executes one such sort.

The simulator’s tick from code/sim/SPEC.md:

flowchart TB
    food_spawn --> motion
    motion --> next_event
    next_event --> apply_eat
    next_event --> apply_reproduce
    next_event --> apply_starve
    apply_eat --> cleanup
    apply_reproduce --> cleanup
    apply_starve --> cleanup
    cleanup --> inspect

food_spawn runs first because its output is food, which motion and next_event read. next_event produces pending_event, which the three appliers consume in parallel (their write-sets are disjoint). cleanup runs after all of them because its read-set includes their writes. inspect runs last because it reads everything and writes nothing.

This is the same shape as a query plan in a database. The query optimiser takes a SQL statement, builds a graph of relational operations (each one a system!), and topo-sorts them into an execution plan. A simulator is a query plan running every tick.

The reason the graph must be acyclic is that a cycle is a contradiction. Suppose system A writes table T, system B reads T and writes U, system A reads U. Now A both produces T (which B reads) and consumes U (which B writes). A and B cannot both run before each other in the same tick.

flowchart LR
    A -->|"T (A writes, B reads)"| B
    B -->|"U (B writes, A reads)"| A

A cycle in the system graph is a design bug; it must be broken - usually by buffering one system’s write so it is consumed next tick instead of this tick.

Designing system order is therefore the same problem as designing a database query plan. Each system is a stage; the DAG is the plan; the program executes the plan. Students who follow this thread end up writing their own minimal query engine without realising it.

The cost of getting the DAG wrong is concrete. A reader that runs before its writer reads stale data - yesterday’s snapshot of a table that was supposed to have been updated. A reader that runs after its consumer reads garbage - a half-written table mid-update. The DAG is the contract that prevents both.

A subtle benefit: once the DAG is explicit, parallelism becomes trivial. Any two systems on the same DAG level - neither one a transitive dependency of the other - can run on different threads. The schedule is implied by the graph. §31 picks this up.

Exercises

1. Draw the DAG. Take the eight simulator systems (motion, food_spawn, next_event, apply_eat, apply_reproduce, apply_starve, cleanup, inspect) and draw the dependency graph yourself, deriving the edges from each system’s read-set and write-set in code/sim/SPEC.md. Compare with the diagram above.

2. Spot the cycle. Suppose apply_starve writes to food (returning fuel to the world when a creature dies). Now apply_starve writes food, which food_spawn reads. food_spawn writes food, which next_event reads. next_event writes pending_event, which apply_starve reads. Where’s the cycle? How would you break it?

3. Topological sort by hand. Given:

   • A writes X
   • B reads X, writes Y
   • C reads X, writes Z
   • D reads Y and Z, writes W

   Which systems can run in parallel? What’s a valid execution order? Are there multiple valid orders?

4. Compose two systems. Write motion (operation, writes pos) and next_event (operation, writes pending_event). Wire them into a tick that runs motion then next_event. Inspect pending_event after the tick.

5. Add cleanup. Add a cleanup system that processes to_remove and to_insert (both initially empty Vecs). Wire it after next_event. Confirm the DAG remains acyclic.

6. (stretch) A query planner. Take five hand-written SQL queries (each one a system shape) and draw the relational-algebra plan for each. Compare with how motion → next_event → apply_* decomposes the simulator. The shape is the same.

Reference notes in 14_systems_compose_into_a_dag_solutions.md.

What’s next

§15 - State changes between ticks is the rule that makes the DAG actually work: mutations buffer; the world transitions atomically.

Solutions: 14 - Systems compose into a DAG

Exercise 1 - Draw the DAG

Drawing it by hand from the read-sets and write-sets in code/sim/SPEC.md reproduces the diagram in the chapter. Forks (after next_event) and joins (before cleanup) are the structural fingerprints of parallel-friendly stages.

Exercise 2 - Spot the cycle

The proposed change creates:

• apply_starve writes food
• food_spawn reads food, writes food
• next_event reads food, writes pending_event
• apply_starve reads pending_event

So apply_starve → food_spawn → next_event → apply_starve - a cycle.

Three ways to break it:

1. Buffer the write. apply_starve pushes to food_to_drop (a side table); a separate next-tick food_drop system applies it.
2. Reorder the policy. Move “food appears where creatures died” out of the inner loop entirely - make food spawn pre-emptive, decoupled from death events.
3. Accept the latency. Allow apply_starve to write to a next-tick food buffer that food_spawn reads next tick. The food appears one tick late.

The first is the standard fix: introduce a side table, defer the cross-system write to the next tick.

Exercise 3 - Topological sort

A writes X
B reads X, writes Y
C reads X, writes Z
D reads Y and Z, writes W

Dependencies: A → B, A → C, B → D, C → D. B and C are at the same DAG level: they share a read of X but their write-sets (Y and Z) are disjoint, so they can run in parallel. Valid orders include A, B, C, D and A, C, B, D. Both are correct.

Exercise 4 - Compose two systems

#![allow(unused)]
fn main() {
fn tick(world: &mut World, dt: f32) {
    motion(&mut world.creatures.px, &mut world.creatures.py,
           &world.creatures.vx, &world.creatures.vy, dt);
    next_event(&world.creatures.px, &world.creatures.py,
               &world.food, &mut world.pending_event);
}
}

The order is forced: motion writes pos, next_event reads pos. Reverse the order and next_event reads stale positions.

Exercise 5 - Add cleanup

#![allow(unused)]
fn main() {
fn tick(world: &mut World, dt: f32) {
    motion(&mut world.creatures.px, &mut world.creatures.py,
           &world.creatures.vx, &world.creatures.vy, dt);
    next_event(&world.creatures.px, &world.creatures.py,
               &world.food, &mut world.pending_event);
    cleanup(world);
}
}

cleanup reads to_remove and to_insert, writes creatures. Neither motion nor next_event touches those tables, so cleanup runs at the end with no DAG conflicts.

Exercise 6 - A query planner

A SQL plan for SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE o.amount > 100 decomposes into:

1. scan(orders) (operation)
2. filter(amount > 100) (filter)
3. join(users, filtered_orders) (operation, two-input)
4. project(name) (operation)

Each is a system with a read-set and a write-set. The plan is a DAG. The simulator’s tick is the same shape with the systems in code/sim/SPEC.md substituting for relational operators. A compiled SQL plan and a simulator tick are isomorphic structures running at different cadences.

15 - State changes between ticks

Concept node: see the DAG and glossary entry 15.

Inside a tick, the population is frozen: no creature is born or dies mid-tick. Structural changes - insertions and removals - are queued, not applied, and committed in one atomic sweep at the tick boundary. Value updates are different: they flow along the DAG in tick order, each system reading its inputs as the upstream writers left them, never half-written.

This is the rule that makes the DAG from §14 actually work. The danger is a system reading another’s half-finished work: if next_event began reading pos while motion were still writing it, half the creatures would have moved and half would not, and what next_event reads would no longer be well-defined. Two rules remove the danger. Order: a system runs only after the systems it reads from have completed, so it sees their finished output, never a partial write. Frozen membership: no system adds or removes a row mid-tick, so the set of creatures every system iterates is identical. Together they make the tick a clean function world_t+1 = step(world_t, inputs_t) - values move forward along the DAG, the population holds still, and cleanup commits the queued births and deaths only at the boundary.

Concretely: apply_starve does not call creatures.swap_remove(slot). It calls to_remove.push(creature_id). The creatures table is unchanged for the rest of the tick. After every system has run, cleanup consumes to_remove and to_insert together, applying every queued change in one sweep. Now the next tick begins with a consistent new world state.

This pattern is called double buffering: there is the committed world the systems read this tick, and the buffer of queued changes (to_remove, to_insert) that cleanup commits to produce the world the next tick reads. The pattern shows up everywhere - graphics frame buffers, database transactions, event-sourced systems. The rule is always the same: structural writes accumulate, then commit.

Two costs to absorb. First, every queued birth or death is one extra row pushed to a to_remove or to_insert table. Second, the cleanup pass is now its own system in the DAG. The benefit dwarfs the costs: every other system in the book composes cleanly, and parallelism becomes easy. With in-tick mutation, every parallel scheduling decision becomes a race condition. With buffered mutation, races are structurally impossible - disjoint write-sets are disjoint by construction.

A subtle case is insertions. A creature born during a tick (via apply_reproduce) does not appear in any system’s read-set during that tick - it is in to_insert, not in creatures. The newborn lives its first life on the next tick. This is the right behaviour for almost every simulation: it gives every creature an equal first tick of life. The alternative - applying inserts mid-tick - is a closed-loop bug factory.

Within one system, the writes can be in-tick: a system that updates pos for every creature in a loop applies each write immediately, because the rest of the system is the only reader and the only writer. The buffering rule is between systems, not between iterations within one system. Inside a system, the writes are sequential; between systems, the writes are batched.

The shape that emerges is: read everything into local arrays at system entry; do work; write outputs to buffers at system exit; commit at tick boundary. It is the same shape as the audio engine’s frame buffer, the database’s transaction commit, and the version-controlled file system’s commit-and-merge. They all solve the same problem: how do you read consistent state while the world is changing?

Exercises

These build on the simulator skeleton. Your to_remove: Vec<u32> and to_insert: Vec<CreatureRow> should already exist.

1. The bug. Write a function that iterates creatures and calls creatures.swap_remove(i) whenever energy[i] <= 0.0. Run it on a 100-creature world where 30 are starving. What goes wrong? (Hint: skipped iterations, half the starvers survive.)
2. The fix. Rewrite the function to push the index into to_remove instead. After the loop completes, apply all removals in one pass. Verify all 30 starvers die.
3. The cleanup pass. Write fn cleanup(world: &mut World, to_remove: &mut Vec<u32>, to_insert: &mut Vec<CreatureRow>). Apply removals first (using swap_remove), then insertions. Why this order, and not the other?
4. Show two ticks. Run the loop for two ticks. After tick 1, log the population. After tick 2, log it again. Confirm that creatures killed in tick 1’s apply_starve do not appear in tick 2’s input.
5. Insertions are tick-delayed. A creature reproduces in tick 5: parent in creatures, two offspring in to_insert. After cleanup, the offspring are in creatures. In tick 6 the offspring receive their first system pass. Confirm by adding an age_in_ticks column and watching offspring start at 0 in tick 6, not in tick 5.
6. (stretch) A bad design that almost works. Try to apply mutations in-tick carefully - collect dead creatures first, then process them in reverse-index order. Show one specific case where this still corrupts state. (Hint: a reproduction produces an offspring whose new index conflicts with an in-progress death.)

Reference notes in 15_state_changes_between_ticks_solutions.md.

What’s next

§16 - Determinism by order is the property the buffering rule guarantees: same inputs, same system order, same outputs. Reproducibility is structural.

Solutions: 15 - State changes between ticks

Exercise 1 - The bug

// BUG: do not do this.
let mut i = 0;
while i < creatures.len() {
    if energy[i] <= 0.0 {
        creatures.swap_remove(i);
        energy.swap_remove(i);
        // Do NOT advance i - the swap put a fresh creature here.
    } else {
        i += 1;
    }
}

This almost works - but the fix else { i += 1 } is fragile. Forget it once and you end up incrementing past the swapped-in creature, skipping the check. Worse, if apply_starve is one of three systems that both read and write creatures mid-tick, the indexes other systems hold become stale silently.

Exercise 2 - The fix

#![allow(unused)]
fn main() {
fn apply_starve(energy: &[f32], to_remove: &mut Vec<u32>, ids: &[u32]) {
    for i in 0..energy.len() {
        if energy[i] <= 0.0 {
            to_remove.push(ids[i]);
        }
    }
}
}

The function only reads. Mutation lives in cleanup, which runs once after every system has had its say. All 30 starvers die - no accidental skips, no mid-loop index hazards.

Exercise 3 - The cleanup pass

#![allow(unused)]
fn main() {
fn cleanup(world: &mut World, to_remove: &mut Vec<u32>, to_insert: &mut Vec<CreatureRow>) {
    // Removals first
    for id in to_remove.drain(..) {
        let slot = world.id_to_slot[id as usize];
        for col in world.columns_mut() {
            col.swap_remove(slot);
        }
        world.id_to_slot[id as usize] = u32::MAX; // mark dead
    }

    // Insertions second
    for row in to_insert.drain(..) {
        let slot = world.append_creature(row);
        world.id_to_slot[row.id as usize] = slot as u32;
    }
}
}

Removals first because they free slots that an insertion might reuse if you implemented slot recycling. If you insert before removing, you grow the table needlessly only to immediately shrink it.

Exercise 4 - Two ticks

After tick 1, log creatures.len(). The 30 dead creatures are still in creatures during the systems of tick 1 (each system saw them) but cleanup removed them at tick 1’s boundary. Tick 2’s input has 70 creatures. A creature killed in tick 1 is gone for tick 2.

Exercise 5 - Insertions are tick-delayed

Add an age_in_ticks: Vec<u32> column. Initialise to 0. Increment every creature in a system at the end of each tick. An offspring inserted in tick 5 is in to_insert during tick 5 → moved to creatures by cleanup at the end of tick 5 → first visible to systems in tick 6. Its age_in_ticks starts at 0 and reaches 1 after tick 6’s increment. The newborn never receives tick 5’s update.

Exercise 6 - A bad design

Tick start: creatures = [A, B, C, D].
A starves: collected.
B reproduces, producing X.
C is fine.
D starves: collected.

In-tick application, reverse-index order:
  swap_remove D from slot 3 → creatures = [A, B, C]
  swap_remove A from slot 0 → creatures = [C, B]   (C was at slot 2, gets swapped in)
  append X                  → creatures = [C, B, X]

But X's ID was supposed to map to its slot for everyone holding a reference;
some other system collected before our `apply_starve` was holding `id(X) → ?`
because X did not exist yet. Now X exists at slot 2. The reference is stale or invalid.

The principled fix is the rule: between systems, no mutations. Defer to cleanup. The reverse-index trick fixes one specific bug while opening the door to many.

16 - Determinism by order

Concept node: see the DAG and glossary entry 16.

A program is deterministic if the same inputs and the same execution produce the same outputs, every time. Sounds obvious. It is not - most modern programs are not deterministic by default. Threads run in OS-scheduled order. Hash maps may iterate in randomised order. The system clock differs by run. rand::thread_rng() differs by process.

In an ECS architecture, determinism is structural. Same world state at tick start + same system order + same inputs (events, RNG seed) = same world state at tick end. Bit-identical. Every time.

This is not a quality goal; it is a precondition for almost everything the book builds on:

• Replay. The world is the log decoded (§37). Replay reconstructs world state by re-running the inputs through the same system sequence. Without determinism, replay is impossible.
• Testing. A property test fixes an RNG seed and asserts the simulator behaves identically across runs. Without determinism, every test is flaky.
• Distributed simulation. Multiple machines run identical copies of the world. Without determinism, they drift apart by tick 1.
• Debugging. A bug at tick 4783 should appear at tick 4783 every run. Without determinism, debugging real-time bugs becomes guesswork.

The recipe for determinism is simple: forbid every source of non-determinism in the inner systems.

• No HashMap in iteration order. Use Vec or BTreeMap, which iterate in deterministic order.
• No system clock. Get time from input events, not from Instant::now(). Time is a value passed into the system, not read from the OS.
• One RNG, seeded. A single Rng with a fixed seed, used in a defined order. Each system that needs randomness reads from it in DAG order.
• No threads inside a system. A system runs single-threaded internally. Parallelism happens between systems with disjoint write-sets (§31), not inside one system.
• Buffered mutations. §15’s rule: mutations apply at tick boundaries, not mid-tick.

These rules are restrictive. They are also the price of every benefit listed above. Most modern programs decline to pay this price and accept the costs - flaky tests, unreproducible bugs, divergent distributed simulation. The book pays the price.

The cost of determinism is not absolute. Within a system, the implementation is free to use whatever it likes - SIMD intrinsics, branch hints, compile-time tricks - as long as the inputs and outputs are bit-identical to what the abstract specification demands. The discipline is at the system boundary: between systems, everything must be reproducible.

Note - Parallel reductions are the exception. “Bit-identical” is easy to lose the moment a reduction runs in parallel. Floating-point addition is not associative: (a + b) + c is not always a + (b + c) in the last bits. A parallel sum that splits the data across threads and merges the partials adds in a different grouping than a serial sum, so the same seed with a different thread count can produce different bits. That is the canary: if your world hashes diverge only when you change core count, suspect a parallel floating-point reduction. Two escape hatches keep determinism. Fix the reduction order: always fold the partials in a defined sequence, independent of how many threads produced them. Or accumulate in integers: scale to fixed-point, sum exactly, scale back. Integer accumulation is exact by construction; fixed-order floating-point is reproducible but still rounds. This is the determinism gotcha behind the replay-across-heterogeneous-hardware claim above - drop it and “same seed, same answer” quietly stops being true across machines with different core counts.

A test for determinism is concrete. Run the simulator twice with the same seed, the same input event log, the same system order. After 1 000 ticks, hash the entire world state. If the hashes match, you are deterministic. If they do not, find the system whose output first differs, and trace the source of variability. Often: a HashMap, a system clock, a thread.

A simulator that is deterministic is also a simulator that can be tested. Once that property holds, every other quality goal - performance, parallelism, distribution - becomes safe to optimise toward. Without determinism, every optimisation is a coin flip.

The full payoff of determinism arrives at the save and load phase named in §11. The simulator can be paused, its tables serialised to disk, reloaded later, and resumed - and the result must be indistinguishable from a run that never paused. The mechanics arrive in §36 - Persistence is table serialization: a snapshot is the world’s tables written as a stream of (entity, key, value) triples - the same shape they have in memory. Combined with the input event log, replay is structural - read the snapshot, replay events through the same DAG with the same seed, you reconstruct the world at any later tick exactly. Determinism (this section), serialization (§36), and log-as-world (§37) are the three legs of replay.

Exercises

1. Hash the world. Write a function that takes the simulator state and produces a u64 hash by feeding every column through std::hash::Hasher. Use this to compare world states across runs.
2. Two identical runs. Run the simulator twice with the same RNG seed. Hash the world at tick 100. Are they equal?
3. Introduce non-determinism deliberately. Replace your seeded RNG with rand::thread_rng() (or wallclock-seeded). Run twice. Show the hashes differ.
4. Find the culprit. Suppose your hashes differ. Hash the world after each system in the DAG. Identify which system’s output first differs, and what source of non-determinism it pulls from.
5. HashMap in iteration order. Build a HashMap<u32, f32> of 10 entries and iterate it twice within one program. Print the order each time. Are they the same? Try with BTreeMap. Try across two runs of the same program.
6. Time as input. Refactor a system that uses Instant::now() to instead take current_time: f64 as a parameter. The system is now deterministic; the source of current_time is the only place non-determinism can enter.
7. (stretch) A property test. Hand-roll a simple property test: generate 100 random seeds. For each, run the simulator for 100 ticks. Hash the resulting world. Verify that the same seed always produces the same hash, and that different seeds usually produce different hashes.

Reference notes in 16_determinism_by_order_solutions.md.

What’s next

You have closed Time & passes. The next phase is Existence-based processing, starting with §17 - Presence replaces flags. The simulator’s hunger and starvation systems are about to lose their booleans.

Solutions: 16 - Determinism by order

Exercise 1 - Hash the world

#![allow(unused)]
fn main() {
use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;

fn hash_world(world: &World) -> u64 {
    let mut h = DefaultHasher::new();
    let c = &world.creatures;
    c.len().hash(&mut h);
    for &x in &c.px { x.to_bits().hash(&mut h); }
    for &y in &c.py { y.to_bits().hash(&mut h); }
    for &vx in &c.vx { vx.to_bits().hash(&mut h); }
    for &vy in &c.vy { vy.to_bits().hash(&mut h); }
    for &e in &c.energy { e.to_bits().hash(&mut h); }
    h.finish()
}
}

Floats are hashed via to_bits() because f32::hash is not trait-implemented (a float can be NaN with multiple bit patterns; the language refuses to choose). to_bits() is bit-equality, which is what determinism requires.

Exercise 2 - Two identical runs

#![allow(unused)]
fn main() {
let mut world1 = init_world(seed: 0xCAFE);
let mut world2 = init_world(seed: 0xCAFE);
for _ in 0..100 { tick(&mut world1); }
for _ in 0..100 { tick(&mut world2); }
assert_eq!(hash_world(&world1), hash_world(&world2)); // bit-identical
}

If the hashes match, your simulator is deterministic for this run length and seed. (Not a proof - just one data point - but a strong one.)

Exercise 3 - Deliberate non-determinism

Replace seeded_rng with thread_rng() or a wall-clock-seeded RNG. Re-run. The hashes differ. The visible state of the world after 100 ticks is structurally the same shape but populated with different numbers.

Exercise 4 - Find the culprit

Hash the world after every system. The first system whose post-hash differs between runs is the culprit. A few common sources:

• A system reads from a HashMap whose iteration order is randomised.
• A system reads Instant::now() or SystemTime::now().
• A system spawns a thread; the thread’s writes race with the main thread’s.

Once located, the source is usually obvious. The fix is to remove the source - a deterministic alternative always exists.

Exercise 5 - HashMap iteration order

std::collections::HashMap uses RandomState by default - its iteration order varies between processes (and sometimes within one process across rebuilds, depending on the Rust version). BTreeMap iterates in sorted-key order, deterministic across runs. For ECS use, prefer Vec<(K, V)> (sequential, deterministic, cache-friendly) over either.

Exercise 6 - Time as input

#![allow(unused)]
fn main() {
// Before
fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32]) {
    let dt = some_global_clock(); // non-deterministic
    /* ... */
}

// After
fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], dt: f32) {
    /* ... */
}
}

dt enters from the caller. The caller may compute it from Instant::now() (production) or read it from a recorded log (replay). The system itself does not know the difference.

Exercise 7 - A property test

#![allow(unused)]
fn main() {
fn property(seed: u64) -> bool {
    let h1 = run_and_hash(seed);
    let h2 = run_and_hash(seed);
    h1 == h2
}

fn run_and_hash(seed: u64) -> u64 {
    let mut world = init_world(seed);
    for _ in 0..100 { tick(&mut world); }
    hash_world(&world)
}

for seed in 0..100u64 {
    assert!(property(seed), "non-deterministic at seed {seed}");
}
}

If any seed produces different hashes across runs, the simulator is non-deterministic. Different seeds usually (not always - the hash space has collisions) produce different hashes; that confirms the simulator is sensitive to its inputs, which is the dual property to determinism.

17 - Presence replaces flags

Concept node: see the DAG and glossary entry 17.

A creature can be hungry. Two ways to model it.

The instinct most programmers arrive with is a boolean: is_hungry: bool on every creature, set to true when energy drops below a threshold, set to false when energy is restored. Every system that cares about hunger checks the flag: if creature.is_hungry { ... }. This is everywhere; it is the natural choice; it is what most programmers reach for.

The data-oriented alternative is membership. There is a hungry table - a Vec<u32> of the slots of hungry creatures, the rows they occupy in the columns. A creature is hungry if and only if its slot is in hungry. The flag does not exist as a field; it exists as a fact about which table the creature appears in. (Why the slot and not the stable id? Because a system that acts on the hungry wants to reach straight into the columns; the slot is that reach. §23 and §26 make the choice precise; for now, the table lists rows.)

The substitution looks small: a bool field becomes a row in another table. The implications are not.

Dispatch changes shape. The flag version is a per-creature filter inside every consuming system - walk all creatures, check the flag, do work if true. The membership version skips the filter - walk hungry, do work for every entry. At 1 000 000 creatures with 100 000 hungry, the flag version processes 1 000 000 rows; the membership version processes 100 000 - a 10× difference in work, and a 10× difference in memory bandwidth. §19 names this.

Storage changes shape. A flag column stores one byte per creature whether the flag is set or not. A creature with eight possible states needs eight bool fields = 8 bytes per creature; a million creatures store 8 MB of flags, most of which are false. A presence table stores only the entries that are set - if 10 % of creatures are hungry, the hungry table is 10 % the size of the flag column.

Persistence changes shape. Serialising a flag column writes the flag for every creature, including the ones where it is false. Serialising a presence table writes only the entries that exist. The latter is also closer to the natural shape of an event log (§37): a hungry_added event per entry, and that is the whole story.

Concurrency changes shape. Two flag fields on the same creature struct may share a cache line; concurrent writers to either field fight over it (§33 - false sharing). Two presence tables are physically separate Vecs; concurrent writers to disjoint tables never collide (§31).

The clean way to phrase the move comes from Richard Fabian’s Data-Oriented Design, in the chapter on Existence Based Processing. Instead of asking each room about its doors, ask the doors-table which doors are in this room. The query is reversed; the lookup is reversed; the work shrinks. Most programs spend their lives doing the wrong direction; the data-oriented mindset is to reverse it.

A production example: in a real ECS daemon, an admission decision is is_admitted(peer) = established_contacts.contains_key(peer). There is no is_admitted: bool on a peer; there is only the question “is this peer’s id in the table?”. O(1), no I/O, no enum.

Presence is not the only valid representation. A bool flag is sometimes right - when nearly every entity has the state set; when the predicate is computed cheaply on the fly; when the data is short-lived and persistence does not matter. But in this book, presence is the default; flags are a tradeoff to earn.

Exercises

These extend the §0 simulator skeleton.

1. Add a hungry table. Add let mut hungry: Vec<u32> = Vec::new(); to your world (it holds slots). It is empty at start.
2. Populate it. Write a system fn classify_hunger(energy: &[f32], hungry: &mut Vec<u32>). Walk creatures; if energy[i] < HUNGER_THRESHOLD and slot i is not already in hungry, push i. (For now use linear scan to check membership; we will fix this in §23.)
3. Build the flag version. Add a parallel is_hungry: Vec<bool> indexed by creature slot. Write the equivalent classification system that sets/clears the bool.
4. Time both at 1M creatures, 10% hungry. Build a 1 000 000-creature world with 10% energy starvation. Time classify_hunger (presence) and the flag-setting version. Note the ratio of bytes touched: the flag version writes 1 MB, the presence version writes ~100 KB plus the cost of the membership check.
5. The membership query. Write fn is_hungry_p(hungry: &[u32], slot: u32) -> bool (presence: scan the table) and fn is_hungry_f(is_hungry: &[bool], slot: usize) -> bool (flag). Time both at 1M creatures. Note: presence is O(N) scanned naively; the flag is O(1). §23 - Index maps is the fix that makes presence O(1) too, without a per-creature boolean - a sparse set.
6. The “how many are hungry” query. Write it both ways. Presence: hungry.len(). Flag: is_hungry.iter().filter(|&&b| b).count(). Compare. The presence version is constant-time; the flag version walks all 1M.
7. (stretch) Persist both. Serialise both representations to a file. Note the disk size for 1M creatures with 10% hungry. The presence version stores ~100 KB; the flag version stores ~1 MB even though most flags are false.

Reference notes in 17_presence_replaces_flags_solutions.md.

What’s next

§18 - Add/remove = insert/delete names what changes between the two representations: in the presence world, state transitions are structural moves between tables, not flag flips.

Solutions: 17 - Presence replaces flags

Exercise 2 - classify_hunger

#![allow(unused)]
fn main() {
const HUNGER_THRESHOLD: f32 = 5.0;

fn classify_hunger(energy: &[f32], hungry: &mut Vec<u32>) {
    for i in 0..energy.len() {
        let slot = i as u32;
        let is_now = energy[i] < HUNGER_THRESHOLD;
        let was_in = hungry.iter().any(|&s| s == slot);
        if is_now && !was_in {
            hungry.push(slot);
        } else if !is_now && was_in {
            if let Some(pos) = hungry.iter().position(|&s| s == slot) {
                hungry.swap_remove(pos);
            }
        }
    }
}
}

The loop index i is the slot, so hungry holds slots. The linear iter().any is O(N). At 100 000 hungry creatures × 1 000 000 creatures per tick, this is 10¹¹ comparisons - far too slow. §23 - Index maps replaces it with an O(1) lookup, a sparse set rather than a per-creature boolean. For now the linear version makes the conceptual point.

Exercise 4 - Bytes touched

The presence write touches at most 100 000 × 4 bytes ≈ 400 KB (each new slot is a u32). The flag write touches all 1 000 000 × 1 byte = 1 MB.

The presence read (the membership check) is more expensive without an index - every classification check costs O(hungry.len()), and the worst case is total O(N²). The flag read is O(1). §23 is the fix; until then, the chapter argues for presence on storage and persistence grounds, not on read-cost grounds.

Exercise 5 - Membership queries

#![allow(unused)]
fn main() {
fn is_hungry_p(hungry: &[u32], slot: u32) -> bool {
    hungry.iter().any(|&s| s == slot)
}

fn is_hungry_f(is_hungry: &[bool], slot: usize) -> bool {
    is_hungry[slot]
}
}

is_hungry_p is O(N); is_hungry_f is O(1). The chapter ends here intentionally - until the index map arrives in §23, the presence cost is real for queries. The shape of the data, however, is what we are committing to. The cost gets fixed by adding more structure - a sparse set (§23) that restores O(1) without a per-creature flag - not by reverting to flags.

Exercise 6 - Counting

#![allow(unused)]
fn main() {
let n_hungry_p = hungry.len();         // O(1)
let n_hungry_f = is_hungry.iter()      // O(N)
    .filter(|&&b| b).count();
}

Counting is the easiest case for presence: the table’s length is the answer. The flag version walks the full vector even though the answer is the same.

Exercise 7 - Persistence

A naive bincode-style serialisation:

• Presence: hungry.len() * 4 + 8 bytes (the Vec<u32> plus a length prefix). At 10 % hungry: ~400 KB.
• Flag: is_hungry.len() bytes. At 1M creatures: ~1 MB.

Both can be compressed; both can be written incrementally. The key observation is that the presence representation reflects what is true: the file is small when the world is mostly at rest. The flag representation is the same size whether or not anything is hungry.

18 - Add/remove = insert/delete

Concept node: see the DAG and glossary entry 18.

In the flag world, a state transition is a write. To make a creature hungry, set is_hungry = true. To stop it being hungry, set is_hungry = false. The flag was always there; only its value changed.

In the presence world, a state transition is a move between tables. To make a creature hungry, insert a row into hungry. To stop it being hungry, remove the row. The state has no field to flip; it has only the question of which table the creature is currently a row of.

Code-wise, the difference is small:

// flag
fn become_hungry_flag(is_hungry: &mut [bool], i: usize) {
    is_hungry[i] = true;
}

// presence
fn become_hungry_presence(hungry: &mut Vec<u32>, i: u32) {
    hungry.push(i);
}

fn stop_being_hungry_presence(hungry: &mut Vec<u32>, i: u32) {
    if let Some(pos) = hungry.iter().position(|&s| s == i) {
        hungry.swap_remove(pos); // O(N) scan for now; §23's sparse set makes it O(1)
    }
}

Two consequences worth naming.

The transition is structural. When a creature crosses the hunger threshold, a row in hungry actually appears or disappears. There is no in-place mutation; the table grows by one or shrinks by one. This is why §22 (mutations buffer; cleanup is batched) exists - adds and removes during a tick must be queued, then applied at the boundary, so that the iteration in progress does not see half the change. The deferred-cleanup pattern is born in this section.

The vocabulary disappears. There is no set_hungry(true), no set_hungry(false), no is_hungry() accessor pair. There is become_hungry (insert) and stop_being_hungry (remove), and even those are usually inlined into the system that detects the transition. The data-oriented program does not have getters and setters; it has systems that move rows between tables.

A useful test: can you describe the transition without naming a bool? “This creature became hungry” - well, did anything change? Yes: the hungry table grew by one entry. “This creature stopped being hungry” - the table shrank by one entry. Every state change in the system has a structural counterpart, and the structural counterpart is the canonical description.

Note - “Hungry” generalises further than this chapter uses it. In an MMORPG, the presence table for “creatures the player needs to know about” is the ones inside the player’s render radius - and the radius itself can shrink dynamically when CPU is tight, trading visible-creature count against the tick-budget headroom from §4. The presence table is a query, not a metaphysical state; its entries change when the system asks a different question. “Alive,” “hungry,” “in-scope,” “subscribed,” “active-this-frame” - same shape, different question, same discipline of inserts and removes between tables.

The same pattern handles richer transitions. Imagine a creature that can be hungry, sleepy, or dead. Three tables: hungry, sleepy, dead. A creature transitions by moving between them. Becoming sleepy while hungry adds a row to sleepy (it can be in both). Dying removes the creature from hungry and sleepy (cleanup affects all relevant presence tables) and adds to dead. The transition is a multi-table operation, but each table is still just a list of slots.

This shape - state changes as inserts and removes - is the precondition for everything else EBP gives you. The dispatch in §19 iterates over the table directly, so the table’s contents being the canonical state of the world is structurally necessary. There is no flag to consult; there is only what is in the table right now.

Exercises

1. Hunger transitions. Use your hungry table from §17. Each tick: read energy; for any creature that crossed below the threshold, push to hungry; for any that crossed back above, swap-remove. Run for 100 ticks with energy varying randomly; verify hungry always contains exactly the creatures whose current energy is below threshold.
2. No flag, no setter. Search your code for any boolean field on a creature. Replace it with a presence table. The setter and getter both disappear.
3. A second presence state. Add a sleepy table. A creature is sleepy if its energy is high enough that it does not need to eat right now. A creature can be in both sleepy and hungry? No - by definition the conditions are mutually exclusive. (Or: design them so they are.) Verify the invariant by checking after each tick that no creature appears in both tables.
4. Death. Add a dead table. When a creature’s energy drops below zero, push to dead and remove from hungry (and from sleepy if present). The cleanup logic is now multi-table; introduce a small transition_to_dead(i) helper that handles all the affected presence tables.
5. The transition log. Add events: Vec<(u64, u32, &'static str)> (tick number, creature id, event name). Every insert/remove emits a row. Note the field is the entity id, not the slot: the membership tables move by slot, but the log is a boundary artifact read back later (§37), when slot positions no longer apply, so it records identity. After 100 ticks, the events log is the canonical history - every state change recorded.
6. (stretch) Reconstruct from the log. Given only the events log and the initial creatures table, reconstruct the final hungry, sleepy, and dead tables. The reconstruction is a one-shot replay; if it produces the same tables as the live simulation, your transitions are correctly captured.

Reference notes in 18_add_remove_insert_delete_solutions.md.

What’s next

§19 - EBP dispatch names the dispatch shape that the table-membership representation makes free.

Solutions: 18 - Add/remove = insert/delete

Throughout, i is a slot and the membership tables hold slots; the event log records the stable entity id (id[i]), because the log is read back later (§37) when slots no longer apply.

Exercise 1 - Hunger transitions

#![allow(unused)]
fn main() {
fn classify_hunger(energy: &[f32], hungry: &mut Vec<u32>) {
    let in_hungry: Vec<bool> = (0..energy.len())
        .map(|i| hungry.contains(&(i as u32)))
        .collect();
    for i in 0..energy.len() {
        let slot = i as u32;
        let starving = energy[i] < HUNGER_THRESHOLD;
        match (starving, in_hungry[i]) {
            (true, false) => hungry.push(slot),
            (false, true) => {
                if let Some(p) = hungry.iter().position(|&s| s == slot) {
                    hungry.swap_remove(p);
                }
            }
            _ => {} // no transition
        }
    }
}
}

The contains precompute is O(N) per slot here, O(N²) overall; §23’s sparse set makes each membership test O(1). After each tick, a sanity check: hungry contains exactly the slots whose energy < HUNGER_THRESHOLD. Verifying this every tick is the kind of test §43 names as “tests are systems”.

Exercise 2 - No flag, no setter

The conversion is mechanical. Find every is_*: bool field on a creature struct, delete it, add a presence table for the corresponding state. Replace creature.is_hungry = true with hungry.push(slot) and creature.is_hungry = false with a swap_remove. The setter and getter pair disappear.

The diff usually shrinks the codebase. Most flag-based systems have boilerplate - assertion that the flag is in the correct state, log on flag change, setter that fires events - that becomes redundant once the transition is itself a structural move.

Exercise 3 - A second presence state

#![allow(unused)]
fn main() {
const SLEEPY_HIGH: f32 = 50.0;

fn classify_sleepy(energy: &[f32], sleepy: &mut Vec<u32>, hungry: &[u32]) {
    for i in 0..energy.len() {
        let slot = i as u32;
        let is_now = energy[i] >= SLEEPY_HIGH;
        let in_hungry = hungry.contains(&slot);
        let in_sleepy = sleepy.contains(&slot);
        match (is_now, in_sleepy, in_hungry) {
            (true, false, false) => sleepy.push(slot),
            (false, true, _) => {
                if let Some(p) = sleepy.iter().position(|&s| s == slot) {
                    sleepy.swap_remove(p);
                }
            }
            _ => {}
        }
    }
}

// Invariant check
fn invariant(hungry: &[u32], sleepy: &[u32]) {
    for &s in hungry {
        debug_assert!(!sleepy.contains(&s), "slot {} in both hungry and sleepy", s);
    }
}
}

Mutually exclusive states are enforced by whoever can transition into them. If only the classification system can add to either, and it reads energy and decides which table to add to, the invariant is maintained by construction.

Exercise 4 - Death

#![allow(unused)]
fn main() {
fn transition_to_dead(
    i: u32,
    hungry: &mut Vec<u32>,
    sleepy: &mut Vec<u32>,
    dead: &mut Vec<u32>,
) {
    if let Some(p) = hungry.iter().position(|&s| s == i) { hungry.swap_remove(p); }
    if let Some(p) = sleepy.iter().position(|&s| s == i) { sleepy.swap_remove(p); }
    dead.push(i);
}
}

The helper makes the multi-table cleanup explicit and centralised. Future systems that add new presence states only need to update this one helper to handle deaths correctly.

Exercise 5 - The transition log

The membership tables move by slot, but the event records the entity - read it from the id column at the slot:

#![allow(unused)]
fn main() {
events.push((tick, id[i], "became_hungry"));
// ... or
events.push((tick, id[i], "stopped_being_hungry"));
}

After 100 ticks the log is a complete history. Logging the entity rather than the slot is what makes the log survive: by the time it is read back, swap_remove and sort (§21, §28) have moved slots around, but the entity id still names the same creature.

Exercise 6 - Reconstruct from the log

The log records entities, so replay rebuilds entity sets:

#![allow(unused)]
fn main() {
fn replay(events: &[(u64, u32, &str)]) -> (Vec<u32>, Vec<u32>, Vec<u32>) {
    let mut hungry: Vec<u32> = Vec::new();  // entities, not slots
    let mut sleepy: Vec<u32> = Vec::new();
    let mut dead:   Vec<u32> = Vec::new();
    for (_t, entity, kind) in events {
        match *kind {
            "became_hungry" => hungry.push(*entity),
            "stopped_being_hungry" => {
                if let Some(p) = hungry.iter().position(|&e| e == *entity) {
                    hungry.swap_remove(p);
                }
            }
            "became_sleepy" => sleepy.push(*entity),
            // etc.
            "died" => {
                hungry.retain(|&e| e != *entity);
                sleepy.retain(|&e| e != *entity);
                dead.push(*entity);
            }
            _ => {}
        }
    }
    (hungry, sleepy, dead)
}
}

To compare with the live simulation, map each live slot through the id column to its entity (id[slot]) and compare the entity sets. They match if and only if every transition was logged. This is replay in miniature; the same shape generalises to the full simulator at §37.

19 - EBP dispatch

Concept node: see the DAG and glossary entry 19.

A system that needs to act on hungry creatures has two ways to find them.

Filtered iteration. Walk all creatures; for each, ask “is it hungry?”; do work if yes:

for slot in 0..creatures.len() {
    if is_hungry[slot] {
        drive_hunger_behaviour(slot);
    }
}

Existence-based dispatch. Walk the hungry table directly; do work for every entry:

for &i in hungry.iter() {
    drive_hunger_behaviour(i);
}

The two produce the same result. The two have very different costs.

The filtered version walks 1 000 000 rows when 100 000 are hungry - 900 000 of those iterations are wasted. Each wasted iteration loads a cache line, runs a branch, and does nothing. The branch is predictable on the way into a false flag (the predictor learns “mostly false”) and unpredictable at the boundaries (where flags change), so the cost is dominated by memory bandwidth: 1 MB of is_hungry flags loaded to do 100 000 units of work.

The EBP version walks 100 000 rows. Every iteration does work. There is no per-row branch; the dispatcher is the table. Memory traffic is proportional to active rows, not to population.

The cost difference scales with the sparsity of the state. If 90 % of creatures are hungry, the two approaches are similar (both touch most of the data). If 10 % are hungry, EBP does 10× less work; at 0.1 %, 1000× less. Most simulator states are sparse - a small fraction of creatures are eating at any given tick, a small fraction are reproducing, a small fraction are dying - so EBP’s advantage compounds everywhere.

One honest qualifier, since this book measures: that ratio is in work and memory traffic. In wall time a scattered subscription shows less of it - a tenth of the rows, but each entry a cache miss into the columns - so at 10 % active the measured speedup is nearer 1.5× than 10× until the subscription is compacted (§26) and the gather streams. The work win is immediate; the time win follows locality. At high sparsity (1 %) the gather is rare enough that even scattered it wins outright.

A useful intuition: it is the difference between a wandering shopper trying to remember what they need and a shopper with a list. The list version is shorter, faster, and correct by construction. You do not consult the list to ask “is this aisle on my list?” - you walk down the list and visit each aisle once.

The shape EBP produces in code is also a clue. A system that uses EBP looks like:

#![allow(unused)]
fn main() {
fn drive_hunger(hungry: &[u32], energy: &mut [f32], dt: f32) {
    // `hungry` holds slots (column positions), not entity ids
    for &i in hungry {
        energy[i as usize] -= HUNGER_BURN_RATE * dt;
    }
}
}

Read-set: hungry. Write-set: energy (only the slots listed in hungry). The signature is the contract - exactly the contract from §13. EBP is not a separate idea; it is the natural shape that a system takes when its inputs are presence tables.

Because hungry holds slots, each entry indexes the columns directly - there is no id-to-slot lookup inside the loop. An entity-id list would not work here: it would need the §10 id-to-slot hop, and worse, it would read the wrong rows after any sort or swap_remove (§9). That directness is the whole point of keying the table by slot; §26 measures what it is worth (and why the table holds slots, not ids), once the lifecycle in §24 makes slots stable enough to store.

EBP also composes cleanly with parallelism. A million creatures with 100 000 hungry can be split across eight threads - each thread takes a 12 500-row slice of hungry and does its work. The threads never need to consult creatures that are not hungry; their loads do not interfere. §31 develops this.

The takeaway: EBP is the dispatch that falls out of §17’s presence-replaces-flags substitution. You do not need to choose to use EBP - once your state is in presence tables, every system naturally iterates them. The flag version does not even arise.

Exercises

1. Compare the two. Implement both drive_hunger_filtered (walks creatures, checks flag) and drive_hunger_ebp (walks hungry). Run both on a 1M-creature world with 10 % hungry. Time both. Note the ratio.
2. Sparsity test. Repeat exercise 1 at three sparsity levels: 1 %, 10 %, 50 %, 90 % of creatures hungry. Plot the cost per tick. The filtered version should stay roughly constant in cost; the EBP version’s cost should be roughly proportional to the fraction.
3. A multi-state system. A creature can be in any combination of hungry, sleepy, dead. Write three EBP systems: drive_hunger, drive_sleep, drive_death. Each iterates only its own presence table. Compare with a single filtered loop that handles all three with if-else.
4. The branch you do not write. Compile both versions in release mode. Look at the assembly (cargo rustc --release -- --emit asm). Confirm the EBP version has no cmp / je for the per-row check, while the filtered version has one. Note that the filtered version’s branch is correctly predicted, but the cache line is still read.
5. EBP with &[T] slices. Most exercises so far use Vec<u32> for the presence table; in production, systems take &[u32] slices. Refactor your drive_hunger_ebp to take hungry: &[u32]. Confirm it still compiles cleanly with the rest of the system DAG.
6. (stretch) A naive EBP bug. A system that iterates hungry while also calling hungry.push on the table corrupts iteration. (You knew this from §9 and §22.) Construct a small case that demonstrates the bug. Then fix it via deferred cleanup.

Reference notes in 19_ebp_dispatch_solutions.md.

What’s next

§20 - Empty tables are free names the consequence at scale: cost is proportional to active rows, not to population.

Solutions: 19 - EBP dispatch

Exercise 1 - Compare the two

#![allow(unused)]
fn main() {
const HUNGER_BURN: f32 = 0.1;

fn drive_hunger_filtered(
    is_hungry: &[bool], energy: &mut [f32], dt: f32,
) {
    for i in 0..is_hungry.len() {
        if is_hungry[i] {
            energy[i] -= HUNGER_BURN * dt;
        }
    }
}

fn drive_hunger_ebp(
    hungry: &[u32], energy: &mut [f32], dt: f32,
) {
    for &i in hungry {
        energy[i as usize] -= HUNGER_BURN * dt;
    }
}
}

At 1M creatures, scan-all-and-branch vs the slot-keyed subscription gather, on a modern desktop (ebp_partition; per-machine spread in code/README):

• At 10 % subscribed: filtered ~0.58 ms, EBP ~0.36 ms - about 1.6×. The subscription does a tenth of the work, but its slots are scattered through the column, so the gather misses cache and spends most of the bandwidth win on the misses.
• At 1 % subscribed: filtered ~0.52 ms, EBP ~0.04 ms - about 14×. Sparse enough that a scattered gather still beats scanning a million flags.

The headline “10× less work at 10 %” is real as work and memory traffic; it shows up in wall time only once the subscription is compacted so the gather streams (§26’s locality, several× there). Scattered, the 10 % win is modest; dense, it is the full order of magnitude. This is why EBP and the §26 compaction belong together.

Exercise 2 - Sparsity test

(Modern desktop; numbers vary by chip - see code/README. The shape is what matters.) Filtered rises gently with the active fraction: it reads all N flags every time but only writes the active ones. The scattered subscription rises faster and crosses filtered near full participation - at 100 % it is a scan with extra bookkeeping (the §26 anti-pattern). Two things pull the EBP curve down: high sparsity (the 1 % row), and compaction (§26), which turns the scattered gather sequential. The durable claim is the work ratio - touch the subset, not the population - and wall time follows once the gather is dense.

Exercise 3 - Multi-state systems

#![allow(unused)]
fn main() {
drive_hunger(&hungry, &mut energy, dt);
drive_sleep(&sleepy, &mut energy, dt);
drive_death(&dead, &mut energy, dt);
}

Three EBP systems, each iterating its own table. Each is bandwidth-bound by the active count, not by the population. The single-filtered-loop alternative looks like:

#![allow(unused)]
fn main() {
for slot in 0..1_000_000 {
    if is_hungry[slot] { /* hunger work */ }
    else if is_sleepy[slot] { /* sleep work */ }
    else if is_dead[slot] { /* dead work */ }
}
}

• and walks 1M rows × 3 flag checks per row × cache-bandwidth cost. Three EBP systems combined are typically 5-20× cheaper than the single filtered version, depending on sparsity.

Exercise 4 - The branch you do not write

The filtered version’s inner loop generates roughly:

mov   al, [is_hungry + slot]
test  al, al
je    .skip
; ... work ...
.skip:

The EBP version’s inner loop has no je for membership; the dispatch is the iteration. Freed from the branch, the loop is a straight gather over the slots listed in hungry - no per-row test, and no id-to-slot indirection to serialise it.

Exercise 5 - &[T] slices

#![allow(unused)]
fn main() {
fn drive_hunger(
    hungry: &[u32],            // <- slice of slots, not Vec
    energy: &mut [f32],
    dt: f32,
) { /* ... */ }
}

The function takes the minimal data it needs. The caller passes &world.hungry and the autoderef does the rest. This is the usual shape for systems and integrates cleanly with the parallel scheduling described in §31.

Exercise 6 - The naive bug

#![allow(unused)]
fn main() {
// BUG: do not do this.
for &i in hungry.iter() {
    if /* some condition */ {
        hungry.push(/* a new slot */); // mutating while iterating
    }
}
}

Rust’s borrow checker actually catches this one - hungry.iter() holds a & reference; hungry.push needs &mut. The code does not compile. The lesson is that the data-oriented discipline (deferred cleanup, §22) is what Rust’s borrow checker enforces structurally. Push to a side table; apply at tick boundary.

20 - Empty tables are free

Concept node: see the DAG and glossary entry 20.

If a presence table is empty, the system that iterates it does nothing. No rows, no work. This is the consequence of §19 at the limit, and it is the property that lets the simulator scale gracefully under shifting state.

Concretely: a 1 000 000-creature simulation that has zero hungry creatures right now spends zero cycles in drive_hunger. The system is wired into the DAG, runs every tick, takes a &[u32] slice of hungry of length 0, iterates zero times, returns. The overhead is one function call and one slice creation - measured in nanoseconds, not milliseconds.

This is not “fast in the empty case as an optimisation”. It is “free in the empty case as a structural consequence”. The flag-based version runs through the entire creature table even when all flags are false, paying full memory bandwidth to discover that no work is needed. The EBP version is told there is no work by the simple fact of an empty table.

The effect compounds across many states. A simulation with twenty possible behaviours, each represented as a presence table, pays for the fraction of creatures actually exhibiting each behaviour. Most ticks, most tables are nearly empty. The total work is proportional to the sum of active rows across all tables, not to population × number of behaviours. For a sparsely active world this is one or two orders of magnitude cheaper than the equivalent flag-based design.

A subtle case worth naming: an empty system is not the same thing as a missing system. A drive_hunger system that iterates an empty hungry is still in the DAG, still scheduled, still part of the program’s contract. It is just doing zero rows of work this tick. Removing it from the DAG entirely would change the contract. EBP gives you cheap idle systems, not absent ones.

This property has implications for design.

Activity-based costs. A simulator’s per-tick cost is set by what is active, not by what exists. A million dormant creatures cost nothing to ignore. Only behaving creatures consume budget. Most simulators in production rely on this - game worlds with hundreds of thousands of NPCs but only a few in active play, training simulations with millions of agents but few in critical phases, control systems with thousands of sensors but few in alarmed state.

Structural sparsity. The world is encouraged to be in mostly-resting states. Designs that scatter activity across many small presence tables (lots of cheap idle systems) outperform designs that concentrate activity in a single big “active creatures” flag. The data-oriented mindset is to multiply states (hungry, sleepy, mating, fighting, …) rather than gate behaviour through one master switch.

Persistence is also activity-based. A snapshot of an empty hungry table is one row in the schema and zero rows of data. A snapshot of an is_hungry: Vec<bool> of length 1 000 000 is 1 MB regardless of how many flags are set. Backups, replication, and replay all benefit from the same property.

The flag-based mind sees idle objects as “still present, just inactive”. The data-oriented mind sees idle objects as not in the table. The difference is one of cost: the former pays for what exists; the latter pays for what is happening.

Exercises

1. Time the empty case. With your simulator from §19, run a tick where hungry is empty. Time drive_hunger. It should be in the nanoseconds range - function call plus slice creation, no inner loop.
2. Time the same case in flag form. Run the flag version of drive_hunger against a 1 000 000-creature world where all is_hungry are false. Time it. Should be milliseconds - full table walked.
3. The cost-per-active-creature plot. Run the EBP simulator with hungry.len() ranging over 0, 100, 1 000, 10 000, 100 000, 1 000 000. Time drive_hunger at each. Plot. The line is roughly linear, starting at near-zero.
4. Add four more states. Add sleepy, mating, fighting, idle as presence tables, each with its own driver system. Run a tick where most tables are empty (most creatures are in idle, say). Confirm the per-tick cost is roughly the cost of the idle driver only.
5. Activity histogram. At each tick, log (tick, table_name, len) for every presence table. After 1000 ticks, plot len over time. The plot is the simulator’s activity profile; flat lines mean the world is at rest, bumps mean events are firing.
6. (stretch) Idle systems removed? Argue why removing an empty system from the DAG (rather than running it with zero work) is the wrong move. Hint: it changes the system DAG, breaks determinism if the table is non-empty next tick, and adds dynamic scheduling cost that exceeds the empty-loop overhead.

Reference notes in 20_empty_tables_are_free_solutions.md.

What’s next

You have closed Existence-based processing. The next phase is Memory & lifecycle, starting with §21 - swap_remove. The simulator is about to start making structural changes to its tables - births and deaths, in production volumes - and the lifecycle phase makes those cheap.

Solutions: 20 - Empty tables are free

Exercise 1 - Time the empty case

#![allow(unused)]
fn main() {
let hungry: Vec<u32> = Vec::new();
let id_to_slot: Vec<u32> = (0..1_000_000).collect();
let mut energy = vec![100.0f32; 1_000_000];

let start = Instant::now();
drive_hunger_ebp(&hungry, &id_to_slot, &mut energy, 0.033);
println!("{:?}", start.elapsed()); // typically 50-200 ns
}

Function call overhead plus slice creation. The for loop never executes. The overhead is in the noise of the measurement.

Exercise 2 - Time the flag-only case

#![allow(unused)]
fn main() {
let is_hungry = vec![false; 1_000_000];
let mut energy = vec![100.0f32; 1_000_000];

let start = Instant::now();
drive_hunger_filtered(&is_hungry, &mut energy, 0.033);
println!("{:?}", start.elapsed()); // typically 1-2 ms
}

A million slots are walked even though every flag is false. The branch is correctly predicted (if false every time), but the cache lines for is_hungry are still loaded - that is the bandwidth cost. ~1 MB of bytes moved through L3 to RAM and back.

The ratio between the two cases is roughly 10⁴-10⁵: the empty EBP case is tens of thousands of times faster than the all-false flag case.

Exercise 3 - Cost-per-active plot

(The 1M case is bandwidth-bound and may even cross to slower-than-filtered, since id_to_slot lookups become random reads at that scale.) The line is roughly linear with a fixed-cost intercept of ~100 ns; the slope is ~2 ns/active-row.

Exercise 4 - Multi-state, mostly idle

In a tick where most creatures are in idle and the other tables are empty:

• drive_idle(idle.len() = 999 000) does the work.
• drive_hunger(hungry.len() = 0): ~100 ns.
• drive_sleep(sleepy.len() = 0): ~100 ns.
• drive_mating(mating.len() = 0): ~100 ns.
• drive_fighting(fighting.len() = 0): ~100 ns.

Total: roughly the cost of drive_idle plus 400 ns of overhead from four empty systems. The four idle systems contribute essentially nothing. A flag-based equivalent with five flag fields per creature would walk all five flags for every creature - five times the bandwidth, regardless of activity.

Exercise 5 - Activity histogram

After 1000 ticks:

tick   hungry  sleepy  mating  fighting  idle
0           0       0       0         0  1000
1          14       0       0         0   986
2          27       3       0         0   970
...
50         98      48      12         3   839
51         95      44      11         2   848
...

Plotted, the lines are mostly flat with small bumps at events (a wave of births, a famine, a reproductive bloom). The plot is the simulator’s vital signs. Anomalies show up as sudden spikes; anomalies that do not show up here probably do not exist as state changes worth caring about.

Exercise 6 - Idle systems removed?

Removing an empty system from the DAG looks superficially like a win - saves the function-call overhead - but is wrong on three fronts:

1. Dynamic scheduling cost. Adding/removing systems each tick requires the scheduler to reconsider the DAG. This is more expensive than running the empty system.
2. Determinism break. A system that disappears in tick N and reappears in tick N+1 has a different DAG between the two ticks; replay must reproduce both DAGs exactly, adding state to the replay system that has nothing to do with the simulation.
3. Tick-boundary atomicity. If the table fills during a tick (via to_insert, applied at cleanup), the system that processes it must already be in the schedule for next tick. Removing it would lose the work.

The fixed cost of an empty EBP system - single-digit microseconds at most - is far cheaper than any of these. Idle systems stay; tables empty out and fill up; the schedule does not move.

21 - swap_remove

Concept node: see the DAG and glossary entry 21.

The presence-replaces-flags substitution from §17 raised a problem we deferred. When a creature stops being hungry, you remove its slot from hungry. When a creature dies, you remove its row from every column. Removing rows from a Vec is expensive - vec.remove(i) shifts every later row left by one, costing O(N).

For a 1 000 000-creature simulator with 1 000 deaths per tick, naive remove costs roughly 10⁹ moves per tick - a thousand times the budget of a 30 Hz simulation.

The fix is a built-in Rust method: vec.swap_remove(i).

#![allow(unused)]
fn main() {
let mut v = vec![10, 20, 30, 40, 50];
let removed = v.swap_remove(1);
assert_eq!(removed, 20);
assert_eq!(v, vec![10, 50, 30, 40]); // 50 was moved into slot 1
}

The mechanism is small: read the last element, write it into the deleted slot, shrink the table by one. One read, one write, and a length decrement. O(1) regardless of N.

Cost. A 1 000 000-creature table with 1 000 swap_removes per tick costs ~6 000 memory writes (one per column, six columns) - about 50 nanoseconds. The naive remove would cost a thousand times more¹.

Cost paid. Order is sacrificed. If your code depended on rows being in any particular order, swap_remove reorders them. Two specific consequences:

• Iteration corrupted. If you iterate the table and call swap_remove during iteration, the slot you just visited now holds a different row, but your loop counter has moved past it. Half the rows after a swap_remove get skipped or revisited inconsistently.
• Slot references break. The row that backfilled the hole used to live at the end; now it sits at slot i under an index nobody was told about. Every slot-keyed table from §17 - hungry, sleepy, the rest - now lists a slot that points at the wrong creature. This is the same bug as §9: rearrangement breaks slot-based references, and a slot-keyed hungry is nothing but slot-based references.

Both problems have fixes already named in the book. The iteration corruption is fixed by §22 - Mutations buffer: swap_remove never runs during iteration; it runs during cleanup at the tick boundary, when no system is iterating. The moved-slot problem is fixed in two steps by §23 - Index maps: the cleanup rewrites the moved slot wherever a slot-keyed table holds it (a reindex), and an id_to_slot map lets anything holding a stable id re-find the creature after the move. Then §24 asks the sharper question: if moving a slot is this much trouble, why move it at all? Mark the creature dead, recycle its slot later, and every reference stays standing. swap_remove on death is the honest first cut - the wrong way that earns the right one.

This whole phase - Memory & lifecycle - only matters for variable-quantity tables. Constant-quantity tables like the 52-card deck never grow or shrink, never need swap_remove, never need any of the machinery in this phase. The card game ran for ten chapters without it. The simulator from §1 onward needs all of it, because creatures are born and die every tick.

To reuse the card-game milestone framing: the constant vs variable distinction is what determines whether a programmer reaches into the lifecycle toolbox at all. Once you have a table whose row count varies at runtime, every tool in this phase becomes load-bearing.

Measurements

remove(0) shifts every later row; swap_remove(0) moves one. So the ratio is essentially the table length and grows with N - “a thousand times” is a floor, not a ceiling. Full output: code/README.md.

Exercises

1. Compare timings. Build a Vec<u64> of length 1 000 000. Time 1 000 calls to vec.remove(0). Time the same with vec.swap_remove(0). The ratio is roughly N.
2. Mid-table delete. Build a Vec<u64> of length 1 000 000. Time 1 000 calls to vec.remove(500_000). Time 1 000 calls to vec.swap_remove(500_000). The naive version is about half as expensive as the front delete; the swap version is unchanged.
3. The iteration hazard. Build a Vec<u64> with values 0..100. In a forward loop, iterate and call vec.swap_remove(i) whenever vec[i] % 2 == 0. Compare with the expected output (only odd values remaining). What did you actually get?
4. The fix in one shape: iterate backwards. Repeat exercise 3, but iterate (0..v.len()).rev(). Does it work now? Why does it work?
5. The fix in another shape: deferred cleanup. Repeat exercise 3, but instead of calling swap_remove inside the loop, push the index to to_remove. After the loop, drain to_remove (in reverse order) and apply swap_remove. This is the §22 pattern in miniature.
6. Aligned swap_remove. Build the simulator’s six creature columns. Write fn delete_creature(world: &mut World, slot: usize) that calls swap_remove(slot) on every column in the same order. Verify all columns remain aligned after a sequence of deletes.
7. (stretch) The bandwidth cost. Compute the bytes moved by vec.remove(0) on a 1 GB Vec: roughly the whole 1 GB. Compute the same for vec.swap_remove(0): roughly one element. The ratio is N / 1.

Reference notes in 21_swap_remove_solutions.md.

What’s next

§22 - Mutations buffer; cleanup is batched is the rule that makes swap_remove safe to use: it never runs while any system is iterating.

Solutions: 21 - swap_remove

Exercise 1 - Compare timings

#![allow(unused)]
fn main() {
use std::time::Instant;

let mut v: Vec<u64> = (0..1_000_000).collect();
let t = Instant::now();
for _ in 0..1000 { v.remove(0); }
println!("remove(0): {:?}", t.elapsed());

let mut v: Vec<u64> = (0..1_000_000).collect();
let t = Instant::now();
for _ in 0..1000 { v.swap_remove(0); }
println!("swap_remove(0): {:?}", t.elapsed());
}

Typical: remove(0) takes around 500 ms; swap_remove(0) takes around 5 µs. The ratio is roughly N / 1. swap_remove is essentially free; remove is essentially the cost of the table.

Exercise 2 - Mid-table delete

remove(500_000) shifts ~500 000 elements left by one - half the work of remove(0). swap_remove(500_000) is unchanged: one read, one write, one decrement. The asymmetry is the whole point.

Exercise 3 - The iteration hazard

// Form A: the range fixes its bound at the original length.
for i in 0..v.len() {
    if v[i] % 2 == 0 { v.swap_remove(i); }
}

// Form B: the length is re-read every pass.
let mut i = 0;
while i < v.len() {
    if v[i] % 2 == 0 { v.swap_remove(i); }
    i += 1;
}

Both are wrong, and they fail differently. swap_remove(i) moves the last element into the hole at i, so the element now sitting at i is never inspected - and the vector got shorter.

Form A builds the range 0..v.len() once, capturing the original length. As elements are removed v shrinks, but i keeps climbing toward the original length, so v[i] eventually indexes past the end and panics.

Form B re-reads v.len() each pass, so it never indexes out of bounds - but it advances i past the element that was just swapped into the hole, skipping it. No panic; a silently wrong Vec that is some mix, not “all odd values.”

One mistake, two failure modes: a crash or a quiet lie. Exercise 4 shows the discipline that avoids both.

Exercise 4 - Iterate backwards

#![allow(unused)]
fn main() {
let mut v: Vec<u64> = (0..100).collect();
for i in (0..v.len()).rev() {
    if v[i] % 2 == 0 { v.swap_remove(i); }
}
}

This works because swap_remove(i) moves an element from index len - 1 (which we have already visited) into index i. Future iterations only visit smaller indices, which are unaffected. The reverse-iteration trick is correct, but fragile: future maintainers may forget the invariant.

Exercise 5 - Deferred cleanup

#![allow(unused)]
fn main() {
let mut v: Vec<u64> = (0..100).collect();
let mut to_remove: Vec<usize> = Vec::new();
for i in 0..v.len() {
    if v[i] % 2 == 0 { to_remove.push(i); }
}
for &i in to_remove.iter().rev() {
    v.swap_remove(i);
}
}

The collection and the mutation are separated. Iteration over v runs to completion before the first swap_remove. The reverse-order drain ensures the indices remain valid as the table shrinks. This is the §22 pattern in miniature.

Exercise 6 - Aligned swap_remove

#![allow(unused)]
fn main() {
fn delete_creature(world: &mut World, slot: usize) {
    let c = &mut world.creatures;
    c.px.swap_remove(slot);
    c.py.swap_remove(slot);
    c.vx.swap_remove(slot);
    c.vy.swap_remove(slot);
    c.energy.swap_remove(slot);
    c.id.swap_remove(slot);
    c.generation.swap_remove(slot);
    c.birth_t.swap_remove(slot);
}
}

All eight columns swap_remove the same slot. The row that was at the end is now at slot, with all eight fields aligned. The row that was at slot is gone. The id_to_slot map (§23) gets the same treatment.

Exercise 7 - The bandwidth cost

vec.remove(0) on a 1 GB Vec moves ~1 GB through L3+RAM. vec.swap_remove(0) moves ~16 bytes (one row’s worth). The ratio is N / 1. At 30 Hz, naive remove on a 1 GB table is impossible (~10 s per call); swap_remove is comfortably under a microsecond.

22 - Mutations buffer; cleanup is batched

Concept node: see the DAG and glossary entry 22.

This rule has been forward-referenced through ten chapters. Time to make it concrete.

Mutations during a tick do not apply immediately; they queue, and a single cleanup pass applies them all at the tick boundary. The shape is two side tables on the world, alongside the creature columns:

#![allow(unused)]
fn main() {
to_remove: Vec<u32>,           // creature ids to delete
to_insert: Vec<CreatureRow>,   // new creature rows to add
}

During the tick, every system that wants to delete pushes to to_remove. Every system that wants to add pushes to to_insert. No system mutates the live tables.

At the end of the tick, one system runs:

#![allow(unused)]
fn main() {
fn cleanup(world: &mut World) {
    // 1. Delete all queued removals via swap_remove.
    for id in world.to_remove.drain(..) {
        let slot = world.id_to_slot[id as usize] as usize;
        for col in world.columns_mut() {
            col.swap_remove(slot);
        }
        // (Update id_to_slot - covered in §23.)
    }

    // 2. Append all queued inserts.
    for row in world.to_insert.drain(..) {
        world.append(row);
    }
}
}

Two passes, both bulk operations. The world is in a fully consistent state at the end.

What this fixes.

The iteration-corruption problem from §21 goes away because swap_remove never runs while any system is iterating. By the time cleanup runs, every system has finished. There is no concurrent iteration to confuse.

The race-condition problem from concurrent mutation goes away. Two systems may both want to remove a creature; both push to to_remove; cleanup deduplicates (or is idempotent in the rare case of double-removal). Neither system needs to coordinate.

The composition problem from §14 goes away. Systems read consistent snapshots; they read the world as it was at tick start, not the world as some other system half-rewrote it.

What it costs.

Every mutation is one extra row pushed to a side table. For a simulator with 1 000 deaths and 500 reproductions per tick, that is 1 500 rows of bookkeeping - a few thousand bytes, completely negligible against the cost of running the systems themselves.

The cleanup pass is one additional system in the DAG. It is empty (no work) when no mutations are queued; it iterates to_remove and to_insert when there are. The system is wired in once and never removed.

What it does not fix.

Two systems may both push the same id to to_remove if they independently detect the same death condition. Cleanup either deduplicates (a small set check) or is robust to double-removal (a swap_remove on a slot whose id is no longer there is a no-op if you check). Most simulators dedupe via a small HashSet at cleanup time.

The order of removals vs insertions inside cleanup matters: deletions first, then insertions. If you insert first, an inserted row might land in a slot you are about to delete. Deleting first frees up slots that subsequent inserts can reuse - though slot recycling is its own decision (§24).

The pattern itself is universal. Database transactions buffer writes and commit at the boundary. Graphics pipelines render to a back buffer and swap. Version-controlled file systems collect changes and commit. They all solve the same problem: how do you let many independent operations modify shared state without stepping on each other? The answer is always the same - accumulate, then apply atomically.

Exercises

1. Implement the side tables. Add to_remove: Vec<u32> and to_insert: Vec<CreatureRow> to your simulator’s world struct. They are empty at the start of every tick.
2. Push from apply_starve. Modify your starvation system to push to to_remove instead of calling swap_remove. Verify the system no longer mutates creatures.
3. Push from apply_reproduce. Modify reproduction to push offspring rows to to_insert. Verify reproduction no longer mutates creatures.
4. Implement cleanup. Write the cleanup system. Apply removals first, then insertions. Run a tick with both kinds of mutations; verify the world is consistent after.
5. The dedup question. Two systems push id 42 to to_remove. Run cleanup naively (no dedup). What happens? Now add a small dedup pass at cleanup. Does the result change?
6. Buffers keep their capacity. to_remove.clear() and to_insert.drain(..) empty the side tables but retain their allocated capacity. Print to_remove.capacity() and to_insert.capacity() across 100 ticks of a busy run and confirm they settle to a steady size and stop reallocating. Why does reusing the buffers each tick, rather than allocating fresh ones, matter for the tick budget (§4)?
7. (stretch) A graphics pipeline analogy. A rendering pipeline draws to a “back buffer” while the “front buffer” is being displayed. At the boundary of one frame to the next, the buffers swap. Argue why this is the same pattern as to_remove / to_insert plus cleanup.

Reference notes in 22_mutations_buffer_solutions.md.

What’s next

§23 - Index maps is the missing piece for swap_remove to be useful: a parallel data structure that tracks where every id currently lives.

Solutions: 22 - Mutations buffer; cleanup is batched

Exercises 1-3 - Wire up the side tables

#![allow(unused)]
fn main() {
struct World {
    creatures: Creatures,   // the column set: px, py, vx, vy, energy, id (§23)
    to_remove: Vec<u32>,
    to_insert: Vec<CreatureRow>,
    id_to_slot: Vec<u32>,
    // ...
}
}

apply_starve becomes:

#![allow(unused)]
fn main() {
fn apply_starve(energy: &[f32], ids: &[u32], to_remove: &mut Vec<u32>) {
    for i in 0..energy.len() {
        if energy[i] <= 0.0 { to_remove.push(ids[i]); }
    }
}
}

apply_reproduce becomes:

#![allow(unused)]
fn main() {
fn apply_reproduce(
    energy: &[f32], px: &[f32], py: &[f32],
    to_insert: &mut Vec<CreatureRow>,
    threshold: f32,
) {
    for i in 0..energy.len() {
        if energy[i] >= threshold {
            let half = energy[i] / 2.0;
            // Offspring inherit the parent's position; real ids are assigned at append (§23/§24).
            to_insert.push(CreatureRow { id: NEW_ID, px: px[i], py: py[i], vx: 0.0, vy: 0.0, energy: half });
            to_insert.push(CreatureRow { id: NEW_ID, px: px[i], py: py[i], vx: 0.0, vy: 0.0, energy: half });
        }
    }
}
}

(In practice, new ids come from the slot allocator from §24.)

Exercise 4 - Implement cleanup

#![allow(unused)]
fn main() {
fn cleanup(world: &mut World) {
    // Removals first: swap_remove every column in lockstep.
    for id in world.to_remove.drain(..) {
        let slot = world.id_to_slot[id as usize] as usize;
        let cr = &mut world.creatures;
        let moved_id = *cr.id.last().unwrap();
        cr.px.swap_remove(slot); cr.py.swap_remove(slot);
        cr.vx.swap_remove(slot); cr.vy.swap_remove(slot);
        cr.energy.swap_remove(slot); cr.id.swap_remove(slot);
        world.id_to_slot[moved_id as usize] = slot as u32;
        world.id_to_slot[id as usize] = INVALID;
    }

    // Then insertions: scatter each new row into the columns.
    for c in world.to_insert.drain(..) {
        let slot = world.creatures.len() as u32;
        let cr = &mut world.creatures;
        cr.px.push(c.px); cr.py.push(c.py);
        cr.vx.push(c.vx); cr.vy.push(c.vy);
        cr.energy.push(c.energy);
        cr.id.push(c.id);
        world.id_to_slot[c.id as usize] = slot;
    }
}
}

Removals first because freed slots are not reused (yet - that’s §24’s recycling). If you insert first, you may insert into a slot you are about to delete from.

Exercise 5 - The dedup question

Without dedup, two systems pushing id 42 cause cleanup to call swap_remove twice on the same id. The first call removes the row. The second call attempts to look up id_to_slot[42], finds INVALID, and… what? Either it panics, or it silently no-ops. Most simulators choose silent no-op via an early-return:

#![allow(unused)]
fn main() {
let slot = world.id_to_slot[id as usize];
if slot == INVALID { continue; }
}

With dedup (a HashSet<u32> collected before the cleanup loop), the second call is never made. Both approaches work; the no-op approach is cheaper for most simulators.

Exercise 6 - Buffers keep their capacity

#![allow(unused)]
fn main() {
for tick in 0..100 {
    // ... systems push to to_remove / to_insert ...
    cleanup(&mut world); // drain(..) empties the buffers but keeps their capacity
    println!("{tick}: cap {} {}", world.to_remove.capacity(), world.to_insert.capacity());
}
}

drain(..) empties a Vec without freeing its buffer, so after the first few busy ticks to_remove.capacity() and to_insert.capacity() settle at the high-water mark and stop growing. Reusing the buffers means cleanup does zero allocation in steady state; a fresh Vec per tick would pay an allocation, and a later free, every tick on the hot path for no benefit. This is the §4 budget again: the cheapest allocation is the one you already made.

Exercise 7 - Graphics pipeline analogy

A renderer draws into a “back buffer” while the GPU is displaying the “front buffer”. At vsync, the buffers swap (or the back buffer is presented). The display never sees a partially-drawn frame; the renderer never overwrites a frame mid-scan.

The simulator’s tick is the same: systems write into to_remove and to_insert (the back buffer); cleanup applies them to the live tables (the front buffer); the next tick reads consistent state. The shape - accumulate, commit at the boundary - is universal.

23 - Index maps

Concept node: see the DAG and glossary entry 23.

The slot-keyed tables from §17 and §19 left two questions open, and §21 added a third.

1. Point membership. “Is slot i in hungry?” costs O(N) when answered by scanning the table (hungry.iter().any(|&s| s == i)).
2. Unsubscribe. To swap_remove slot i out of hungry you first need its position in the table - the same O(N) scan.
3. The moved slot. When swap_remove relocates a row (§21), every slot-keyed table that listed the old position now points at the wrong creature.

All three are solved by one idea: an index map - a parallel array from a key to a position, with a sentinel for “absent”. It appears twice in the simulator, the same shape pointing at two different things.

Instance one: id_to_slot. Maps a stable entity to its current column slot. This is what re-finds a creature after a move, and what anything outside the columns (a save, the network, the UI - §26) uses to turn an id back into a slot.

#![allow(unused)]
fn main() {
const INVALID: u32 = u32::MAX;

fn slot_of(id_to_slot: &[u32], entity: u32) -> Option<usize> {
    let slot = id_to_slot[entity as usize];
    if slot == INVALID { None } else { Some(slot as usize) }
}
}

A sentinel (u32::MAX) marks “no slot - this entity has no current row”. The Option makes the missing case explicit.

Instance two: the sparse set. A membership table needs O(1) “is slot i present?” and O(1) unsubscribe, without a per-creature boolean - a boolean would be exactly the flag §17 abolished, one byte per creature whether set or not. The structure is two arrays: a dense: Vec<u32> of the present slots (what the hot loop walks), and a sparse: Vec<u32> indexed by slot, holding each present slot’s position in dense, or INVALID.

#![allow(unused)]
fn main() {
// is slot i present?   sparse[i] != INVALID
// subscribe(i):        sparse[i] = dense.len() as u32; dense.push(i);
// unsubscribe(i):      let p = sparse[i] as usize;
//                      let moved = *dense.last().unwrap();
//                      dense.swap_remove(p);
//                      sparse[moved as usize] = p as u32;
//                      sparse[i] = INVALID;
}

sparse stores positions and a sentinel, not booleans - it is the index-map pattern again, pointing into the membership table instead of into the columns. It answers “present?” and “where, so I can remove it in O(1)?”, which a boolean could not. This pair, a dense list plus a sparse index, is the sparse set - the membership structure every ECS ships.

Maintenance. Both maps must be kept current whenever a row moves. Take the move that hurts most, swap_remove (§21): the last row, at slot last, backfills the freed slot i.

• id_to_slot. Set id_to_slot[moved_entity] = i; set id_to_slot[deleted_entity] = INVALID.
• Every slot-keyed table. Wherever a dense array listed slot last, rewrite it to i - a reindex through the move. With the sparse set this is O(1): the moved creature’s position is sparse[last], so dense[sparse[last]] = i, then sparse[i] = sparse[last]; sparse[last] = INVALID (when the moved creature was a member).
• Append. A new row at slot n sets id_to_slot[new_entity] = n.
• Sort or shuffle. Reordering for locality (§28) moves every slot; both maps are rewritten in lockstep with the new order.

The cleanup system from §22 is the natural home for all of this. Every move goes through cleanup; cleanup keeps the maps in step. This is also why §24 prefers not to move slots on death at all: every avoided move is a reindex never paid.

Cost. The map adds one u32 per id ever issued, including ids that are currently dead but whose slots have not been recycled. For a simulator that issues a million ids over its lifetime but has 100 000 alive at any moment, the map is 4 MB. That is a real cost - bigger than the alive table itself if the table has narrow columns. Mitigations include:

• Generational ids (§10) bound the map’s size to the maximum live + recycled count, not the total ever issued.
• A HashMap<u32, u32> trades a constant-time lookup overhead for tighter memory; useful when ids are sparse.
• A separate id allocator that recycles dead ids, so the map’s size matches the high-water mark of live ids.

For most simulators, the dense Vec<u32> is the right shape. It is one cache line per 16 ids; cleanup streams sequentially through it.

Each maintained membership table carries its own sparse array of the same magnitude (one u32 per slot). So the cost is paid once for id_to_slot and once more per slot-keyed table kept incrementally. A table you instead rebuild from scratch each tick needs no sparse index at all - only its dense list, cleared and refilled. Rebuild when the membership churns almost completely each tick; maintain incrementally when it is stable and only a few entries change per tick. It is the same cost-versus-churn judgment as everywhere else in the book.

The pattern in the wild. Every ECS engine ships an index map. Bevy’s Entity is a 64-bit handle whose unpacking is essentially a slot lookup with a generation check. slotmap’s SlotMap keeps an internal map. Database engines maintain index maps as B-trees over primary keys. The shape - id-to-slot lookup, maintained on every move - is universal.

Combined with §10’s stable ids and §24’s slot recycling, the index map is the third piece of the generational arena - the canonical handle-based data structure in modern systems software.

Exercises

1. Build the map. Add id_to_slot: Vec<u32> to your simulator. Initialise to INVALID for all ids. When a creature is appended at slot N, set id_to_slot[id] = N.
2. Build the sparse set. Give hungry a sparse: Vec<u32> alongside its dense list. Implement subscribe(i), unsubscribe(i), and is_member(i) - each O(1), no boolean. Confirm is_member always agrees with a linear scan of the dense list across a run of subscribes and unsubscribes.
3. Maintain both maps on swap_remove. Modify your cleanup so that after creatures.swap_remove(slot):
   • id_to_slot[deleted_entity] = INVALID; id_to_slot[moved_entity] = slot (the last row, now at slot)
   • every slot-keyed table is reindexed: wherever it listed the moved row’s old slot, rewrite it to slot. Verify hungry still lists exactly the hungry creatures after a sequence of deaths.
4. Time the difference. At 1 M creatures, call is_member(random_slot) 100 000 times per tick. Compare the linear scan of the dense list (§17) with the sparse-set lookup (§23). The ratio is roughly N - about a million.
5. The bandwidth cost. At 1 M ids, id_to_slot is 4 MB. Cleanup’s update of the map writes ~12 bytes per swap_remove (delete row’s slot, moved row’s slot, plus bookkeeping). Compute the cleanup cost in microseconds for 1 000 deletes per tick; compare to the budget at 30 Hz.
6. Compaction compatibility. When creatures are reordered for locality (the §28 compaction), every slot moves. Rewrite id_to_slot and every slot-keyed table in lockstep with the new order. Verify both id-held references and slot-keyed memberships are still correct after the sort.
7. (stretch) A from-scratch generational arena. Combine §10’s generation: Vec<u32>, §22’s deferred cleanup, and §23’s id_to_slot map into a SlotMap<T> struct. Compare the shape with slotmap::SlotMap - same machinery, organised differently.

Reference notes in 23_index_maps_solutions.md.

What’s next

§24 - Append-only and recycling names two strategies for what happens to a slot after it has been freed. The choice is decided by access pattern, not by taste.

Solutions: 23 - Index maps

Exercise 1 - Build the map

#![allow(unused)]
fn main() {
const INVALID: u32 = u32::MAX;

// The entity table: one column per field, aligned by slot.
struct Creatures {
    px: Vec<f32>, py: Vec<f32>,
    vx: Vec<f32>, vy: Vec<f32>,
    energy: Vec<f32>,
    id: Vec<u32>,
}
impl Creatures {
    fn len(&self) -> usize { self.id.len() }
}

struct World {
    creatures: Creatures,
    id_to_slot: Vec<u32>, // length = high-water mark of ids ever issued
    next_id: u32,
}

// `CreatureRow` is a transient value: the fields of one new creature,
// scattered into the columns by `append`, never stored.
fn append(world: &mut World, c: CreatureRow) -> u32 {
    let id = world.next_id;
    world.next_id += 1;
    while world.id_to_slot.len() <= id as usize {
        world.id_to_slot.push(INVALID);
    }
    let slot = world.creatures.len() as u32;
    let cr = &mut world.creatures;
    cr.px.push(c.px); cr.py.push(c.py);
    cr.vx.push(c.vx); cr.vy.push(c.vy);
    cr.energy.push(c.energy);
    cr.id.push(id);
    world.id_to_slot[id as usize] = slot;
    id
}
}

The map grows lazily as new ids are issued. INVALID marks dead/never-used slots.

Exercise 2 - Build the sparse set

The membership table is a sparse set: a dense list of the present slots (what the hot loop walks) plus a sparse index from slot to its position in dense, or INVALID. No per-creature boolean.

#![allow(unused)]
fn main() {
struct Subscription {
    dense:  Vec<u32>,   // present slots
    sparse: Vec<u32>,   // slot -> position in dense, or INVALID
}

impl Subscription {
    fn is_member(&self, i: u32) -> bool {
        let p = self.sparse[i as usize];
        p != INVALID && self.dense[p as usize] == i
    }
    fn subscribe(&mut self, i: u32) {
        if self.is_member(i) { return; }
        self.sparse[i as usize] = self.dense.len() as u32;
        self.dense.push(i);
    }
    fn unsubscribe(&mut self, i: u32) {
        let p = self.sparse[i as usize];
        if p == INVALID { return; }
        let moved = *self.dense.last().unwrap();
        self.dense.swap_remove(p as usize);
        self.sparse[moved as usize] = p;
        self.sparse[i as usize] = INVALID;
    }
}
}

is_member is O(1) - two array reads - against the §17 linear scan’s hundreds of microseconds at 1 M. And unlike a Vec<bool> flag, the sparse index hands back the dense position, which is what makes unsubscribe O(1) too. It is the same index-map shape as id_to_slot, pointing into the membership table instead of into the columns.

Exercise 3 - Maintain on swap_remove

#![allow(unused)]
fn main() {
// reindex one slot-keyed table after a row moves from `old` to `new`.
fn reindex_move(sub: &mut Subscription, old: u32, new: u32) {
    let p = sub.sparse[old as usize];
    if p == INVALID { return; }          // the moved creature was not a member
    sub.dense[p as usize] = new;
    sub.sparse[new as usize] = p;
    sub.sparse[old as usize] = INVALID;
}

fn delete_by_id(world: &mut World, id: u32) {
    // The dead creature was already unsubscribed from every table at death (§18).
    let slot = world.id_to_slot[id as usize] as usize;
    let cr = &mut world.creatures;
    let moved_id = *cr.id.last().unwrap(); // the row swap_remove relocates into `slot`
    cr.px.swap_remove(slot); cr.py.swap_remove(slot);
    cr.vx.swap_remove(slot); cr.vy.swap_remove(slot);
    cr.energy.swap_remove(slot); cr.id.swap_remove(slot);
    let moved_old_slot = world.creatures.len() as u32; // length AFTER the removes

    // id_to_slot: re-find the moved entity; retire the dead one.
    world.id_to_slot[moved_id as usize] = slot as u32;
    world.id_to_slot[id as usize] = INVALID;

    // every slot-keyed table: the survivor moved from `moved_old_slot` to `slot`.
    reindex_move(&mut world.hungry, moved_old_slot, slot as u32);
    // ... and any other subscription the survivor might be in.
}
}

Two repairs per move, both O(1): id_to_slot re-finds the entity that relocated, and reindex_move rewrites that creature’s slot wherever a subscription listed it. When the deleted row was already the last one, moved_old_slot == slot and reindex_move is a harmless no-op (the dead creature is no member). One swap_remove per column plus a handful of map writes - a few dozen bytes per delete, well under 10 ns at ~12 GB/s. This is the cost §24 avoids entirely by not moving slots on death.

Exercise 4 - Time the difference

At 1 M creatures, the linear-scan presence check costs ~1 ms. The indexed version costs ~50 ns. Run 100 000 queries per tick:

• Linear: 100 000 × 1 ms = 100 seconds. Impossible.
• Indexed: 100 000 × 50 ns = 5 ms. Fits 30 Hz with margin.

The factor of N (a million) shows up in real wall time.

Exercise 5 - Bandwidth cost of cleanup

1 000 deletes per tick × 12 bytes each = 12 KB written per tick. At ~12 GB/s memory bandwidth, that is ~1 µs. Compare to a 30 Hz budget of 33 ms: ~0.003 % of the tick. The cleanup pass is essentially free; the system can afford to run every tick without measurable cost.

Exercise 6 - Compaction compatibility

#![allow(unused)]
fn main() {
fn sort_creatures_for_locality(world: &mut World) {
    let n = world.creatures.len();
    let mut order: Vec<usize> = (0..n).collect();
    let cr = &world.creatures;
    order.sort_by_key(|&i| spatial_cell(cr.px[i], cr.py[i]));

    // Gather every column into the new order, in lockstep.
    let px = order.iter().map(|&i| cr.px[i]).collect();
    let py = order.iter().map(|&i| cr.py[i]).collect();
    let vx = order.iter().map(|&i| cr.vx[i]).collect();
    let vy = order.iter().map(|&i| cr.vy[i]).collect();
    let energy = order.iter().map(|&i| cr.energy[i]).collect();
    let id: Vec<u32> = order.iter().map(|&i| cr.id[i]).collect();
    world.creatures = Creatures { px, py, vx, vy, energy, id };

    // Rewrite id_to_slot: every slot moved.
    for (new_slot, &cid) in world.creatures.id.iter().enumerate() {
        world.id_to_slot[cid as usize] = new_slot as u32;
    }

    // Reindex every slot-keyed table through the same permutation.
    // order[new] = old, so invert it to map old slot -> new slot.
    let mut new_pos = vec![0u32; n];
    for (new, &old) in order.iter().enumerate() {
        new_pos[old] = new as u32;
    }
    reindex_subscription(&mut world.hungry, &new_pos);
    // ... and every other subscription.
}

fn reindex_subscription(sub: &mut Subscription, new_pos: &[u32]) {
    for s in sub.dense.iter_mut() { *s = new_pos[*s as usize]; }
    for p in sub.sparse.iter_mut() { *p = INVALID; }
    for (pos, &slot) in sub.dense.iter().enumerate() {
        sub.sparse[slot as usize] = pos as u32;
    }
}
}

Every slot moves, so both maps are rewritten: id_to_slot, so id-held references (a save, the network, the UI - §26) still resolve; and each subscription’s dense/sparse, so the slot-keyed memberships still point at the right creatures. Id references and slot references are each repaired by the sort, through their own map. Nothing holds a bare slot across the sort and expects it to survive - it survives only because cleanup rewrites it.

Exercise 7 - From-scratch generational arena

#![allow(unused)]
fn main() {
struct SlotMap<T> {
    items: Vec<T>,
    generation:   Vec<u32>,
    free:  Vec<u32>,
}

impl<T: Clone + Default> SlotMap<T> {
    fn insert(&mut self, t: T) -> (u32, u32) {
        if let Some(slot) = self.free.pop() {
            self.items[slot as usize] = t;
            (slot, self.generation[slot as usize])
        } else {
            let slot = self.items.len() as u32;
            self.items.push(t);
            self.generation.push(0);
            (slot, 0)
        }
    }

    fn remove(&mut self, slot: u32) {
        self.generation[slot as usize] += 1;
        self.free.push(slot);
        self.items[slot as usize] = Default::default(); // optional
    }

    fn get(&self, slot: u32, generation: u32) -> Option<&T> {
        if self.generation[slot as usize] == generation { Some(&self.items[slot as usize]) } else { None }
    }
}
}

Compare with slotmap::SlotMap - the same machinery. The crate adds a packed key (slot + generation in one u64), an iterator API, and a null() sentinel. The shape is identical.

24 - Append-only and recycling

Concept node: see the DAG and glossary entry 24.

When a row is removed from a table, its slot is freed. There are two strategies for what happens to that slot.

Append-only. Old slots stay valid forever. The table grows monotonically. New rows always go to the end.

Recycling. Freed slots are reused. The table’s length stays bounded. New rows go into freed slots before the table grows.

Each is correct; they have very different access patterns and costs.

Append-only. Use when:

• History matters. The simulator’s eaten, born, dead logs from code/sim/SPEC.md are all append-only - they record what happened. Removed entries would be lost history.
• Old references must remain valid forever. Some pointer-into-table designs assume the table never shrinks.
• Total volume is bounded by elapsed time, not by population. A 30-second 30 Hz simulation produces at most 900 frames; an append-only frame log is at most 900 rows. No need to recycle.

The cost is monotonic memory growth. A long-running simulator with append-only eaten accumulates millions of rows over hours. Mitigations:

1. Periodic snapshot + truncate (the log is replaced by a recent slice).
2. Tiered storage - recent in memory, older streamed to disk (§30).
3. Just accept the memory, if the run is short.

Recycling. Use when:

• Steady-state size is small even though total inserted is large. The simulator’s creatures table at 100 000 alive with 100 000 deaths and 100 000 births per second - net flow zero, but total ever issued grows linearly. Recycling keeps memory bounded.
• Memory matters. Recycling caps the table at the high-water mark of live rows.

The cost is reference-stability complications. A new row in a recycled slot has the same slot as a previous, removed row. Code holding an old slot reference would silently dereference the new row. The fix is generational ids: each slot has a generation counter that increments on every recycle. References hold (id, generation); dereference checks the generation. A stale reference fails its check.

A slot allocator looks like:

#![allow(unused)]
fn main() {
struct SlotPool {
    free_slots: Vec<u32>,  // freed slots awaiting reuse
    next_slot:  u32,       // high-water mark; the next never-used slot
    generation:        Vec<u32>,  // generation per slot
}

impl SlotPool {
    fn allocate(&mut self) -> (u32, u32) {
        let slot = self.free_slots.pop().unwrap_or_else(|| {
            let s = self.next_slot;
            self.next_slot += 1;
            self.generation.push(0);
            s
        });
        let g = self.generation[slot as usize];
        (slot, g)
    }

    fn free(&mut self, slot: u32) {
        self.generation[slot as usize] += 1;
        self.free_slots.push(slot);
    }
}
}

allocate pops a freed slot if any are available, otherwise grows. free bumps the generation and adds the slot back to the free list. Stale references (with the old generation) cannot dereference the recycled row.

Choosing between them. Match the strategy to the table’s role:

Mixing strategies in one simulator is normal. The discipline is to be explicit about which table is which, and apply the right machinery to each.

Exercises

1. Two append-only logs. Implement eaten and born as append-only Vecs. After 1 000 ticks, examine the log lengths and verify they grow monotonically.
2. A recycling pool. Implement the SlotPool above. Allocate 1 000 slots, free 500, allocate 500 more, observe the slot indices. Did the pool reuse the freed slots, or grow?
3. Stale reference detection. Allocate a slot with (slot, generation=0). Free it. Allocate a new row in the same slot - its generation is 1. Try to dereference the old (slot, 0). The check fails; the reference is recognised as stale.
4. Switch creatures to append-only. Run the simulator with creatures as append-only (no recycling). Run for 10 000 ticks with steady birth and death. Plot the table’s length over time. It grows monotonically; memory increases without bound.
5. Switch eaten to recycling. Run with eaten recycled. After 100 ticks, all “what did this creature eat at tick 50” queries fail because the rows were reused. The history is gone.

Reference notes in 24_append_only_and_recycling_solutions.md.

What’s next

§25 - One writer, many readers is the rule that makes every other discipline in the phase work: each table is written by exactly one system.

Solutions: 24 - Append-only and recycling

Exercise 1 - Append-only logs

#![allow(unused)]
fn main() {
fn record_eat(eaten: &mut Vec<EatEvent>, tick: u64, creature: u32, food: u32) {
    eaten.push(EatEvent { tick, creature, food });
}

fn record_birth(born: &mut Vec<BirthEvent>, tick: u64, parent: u32, offspring: u32) {
    born.push(BirthEvent { tick, parent, offspring });
}
}

After 1 000 ticks of a 100-creature simulation with average 5 events per tick, the logs hold ~5 000 entries each. The len() only grows. The order of entries reflects insertion order, which is deterministic (§16).

Exercise 2 - Recycling pool

Allocate 1 000 → slots 0-999 (high-water mark grows). Free 500 → slots 0-499 enter free_slots. Allocate 500 more → slots 0-499 are reused (LIFO from free_slots.pop()). The high-water mark stays at 1 000; no growth.

If you do not free anything, the pool grows indefinitely. If you free everything, the pool’s allocations always reuse.

Exercise 3 - Stale reference detection

#![allow(unused)]
fn main() {
let mut pool = SlotPool::new();
let (slot, gen0) = pool.allocate();   // (0, 0)
pool.free(slot);                       // generation[0] is now 1
let (slot2, gen1) = pool.allocate();   // (0, 1) - same slot, new generation
assert_eq!(slot, slot2);
assert_ne!(gen0, gen1);

// Old reference (slot, gen0) - should fail.
let valid = pool.generation[slot as usize] == gen0;
assert!(!valid);
}

The check is two reads and a comparison. References that hold the wrong generation get None from the dereference; references that hold the right generation get Some(&row). No data corruption, no aliasing - the system is sound by construction.

Exercise 4 - Append-only creatures

Run with no recycling. Plot creatures.len() over time:

tick     0:        100
tick  1000:       1100
tick 10000:      11000
tick 100000:    101000

Linear growth, unbounded. Memory leaks. The fix is one of: (a) recycle creatures, (b) truncate-and-snapshot periodically, (c) accept the bound for short runs.

Exercise 5 - Recycling eaten

After 100 ticks, 90% of the eaten log has been overwritten by recent events. Asking “what did creature 42 eat at tick 50” returns either a different event (overwriting an old slot) or None if the old slot is on the free list. History is gone.

The lesson is the inverse of exercise 4: each strategy is correct for a kind of table, wrong for the other kind.

25 - One writer, many readers

Concept node: see the DAG and glossary entry 25.

Every table is written by exactly one system. One ECS system, not one function and not one thread: a single named place in the schedule that owns the right to mutate it. Many systems may read it.

The rule is small. Its consequences are everything.

Note - “Ownership” here means the right to write. Rust already gives ownership a precise meaning: who holds a value and is responsible for dropping it. This chapter uses the word for something narrower and related - which single system is allowed to mutate a table, the guarantee Rust’s &mut T expresses at the type level. The overlap is not an accident. Both senses exist to remove ambiguity about who may change a thing, and that is the weight the word carries here: when exactly one writer can touch a table, its contents at any tick are a function of the inputs alone, not of who reached it first. That is what makes the world deterministic. Read “ownership of a table” as “write-ownership” throughout.

Why it works. A row is a tuple (§6) - its fields are aligned by index. A table’s columns must be modified together to maintain alignment. A single writer guarantees this: only one place in the code mutates the table, so only one place can violate alignment, so testing one place is enough.

A table with two writers has two places where alignment can be violated. If they run concurrently, alignment is violated nondeterministically. If they run sequentially, the order matters and must be specified. Either way, the cost of getting it right grows superlinearly with the number of writers.

The disciplines that depend on it. All of these need single-writer ownership to work:

• §31 - Disjoint write-sets parallelize freely. Two systems with disjoint write-sets can run on different threads. The rule guarantees no shared mutation.
• §22 - Mutations buffer. A side-table writer (cleanup) is the only writer of creatures. All other systems push to to_remove and to_insert, which they own.
• §43 - Tests are systems. A test system reads everything and writes nothing. The ownership rule is what guarantees its reads see consistent state.
• The InspectionSystem pattern. A debug inspector holds read-only references to every table. Read-only access composes with single-writer ownership to make races structurally impossible.

What the rule looks like in practice.

#![allow(unused)]
fn main() {
fn motion(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], dt: f32) { /* writes px, py */ }

fn next_event(px: &[f32], py: &[f32], food: &[Food], pending: &mut [Event]) {
    /* reads px, py, food; writes pending_event */
}

fn apply_eat(pending: &[Event], food: &[Food],
             to_remove: &mut Vec<u32>, energy: &mut [f32]) {
    /* reads pending, food; writes to_remove and energy */
}
}

For each table, exactly one writer is allowed:

• px, py: written only by motion.
• pending_event: written only by next_event.
• to_remove, to_insert: written by many systems, but each system writes only its own queued mutations; no one reads them until cleanup.
• creatures, food: written only by cleanup, which materialises every other system’s queued changes.

Multiple systems may contribute to a table by pushing to its side buffer; the actual single writer is cleanup. The architecture preserves the rule even as many systems propose mutations.

The borrow checker enforces this. Rust’s &mut [T] is the type-level expression of single-writer ownership: only one mutable reference can exist at a time. The borrow checker rejects code that violates it. The data-oriented discipline of single-writer-per-table is what Rust’s ownership model is for; the language enforces what the architecture demands.

Bugs that arise from violations. Two systems writing the same column produce inconsistent state. The bug is usually intermittent (depends on schedule), silent (no error reported, just bad data), and late-binding (manifests far from the cause). They are among the hardest bugs in any concurrent system. The single-writer rule eliminates them by construction.

The rule applies recursively. A view table whose entries are derived from another table inherits the ownership rule: a hungry: Vec<u32> is owned by the system that classifies hunger; no other system writes to it.

This is the rule that closes Memory & lifecycle. Without it, the buffering, swap_remove, index maps, and slot recycling are all unsafe in any concurrent or parallel context. With it, everything composes.

Exercises

1. Identify the writers. For each table in your simulator (creatures, food, food_spawner, pending_event, eaten, born, dead, hungry, to_remove, to_insert), name the one system that writes it. If you find a table with two writers, the rule is violated - investigate.
2. A constructed violation. Write two systems that both update creature.energy directly (not via to_remove/to_insert). Run them in sequence; observe correct results. Run them in parallel via std::thread::scope; either Rust’s borrow checker rejects the code, or you observe a race.
3. Refactor. For one of the violations from exercise 1 (or 2), introduce a buffer table that one system writes and the other reads. The two systems are now writer-disjoint.
4. Build an InspectionSystem. Write a system that takes & (immutable) references to every table and returns a WorldSnapshot struct. Run it after every tick. The system is read-only and never violates the rule.
5. Borrow checker. Try to write code where two systems hold &mut to the same Vec. Rust refuses. Note the exact error message - this is the language enforcing the architecture.
6. (stretch) The cleanup system as canonical writer. In your simulator, audit: every mutation of creatures, food, etc. flows through cleanup. Every other system writes only to to_remove, to_insert, or its own outputs. Verify the audit holds for the simulator end-to-end.

Reference notes in 25_ownership_of_tables_solutions.md.

What’s next

You have closed Memory & lifecycle. The simulator’s machinery is now complete: it can grow, shrink, recycle, parallelise, and replay. The next phase is Scale, starting with §26 - Subscription tables, keyed by slot. The simulator’s per-tick cost goes under the microscope.

Solutions: 25 - One writer, many readers

Exercise 1 - Identify the writers

The to_remove / to_insert “many writers” is allowed because each system writes only its own pushes; nobody reads the side table until cleanup.

Exercise 2 - Constructed violation

#![allow(unused)]
fn main() {
// Two systems both write energy directly. Don't do this.
fn apply_eat_bad(food: &[Food], pending: &[Event], energy: &mut [f32]) {
    for ev in pending {
        if ev.kind == EAT { energy[ev.creature as usize] += 1.0; }
    }
}

fn apply_decay_bad(energy: &mut [f32], dt: f32) {
    for e in energy { *e -= 0.1 * dt; }
}
}

Run sequentially: correct, but order matters. Run in parallel via std::thread::scope:

#![allow(unused)]
fn main() {
std::thread::scope(|s| {
    s.spawn(|| apply_eat_bad(&food, &pending, &mut energy));
    s.spawn(|| apply_decay_bad(&mut energy, dt));
});
}

Rust’s borrow checker rejects the code: &mut energy cannot be borrowed twice. The language refuses to compile the violation.

Exercise 3 - Refactor

Add a side buffer:

#![allow(unused)]
fn main() {
fn apply_eat(food: &[Food], pending: &[Event], energy_delta: &mut Vec<(usize, f32)>) {
    for ev in pending { if ev.kind == EAT { energy_delta.push((ev.creature as usize, 1.0)); } }
}

fn apply_decay(energy_delta: &mut Vec<(usize, f32)>, count: usize, dt: f32) {
    for i in 0..count { energy_delta.push((i, -0.1 * dt)); }
}

fn apply_energy(energy: &mut [f32], deltas: &[(usize, f32)]) {
    for &(i, d) in deltas { energy[i] += d; }
}
}

Now apply_eat and apply_decay write to disjoint slices of energy_delta (use Vec::extend_from_slice from per-thread buffers, then merge). The single writer of energy is apply_energy. The rule holds.

Exercise 4 - InspectionSystem

#![allow(unused)]
fn main() {
struct WorldSnapshot {
    creature_count: usize,
    food_count: usize,
    population_alive: usize,
    energy_avg: f32,
}

fn inspect(world: &World) -> WorldSnapshot {
    WorldSnapshot {
        creature_count: world.creatures.len(),
        food_count: world.food.len(),
        population_alive: world.id_to_slot.iter().filter(|&&s| s != INVALID).count(),
        energy_avg: if world.creatures.energy.is_empty() { 0.0 } else {
            world.creatures.energy.iter().sum::<f32>() / world.creatures.energy.len() as f32
        },
    }
}
}

fn(&World) -> Snapshot - read-only; no &mut anywhere. The system can run alongside any other system without violating ownership; multiple parallel readers are fine.

Exercise 5 - Borrow checker

let mut a = vec![1, 2, 3];
let r1: &mut Vec<i32> = &mut a;
let r2: &mut Vec<i32> = &mut a; // ERROR

error[E0499]: cannot borrow `a` as mutable more than once at a time

The error is the language enforcing the architecture. Two &mut borrows of the same Vec cannot coexist. By choosing &mut [T] everywhere our systems take their write-set, the compiler enforces single-writer ownership at compile time.

Exercise 6 - Audit

The audit should find:

• Every direct mutation of creatures happens in cleanup. If it happens elsewhere, mark the location and refactor.
• Every direct mutation of food happens in cleanup. Same rule.
• Every system has a clearly declared write-set, expressed as &mut parameters in its signature.
• No system holds a &mut World. Such a signature would allow it to write any table, violating the rule.

If the audit passes, the simulator is ready for parallelism (§31) without any further refactoring. If it fails, the failure points are the only places that need fixing.

26 - Subscription tables, keyed by slot

Concept node: see the DAG and glossary entry 26.

A system rarely touches every entity. Motion moves all of them, but starvation only reads the hungry, reproduction only the well-fed, a sleep timer only the sleeping. §17 gave the tool for “which entities are in this set”: a membership table. §19 measured the payoff: iterate the 100 000 hungry instead of scanning 1 000 000 and branching, and the work is proportional to the subset, not the population.

Call that membership table a subscription table, and the loop that walks it a system’s hot loop. A creature subscribes to hungry when its energy drops; it unsubscribes when it eats. The subscription is the system’s input; the hot loop is the system. This section settles a question §17 left open: what does a subscription table store, and how fast is the hot loop that reads it?

A wrong turn first: splitting fields.

The instinct many readers arrive with is to split the fields of a creature into hot and cold: put the fields an inner loop touches in one struct, the rarely-read fields in another, so a load does not drag in bytes the loop ignores. In a row-oriented (array-of-structs) world this is a real technique. In the structure-of-arrays world this book has used since §7, it is already done for you: every field is its own column. Reading px never touches birth_t; they are different arrays. There is no row to drag a cold field along with, so there is nothing to split. The bandwidth win that motivates a field split in AoS is the same win SoA already banked back in §7; it is not a separate technique to apply here.

So the attribute table is never split. It stays whole: every column, every slot, reachable by index i. What a system changes is not the table but how it reaches into it. Rather than scan the whole table and branch, a system keeps a subscription table, the slots i it cares about, and indexes straight in: no scan, no field split, direct access. The rest of this section is about making that direct access fast.

What a subscription stores: ids or slots.

A creature has a stable id and a current slot, its position in the attribute columns. id_to_slot maps one to the other. A subscription table can hold either, and the choice is not cosmetic.

• Hold ids. The hot loop reads an id, looks up id_to_slot[id] to find the slot, then gathers the columns. One extra load per element, every tick. The table survives relocation untouched: when cleanup moves entities, only id_to_slot changes.
• Hold slots. The hot loop reads a slot and gathers the columns directly, no redirection. But when cleanup moves entities, every slot in every subscription is now stale and must be rewritten.

The redirection is paid every tick. The rewrite is paid once per cleanup interval. Which loses?

Measured. At 1 000 000 creatures with a tenth subscribed, the id-keyed hot loop runs about twice as slow as the slot-keyed one on a modern desktop¹, and is the slower of the two on every machine measured. The cost is not the extra instruction. id_to_slot is a four-megabyte array and the subscribed ids are scattered through it, so each lookup is a cache miss before the column gather has even started. The slot key skips that miss entirely.

The rewrite the slot key pays in return is small and bounded. It scales with how many subscriptions an entity sits in (rewrite each table the entity appears in), but it happens once per cleanup interval, not once per tick. Across the realistic range, a handful of subscriptions and a cleanup every few dozen ticks, the per-tick saving buries the per-interval rewrite: the amortized verdict favours slot keys at every subscription count and interval tested, on every machine measured. The benchmark is ebp_partition; numbers below and in code/README.md.

So subscription tables hold slots. This is also why the lifecycle keeps stable slots and lets cleanup own the reindex: slot keys are only safe when one system is responsible for rewriting them when entities move. The cleanup can do that for any reference it owns - a subscription, or a cross-entity link stored in a column - remapping them all in one pass.

So what is the stable id for, once the hot loop runs on slots? For every reference the cleanup cannot reach to rewrite. A save file (§36), a replay log (§37) whose events are (entity, key, value), a packet on the wire, an entity the UI has selected, a snapshot a slow background system is still reading (§39): a slot is meaningless to all of them, because the next compaction moves it. They hold the id and resolve it through id_to_slot once, at the boundary (§35), never per element. Slots are an internal, momentary fact; the id (with its generation, §10) is the identity that survives a relocation, a save, and a network hop.

Locality: a slot-keyed loop is fast only when its slots are dense.

A slot-keyed hot loop gathers columns at the slots the subscription lists. If those slots are scattered through the column, which is what churn produces as deaths and births leave holes, the gather misses cache on nearly every element. If they are contiguous, the gather streams. Compacting the live, subscribed entities to the front of the columns turns a scattered gather into a sequential one; measured, that is several times faster². The compaction is not free, but it pays for itself within a few ticks³, and it is the same batch pass that reclaims dead slots. §28 is that pass.

The one case a split would help, in full view.

There is a single scenario where grouping fields would still pay. A hot loop that reads several columns at scattered slots touches one cache line per column per element; interleaving those columns into one record would touch one. That case is real, and worth stating plainly rather than hiding behind the principle. We keep the columns separate anyway. The book’s answer to scatter is to remove it: compaction (§28) makes the subscribed slots dense, a dense gather streams each column at full bandwidth, and the per-column cost the interleaving would have saved is gone, paid back within a few ticks. Interleaving would also forfeit what SoA bought in §7, whole-column streaming and SIMD, on every loop that is not scattered, to win the one that is. So the rule stands with its exception in the open: keep the columns separate, and compact when the gather scatters.

Name the subscription before you build it.

A subscription is earned by a system that genuinely processes a subset. “Most creatures are not hungry on most ticks, so hungry is far smaller than the population” is a sound reason to build one. “Every creature is always in alive, but other engines keep an alive-set” is not. A subscription that holds the whole population is a scan-all with extra bookkeeping, and the measurement says so: at full participation the subscription loop is marginally slower than a plain scan. The subscription wins in proportion to how much it excludes, and not otherwise.

The payoff is not only speed; it is extensibility. An entity’s character is just the set of subscriptions that hold it - a grazer is “the herd-motion table plus the graze-forage edge,” no more - so a new kind of entity is a new subscription, not a new type threaded through the code. The reference simulator adds a predator exactly this way: register a predators subscription, wire two systems (herd-motion and a forage edge onto grazers), and the pass that maintains subscriptions never learns a predator exists, because it maintains every one of them the same way. Diff the sim2 binary against sim and a whole trophic level costs about six lines, with nothing existing edited. Composition, not surgery - the extensibility the architecture keeps promising, made literal and measurable in a diff.

Measurements

The prose quotes the modern-desktop figure; the spread across the reference machines is below. The amortized keying verdict (slot vs id) favours slot keys at every subscription count S and interval G on every machine. Full per-machine output: ebp_partition in code/README.md.

Exercises

These extend the simulator’s creature columns and the id_to_slot map from §23.

1. Build a slot-keyed subscription. Add hungry: Vec<u32> holding the slots of hungry creatures. A creature subscribes (push its slot) when energy[slot] drops below a threshold. Write the hot loop: iterate hungry, gather the columns by slot directly. Verify it touches only the subscribed creatures.
2. Key it by id instead, and time both. Build a second version where hungry holds entity ids and the hot loop resolves each through id_to_slot[entity] before the gather. At 1M creatures with 10% subscribed, time both hot loops. Reproduce the ~2x gap. Where does the id version’s time go? Compare the size of id_to_slot with one cache line.
3. Unsubscribe in O(1). When a creature stops being hungry, remove its slot from hungry with swap_remove. What do you need alongside hungry to find the slot’s position in the table without scanning it? (It is the §23 trick again, one level up.)
4. Reindex on compaction. Relocate the live creatures to the front of the columns (a stand-in for the §24/§28 cleanup), producing an old-slot to new-slot map. Rewrite the slot-keyed hungry through that map; confirm the hot loop still processes the same creatures. Now do the same for the id-keyed version: what has to change? Time both reindex passes.
5. Dense vs scattered. Time the slot-keyed hot loop with the subscription’s slots scattered through the column, then again after compacting them to the front. Reproduce the several-fold speedup. How many ticks of hot-loop saving pay back one compaction pass?
6. The subscription that holds everyone. Subscribe every creature and time the hot loop against a plain for i in 0..n scan. The subscription should be no faster, and slightly slower. Explain why, and state the rule for when a subscription is worth building.
7. (stretch) Two subscriptions, one entity. Put creatures in both hungry and sleepy. On compaction, both tables need rewriting. Measure how the reindex cost grows with the number of subscriptions an entity sits in, and argue why it stays cheaper than the id key’s per-tick redirection for any realistic cleanup interval.

Reference notes in 26_subscription_tables_solutions.md.

What’s next

§27 - Working set vs cache puts numbers on the question this section kept leaning on: how big is the hot loop’s footprint, and what cache level does it fit in?

Solutions: 26 - Subscription tables, keyed by slot

Throughout, i is a slot (a creature’s position in the columns) and entity is a stable id. The numbers below are modern-desktop figures from ebp_partition at N = 1 000 000; the chapter’s Measurements table and code/README.md carry the per-machine spread.

Exercise 1 - Build a slot-keyed subscription

hungry holds the slots of hungry creatures. A creature subscribes when its energy crosses below the threshold; a parallel in_hungry: Vec<bool> (one per slot) stops it being pushed twice.

#![allow(unused)]
fn main() {
let mut hungry: Vec<u32> = Vec::new();        // slots, not ids
let mut in_hungry: Vec<bool> = vec![false; n];

fn classify(energy: &[f32], in_hungry: &mut [bool], hungry: &mut Vec<u32>, thr: f32) {
    for i in 0..energy.len() {
        if energy[i] < thr && !in_hungry[i] {
            hungry.push(i as u32);
            in_hungry[i] = true;
        }
    }
}

// the hot loop: walk the subscription, index the columns directly by slot
fn feed(energy: &mut [f32], hungry: &[u32], feed: f32) {
    for &i in hungry {
        energy[i as usize] += feed;
    }
}
}

The hot loop never looks at a creature that is not hungry. There is no scan of the attribute table and no branch.

Exercise 2 - Key it by id instead, and time both

#![allow(unused)]
fn main() {
fn feed_id(energy: &mut [f32], hungry_id: &[u32], id_to_slot: &[u32], feed: f32) {
    for &entity in hungry_id {
        let i = id_to_slot[entity as usize] as usize; // the extra hop
        energy[i] += feed;
    }
}
}

Measured, 10 % subscribed: the slot-keyed loop runs about 0.38 ms/pass, the id-keyed loop about 0.85 ms/pass - roughly 2x. The id version’s extra time is the id_to_slot[entity] load. id_to_slot is a 4 MB array (one u32 per id at 1M ids) and the subscribed entities are scattered through it, so consecutive entries almost never share a 64-byte cache line: the lookup is a miss before the column gather has even started. The slot key has no such array to consult.

Exercise 3 - Unsubscribe in O(1)

To swap_remove slot i out of hungry you need its position in the table, not the slot value. Keep a parallel pos: Vec<u32> mapping a slot to its index in hungry - the §23 trick, one level up (there it mapped id to slot; here it maps slot to table-position).

#![allow(unused)]
fn main() {
fn unsubscribe(hungry: &mut Vec<u32>, pos: &mut [u32], in_hungry: &mut [bool], i: u32) {
    let p = pos[i as usize] as usize;
    let moved = *hungry.last().unwrap();   // the slot that will backfill p
    hungry.swap_remove(p);
    pos[moved as usize] = p as u32;        // its new table-position
    in_hungry[i as usize] = false;
}
}

pos is itself indexed by slot, so it rides along with the columns and must be rebuilt at compaction (Exercise 4), the same as any other slot-keyed structure.

Exercise 4 - Reindex on compaction

Compaction produces old_to_new[old_slot] = new_slot for the survivors. The slot-keyed subscription is rewritten through it in one pass:

#![allow(unused)]
fn main() {
fn reindex(hungry: &mut [u32], old_to_new: &[u32]) {
    for i in hungry.iter_mut() {
        *i = old_to_new[*i as usize];
    }
}
}

The id-keyed version needs no change to hungry_id: it holds entities, and entities do not move. Only id_to_slot is rebuilt over the live set - which the slot-keyed design also rebuilds, because identity at the boundary still resolves through it.

Measured, ~100 000 live: the id-keyed reindex is about 0.13 ms (rebuild id_to_slot); the slot-keyed reindex is about 0.17 ms at one subscription per entity (the same rebuild plus the table rewrite). Both run once per cleanup interval, not once per tick - which is why the slot key wins overall (Exercise 7).

Exercise 5 - Dense vs scattered

Measured: the slot-keyed hot loop costs about 0.37 ms/pass when the subscribed slots are scattered through the column, and about 0.084 ms/pass after compacting them to the front - roughly 4.4x. The compaction pass itself costs about 0.88 ms (one batch reindex of the ~100 000 live). It saves ~0.29 ms/pass, so it pays for itself after about 3 passes (ticks) and is pure profit until churn re-scatters the slots. This is the same pass as §28 and the same pass that reclaims dead slots (§24).

Exercise 6 - The subscription that holds everyone

Measured at full participation: the subscription loop costs about 1.20 ms/pass, a plain for i in 0..n scan about 1.03 ms/pass. The subscription is slower. It does the same N column touches, plus an extra read of the subscription array per element, and it gives up the clean 0..n bound the compiler optimises best.

The rule: build a subscription when the subset is durably a fraction of the population (hungry, sleeping, in-combat). For a set that holds nearly everyone on most ticks (alive), scan. A subscription earns its keep by what it excludes; one that excludes nothing is bookkeeping with no payoff.

Exercise 7 - Two subscriptions, one entity

On compaction every subscription an entity sits in must be remapped, so the slot-keyed reindex grows with the count S. Measured, ~100 000 live: about 0.17 ms at S = 1, 0.23 ms at S = 2, 0.36 ms at S = 4. The id-keyed reindex stays flat (~0.13 ms) because the id tables are never touched.

The slot key still wins, decisively, because the two costs live on different clocks. The reindex is paid once per cleanup interval; the id key’s redirection (~0.47 ms/pass) is paid every tick. Over a 30-tick interval the slot key saves about 14 ms in the hot loop and spends at most ~0.4 ms more in reindex. For any realistic interval and any small S, slots win - which is the measured basis for keying subscriptions by slot.

27 - Working set vs cache

Concept node: see the DAG and glossary entry 27.

The working set of a loop is the data it touches per pass. The cache hierarchy (§1) is what holds that data. The two together decide the loop’s speed.

Which cache level holds the working set decides the loop’s speed. The numbers below are measured on the four reference machines (code/README.md), not theoretical - the modern desktop sits at the low end of each range, the Pi 4 at the high end. A flat streaming sum stays under ~0.5 ns/element in L1 (cache_cliffs). Motion’s 20-byte loop, swept sequentially, measures ~0.3-4 ns/creature in L2 (10K creatures), ~0.4-10 ns in L3 (1M), and ~0.7-17 ns once it spills to RAM (10M)¹. Sequential access stays bandwidth-bound and cheap on every machine; the expensive regime is random order, ~30-390 ns/creature at 10M³.

If you ran §1’s exercises and exercise 2 below, you have your own machine’s numbers. Treat the spread above as the envelope between slow and fast hardware, not an absolute.

Computing the working set is mechanical. Motion’s inner loop reads px, py: f32 = 8 bytes, vx, vy: f32 = 8 bytes, energy: f32 = 4 bytes. Total: 20 bytes per creature. At N creatures, working set = 20 × N bytes.

Sequential streaming barely shows the cliff: the prefetcher keeps motion bandwidth-bound, so per-creature time climbs only gently into RAM - roughly 0.3 ns at 10K rising to 0.7 ns at 10M on a modern desktop, 4 ns rising to 17 ns on a Pi 4. The steep cliff is in random order (exercise 5): at 10M creatures, random access costs ~30 ns/creature on the desktop and ~390 ns on the Pi. The per-machine ladder is motion_working_set in code/measurement; the numbers are in code/README.md.

This is what §4’s “cliff” was about, made concrete for your simulator. The transition points are not magic - they are arithmetic over your cache sizes.

The working set is exactly the columns the loop reads, no more: SoA hands you that for free, because there is no wider row to trim (§7 again). From there, two levers push the cliff outward. A narrower field (§2) cuts the bytes per creature - dropping energy from f64 to f32 is 4 bytes per creature off the set. And a system that touches only a subset can subscribe (§26), cutting the count rather than the width. Motion reads every creature, so its lever is field width and access order, not subscription.

The implication is design discipline:

• Decide the target N before the schema. The schema must fit the cache that fits N.
• Audit the inner loops. Sum the bytes per row touched. Compare to your cache sizes.
• When you cross a transition, measure - do not assume. The prefetcher and the OS will sometimes save you, sometimes not.
• The narrowest field that holds the value (§2) is not aesthetic; it is the cliff’s distance.

This is not premature optimisation. It is layout-aware design - making the schema fit the machine that will run it. A schema that ignores the cache works for small N and breaks at the scales the simulator was meant for.

Measurements

Sequential motion stays cheap into RAM (the prefetcher keeps it bandwidth-bound); the cliff is in random order. The §27 “ns/element ladder” is really a random-access ladder. Full output: code/README.md.

Exercises

1. Compute your working sets. For each system in your simulator, compute bytes per row × N for N = 1K, 10K, 100K, 1M, 10M. Note which cache level each falls into for your machine.
2. Find your cliff. Time motion at N = 1K, 10K, 100K, 1M, 10M. Plot ns-per-element against N. The transitions should match your cache sizes.
3. The unused column costs nothing. Add a birth_t: f64 column that motion never reads. Recompute motion’s working set and repeat exercise 2. The cliff should not move: in SoA a column a loop does not read sits in its own array, untouched, so it adds zero to that loop’s working set. (In an array-of-structs world it would have widened every row and moved the cliff inward - the difference SoA buys you.)
4. A wider field. Change energy: f32 to energy: f64. Recompute the working set. Repeat exercise 2. The cliff should move inward (closer to smaller N).
5. Random vs sequential. Repeat motion’s loop with for &i in random_indices instead of for i in 0..N. At 10M creatures the per-element time rises by roughly 25-45× (random RAM access vs sequential). A single-pointer chase shows a wider gap; motion’s is smaller because each creature amortises five columns.
6. (stretch) The L1 sweet spot. Find the N at which motion’s working set fills L1 to roughly 75 %. Run the loop in tight repetition and compare to the closest L2-only neighbour. For sequential motion the difference is small - measured 1.0-1.2× across the four reference machines (l1_sweet_spot)⁴, because the loop is bandwidth-bound at both sizes and the prefetcher hides the L1/L2 boundary. The dramatic L1 win shows up when the access is random (exercise 5), not streaming.

Reference notes in 27_working_set_vs_cache_solutions.md.

What’s next

§28 - Proximity is a property of position answers the neighbour query the simulator has been deferring - and shows that the spatial index you reach for is one you never have to keep.

Solutions: 27 - Working set vs cache

Exercise 1 - Compute working sets

For motion (reads pos: 8B, vel: 8B, energy: 4B = 20 B per row):

For apply_eat (reads pending: 24B, food: 8B, energy: 4B ≈ 36 B):

A pending event is only active for one tick, so the working set is 36 × pending_count, not 36 × N. Even with 10K events per tick, the working set is 360 KB - comfortably L2.

Exercise 2 - Find your cliff

Run the loop:

#![allow(unused)]
fn main() {
fn motion_bench(px: &mut [f32], py: &mut [f32], vx: &[f32], vy: &[f32], energy: &[f32], dt: f32) {
    for i in 0..px.len() {
        px[i] += vx[i] * dt;
        py[i] += vy[i] * dt;
        // (dummy use of energy to keep it in the working set)
        std::hint::black_box(energy[i]);
    }
}
}

Plot ns-per-element vs N. On a typical desktop:

• 1K: ~0.3 ns
• 10K: ~0.5 ns
• 100K: ~1 ns
• 1M: ~2.5 ns
• 10M: ~6 ns
• 100M: ~10 ns

The transitions at 100K (L2 boundary) and 10M (L3 boundary) are the visible cliffs.

Exercise 3 - The unused column costs nothing

Adding birth_t: f64 leaves motion’s working set at 20 B per creature, and the cliff does not move. Motion reads px, py, vx, vy, energy; birth_t lives in its own array that the loop never touches, so it contributes nothing to the bytes streamed per pass. In an array-of-structs layout the same field would have widened every record from 20 B to 28 B and pulled the cliff inward by ~30 % in N. That gap is the SoA win (§7): the working set is the columns you read, never the columns you store.

Exercise 4 - A wider field

Switching energy: f32 → energy: f64 adds 4 B per row → 24 B total. For a fixed L3 size, the maximum N that fits drops by ~17 %. The cliff moves inward.

Exercise 5 - Random vs sequential

Sequential RAM access stays bandwidth-bound: motion’s loop measures ~0.7-17 ns/creature at 10M (modern desktop to Pi 4, the prefetcher helping). Random RAM access is ~30-390 ns/creature - the prefetcher cannot help. The gap is ~25-45× depending on hardware (motion_working_set, code/README.md). The same algorithm, two access patterns; an order of magnitude or more apart.

Exercise 6 - The L1 sweet spot

L1 is 32 KB. 32 KB / 20 B per row ≈ 1 600 rows fills L1. At 75 % fill, ~1 200 rows. Run motion at N = 1 200 and at N = 10 000. The 1 200 case stays in L1 throughout; the 10 000 case spills.

Typical: 1 200 → 0.2 ns/elem; 10 000 → 0.5 ns/elem. About 2.5× faster for the L1-resident loop. The exact ratio depends on how aggressive the compiler is at vectorising - both loops should auto-vectorise to AVX2 or AVX-512.

28 - Proximity is a property of position

Concept node: see the DAG and glossary entry 28.

Creatures eat the food they encounter. So next_event has to answer, for every creature, which food is within reach? At §1’s ten thousand that is a cheap scan. At §2’s million it is a wall: comparing every creature to every food is O(C×F). Measured, even twenty thousand all-pairs neighbour tests cost ~270 ms on a modern desktop¹ (and close to a second on the older reference machines) - many frames’ budget spent on a fraction of the world.

The reflex is to reach for a spatial index: a quadtree, a grid hash, a structure that lives beside the world, that you insert into and delete from as things move, that you query. It works. But stop and look at what it is: a second copy of information the world already holds - position - with its own maintenance budget, its own allocations, and pointer-chased buckets that miss cache on every hop.

Step back and ask what proximity is. It is a function of position. And position is already owned and streamed, every tick, by the motion system. The cell a creature falls in is one line - cell = f(px, py) - computed in the pass motion is already making, branchless, SIMD-friendly. The index was never necessary. The cell is a property you read off position.

Bin, don’t index. Compute each creature’s cell, then place the creatures into per-cell buckets with a counting sort: histogram the cells, prefix-sum into offsets, scatter the indices into one dense array. No HashMap, no per-cell allocation, no pointer-chasing - three linear passes over contiguous memory. A neighbour query reads the 3x3 block of cells around a point as contiguous ranges.

#![allow(unused)]
fn main() {
// cell id per creature - the SIMD-friendly byproduct of the position stream
let cell: Vec<u32> = (0..n).map(|i| cell_of(px[i], py[i])).collect();

// counting sort into a dense CSR bucket array: offsets[c]..offsets[c+1]
let mut offsets = vec![0u32; ncells + 1];
for &c in &cell { offsets[c as usize + 1] += 1; }
for c in 0..ncells { offsets[c + 1] += offsets[c]; }
let mut items = vec![0u32; n];
let mut cursor = offsets.clone();
for i in 0..n { let c = cell[i] as usize; items[cursor[c] as usize] = i as u32; cursor[c] += 1; }
}

Measured (proximity, 1M creatures): the dense bin answers the neighbour query about 2.8x faster than the bolt-on HashMap on a modern desktop², and is ahead on every machine measured (nearer 1.6x on the small-cache Broadwell NUCs). The hash spends its time allocating buckets and chasing them; the dense bin streams.

The sharpest number is the build itself. Rebuilding the entire spatial structure from scratch costs around one percent of the query it serves³. So the whole reason a bolt-on index exists, “don’t pay to rebuild,” is optimising about one percent of the work. You do not maintain proximity across ticks. You recompute it from the position stream each tick, for free, in the pass motion already makes. Recompute from the stream beats maintain a structure, and the old question of how often to re-sort the world simply evaporates: there is no kept structure to schedule.

Warning - “Binning is O(N)” hides an assumption: constant density. The query reads a 3x3 block of cells, which is cheap only while a cell holds a roughly fixed number of creatures. In a fixed world more creatures means fuller cells, so the block you scan grows with N and the whole pass is O(N²) again, with the grid still in place. The words missing from “O(N)” are “at constant density,” and density is a design choice. Measured on the reference forage (forage_scaling, modern desktop): in a fixed world, cost grew ~8x then ~11x for each 3x increase in population - quadratic; with the world grown to hold density constant, the same query grew ~3-4x - linear. Binning moves the wall, it does not remove it.

When you only need one answer, ask the cell once. The chapter opened with a creature eating the nearest food in reach - one answer per creature, not a list. So you never need every occupant of the 3x3 block; you need one. Collapse each cell to a single representative - the first occupant will do - and a query reads at most nine representatives whatever the density, which restores O(N) even in a fixed world. The price is an approximation: a creature takes its cell’s representative, not provably the nearest of a crowded cell. But the error is bounded by one cell, and the cell is already the resolution the grid quantises position to, so the approximation sits below what the model can see. Measured, the representative held linear (~3x per 3x population) exactly where the exhaustive 3x3 scan went quadratic. The cell stops being a bucket you search and becomes a lookup you read: ask the cell who is in it, do not measure every pair.

The gather still scatters, and that is §26’s job. Binning finds the candidates cheaply, but reading their positions jumps around the columns. Making that gather dense is the compaction from §24/§26: the same batch pass that reclaims dead slots can reorder the survivors by cell (a Z-order curve keeps neighbouring cells adjacent in memory), so a cell’s creatures land on adjacent cache lines. That reorder is the GC’s slow-cadence pass, not a separate spatial sort with its own knob. §28 says which cell; §26 makes reading the cell stream.

#![allow(unused)]
fn main() {
fn cell_of(px: f32, py: f32, cell_size: f32) -> u32 {
    let x = (px / cell_size).floor() as i32;
    let y = (py / cell_size).floor() as i32;
    // simple stripe pack; a Z-order (Morton) hash keeps 2D neighbours close in 1D
    ((x as u32 & 0xFFFF) << 16) | (y as u32 & 0xFFFF)
}
}

Note - Binning answers who is near now, at the sampled positions of this tick. Reusing it for collision smuggles in an assumption: that motion between samples is linear and the step is small. When the step is large, or event-driven with variable length (§12), two fast movers can swap sides inside one step - a long travel vector crossing a short one - and share no cell at any sampled instant. The bin never sees them meet; they tunnel through each other. A finer grid does not fix it; swept (continuous) detection does: solve for the time within the step at which the two linear paths first come within radius. That solve is a quadratic in t, and its root is an event time, not a sample. Binning still earns its keep by handing you the candidate pairs cheaply; the closest-approach solve turns a candidate into a dated collision event.

The same lesson at the global scale: the pack-leader. Swarming beasts look coordinated, but if every beast accounts for every other - cohesion, alignment, separation against all N - the cost is O(N²) (~240 ms at twenty thousand). The way the old games did it: put an abstract, invisible leader at the centre of the pack. The leader does the one expensive thing, deciding where the pack goes; each beast subscribes to the leader and steers relative to it. One centroid pass, every member reads one value: O(N), some 9000x cheaper at the same twenty thousand on a modern desktop⁴ (thousands of times on every machine), and the gap grows with N. Lifelike swarm behaviour, no all-pairs accounting. The “who is near the group” question, like “who is near me,” is answered by a single pass over position, not by a structure every agent maintains.

The meta-lesson is the one worth keeping. Twice now the cheap path was to refuse the obvious data structure - the id_to_slot hop in §26, the spatial index here - and instead let the system that already owns the data produce the answer in the pass it already makes. Ask what the problem is before reaching for a structure to make it fit. Proximity is position; position is already in hand.

Measurements

The prose quotes the modern-desktop figure; the spread across the reference machines is below. Full per-machine output: proximity in code/README.md. The density-wall and representative figures come from forage_scaling on the modern desktop; cross-machine measurement is pending, but the shape they show - quadratic at fixed world, linear once density is held or the cell is collapsed to a representative - does not depend on the machine.

Exercises

1. The all-pairs wall. For N agents in a box, count neighbours within radius r by testing every pair. Time it at N = 1K, 10K, 20K. Confirm the O(N²) curve, and that 20K alone already exceeds a 30 Hz frame budget.
2. Cell as a derived column. Write fn cell_of(px, py, cell_size) -> u32. Compute the cell column for a 1M-creature world in the same loop that would update position. Note that it adds one cheap arithmetic op per creature to a pass you are already making.
3. Dense binning. Build the CSR bucket array (count, prefix-sum, scatter). Answer “neighbours within r” by reading the 3x3 cell block. Verify it gives the same counts as exercise 1.
4. Bolt-on hash vs dense bin. Build the same query with a HashMap<cell, Vec<u32>>. At 1M, time build and query for both. Reproduce the ~2.8x end-to-end gap; note where the hash spends its time (allocation, pointer-chasing) against the dense bin’s contiguous streams.
5. Recompute beats maintain. Measure the dense bin’s build as a fraction of its query. Confirm it is roughly 1%. Argue why maintaining a spatial index incrementally (to “save” the rebuild) optimises the wrong thing.
6. The pack-leader. Steer N agents toward the group two ways: each averaging the other N-1 positions (all-pairs), and each reading one centroid computed in a single pass. Time both; reproduce the O(N²) vs O(N) gap. Argue why the leader gives swarm-like behaviour without any agent knowing about any other.
7. (stretch) Z-order and the compaction. Replace the stripe pack with a Z-order (Morton) hash. Then order the §24 compaction by cell and re-time the neighbour query’s gather (§26). How much of the remaining query cost was the scattered gather?
8. The density wall and the representative. Take your dense-bin query from exercise 3. Grow the population in a fixed world (100K, 300K, 1M) and confirm the per-query cost grows faster than linearly - the 3x3 block is filling, the grid is quadratic again. Now answer the one-nearest query by keeping a single representative per cell (the first occupant) and reading the nine neighbour representatives; re-time the sweep and confirm it holds linear. Then measure how often the representative differs from the true nearest, and confirm every difference lies within one cell - the approximation is bounded by the grid’s own resolution.

Reference notes in 28_proximity_solutions.md.

What’s next

§29 - The wall at 10K → 1M is where these techniques start to bind. Code that ran fine at 10K stops running fine at 1M; the chapter is about finding out where and why.

Solutions: 28 - Proximity is a property of position

Numbers below are the modern-desktop figures from the proximity benchmark; the chapter’s Measurements table and code/README.md carry the four-machine spread.

Exercise 1 - The all-pairs wall

#![allow(unused)]
fn main() {
fn count_all_pairs(px: &[f32], py: &[f32], r: f32) -> u64 {
    let n = px.len();
    let mut total = 0u64;
    for i in 0..n {
        for j in 0..n {
            if i != j {
                let dx = px[i] - px[j];
                let dy = py[i] - py[j];
                if dx * dx + dy * dy <= r * r { total += 1; }
            }
        }
    }
    total
}
}

Measured: ~270 ms at N = 20 000. The cost is N², so 10K is ~4x cheaper and 1M is ~2500x more expensive (a thousand seconds - never). Twenty thousand creatures already blow a 33 ms frame budget eight times over, on a query the world has to answer every tick.

Exercise 2 - Cell as a derived column

#![allow(unused)]
fn main() {
fn cell_of(px: f32, py: f32, cell_size: f32) -> u32 {
    let x = (px / cell_size).floor() as i32 as u32 & 0xFFFF;
    let y = (py / cell_size).floor() as i32 as u32 & 0xFFFF;
    (x << 16) | y
}

// computed in the same pass that integrates motion:
for i in 0..n {
    px[i] += vx[i] * dt;
    py[i] += vy[i] * dt;
    cell[i] = cell_of(px[i], py[i], cell_size); // one extra arithmetic op
}
}

The cell column costs a divide, two floors, and some bit ops per creature, in a loop that is already streaming px/py. It rides on bandwidth motion is already paying. There is no separate structure and no separate pass.

Exercise 3 - Dense binning

#![allow(unused)]
fn main() {
// count -> prefix-sum -> scatter (a counting sort by cell)
let mut offsets = vec![0u32; ncells + 1];
for &c in &cell { offsets[c as usize + 1] += 1; }
for c in 0..ncells { offsets[c + 1] += offsets[c]; }
let mut items = vec![0u32; n];
let mut cursor = offsets.clone();
for i in 0..n {
    let c = cell[i] as usize;
    items[cursor[c] as usize] = i as u32;
    cursor[c] += 1;
}

// query: the 3x3 block of cells around creature i, as contiguous ranges
for (nx, ny) in neighbours_3x3(i) {
    let c = ny * GX + nx;
    for &j in &items[offsets[c] as usize .. offsets[c + 1] as usize] {
        // distance test against j
    }
}
}

The counts match exercise 1 exactly (it is the same query, restricted to candidates that can possibly be within r). Three linear passes build it; the query reads contiguous ranges.

Exercise 4 - Bolt-on hash vs dense bin

The HashMap<u32, Vec<u32>> version inserts each creature into grid.entry(cell).or_default().push(i), and the query does grid.get(&c) for each of nine cells.

Measured at 1M:

End to end the dense bin is ~2.8x faster, its build ~8x faster. The hash pays for a heap allocation per occupied cell, hashes a key for every insert and every one of the nine lookups per query, and chases a pointer into a separately-allocated Vec whose contents are scattered across the heap. The dense bin has one offsets array and one items array; the query indexes offsets and walks a contiguous slice. Same algorithm, opposite memory behaviour.

Exercise 5 - Recompute beats maintain

The dense bin’s build is 3.7 ms; its query is 513 ms. The build is 0.7% of the query. A maintained index exists to avoid paying that rebuild - so it is buying a 0.7% saving, and paying for it with insert/remove bookkeeping on every move, a structure that must stay correct across ticks, and (for a hash) pointer-chasing on every access. Recomputing the whole binning from scratch each tick, in the motion pass, is both simpler and faster. The maintenance you were told you need is maintenance of the wrong thing.

Exercise 6 - The pack-leader

#![allow(unused)]
fn main() {
// all-pairs cohesion: each agent steers toward the average of all others - O(N^2)
for i in 0..n {
    let (mut sx, mut sy) = (0.0, 0.0);
    for j in 0..n { if i != j { sx += px[j]; sy += py[j]; } }
    steer[i] = (sx / (n - 1) as f32 - px[i], sy / (n - 1) as f32 - py[i]);
}

// one leader/centroid: one pass for the centre, every agent reads it - O(N)
let (mut cx, mut cy) = (0.0, 0.0);
for i in 0..n { cx += px[i]; cy += py[i]; }
cx /= n as f32; cy /= n as f32;
for i in 0..n { steer[i] = (cx - px[i], cy - py[i]); }
}

Measured at N = 20 000: all-pairs cohesion ~240 ms, one centroid ~0.03 ms - about 9000x, and the gap grows linearly with N because one side is O(N²) and the other O(N). The leader gives swarm-like motion without any agent knowing about any other: each beast only ever reads the single shared centre (or, for richer behaviour, an invisible leader NPC that does the pack’s navigation). Coordination cost collapses from “every pair” to “one pass.”

Exercise 7 - Z-order and the compaction

A stripe pack (x << 16 | y) puts cells with the same x adjacent; cells one step apart in y are adjacent, but a step in x jumps 65 536 cells away. A Z-order (Morton) curve interleaves the bits so 2D neighbours stay close in the 1D order in both axes:

#![allow(unused)]
fn main() {
fn morton_2d(x: u16, y: u16) -> u32 {
    let mut x = x as u32; let mut y = y as u32;
    x = (x | (x << 8)) & 0x00FF00FF; x = (x | (x << 4)) & 0x0F0F0F0F;
    x = (x | (x << 2)) & 0x33333333; x = (x | (x << 1)) & 0x55555555;
    y = (y | (y << 8)) & 0x00FF00FF; y = (y | (y << 4)) & 0x0F0F0F0F;
    y = (y | (y << 2)) & 0x33333333; y = (y | (y << 1)) & 0x55555555;
    x | (y << 1)
}
}

Binning finds the candidates, but the neighbour query still gathers their positions from scattered slots - that is the bulk of the ~513 ms. Order the §24 compaction by Morton cell so a cell’s creatures sit on adjacent cache lines, and the gather streams (§26’s measured ~4x for scattered-vs-dense). The compaction runs on the GC’s slow cadence, not as a separate per-tick sort: §28 decides which cell; the compaction makes reading a cell sequential.

Exercise 8 - The density wall and the representative

In a fixed world, growing the population grows the density: a cell of fixed size holds more creatures, so the 3x3 block a query reads holds O(N) of them, and the per-query cost is O(N) - over all targets, O(N²). The dense bin did not buy O(N); it bought O(N) at constant density. Sweeping 100K -> 300K -> 1M in a fixed world, the per-query time grows faster than 3x per step (the reference forage grew ~8x then ~11x) - the quadratic, with the grid in place.

The representative breaks it. Keep one occupant per cell (the first scattered in is deterministic and fine); a query reads at most nine representatives regardless of how full the cells are, so the work is O(targets) - linear even at fixed world (the reference held ~3x per 3x). The catch is that you have answered a slightly different question: “the nearest representative,” not “the nearest creature.” But both the kept and the dropped occupants of a cell sit inside the same cell, so the representative is within one cell-width of the true nearest - and the cell is already the resolution at which the grid knows position. Counting the mismatches, you find a large fraction differ under crowding yet none by more than a cell. The approximation is bounded by the grid you already chose; for “eat what you can reach,” that is free.

Two routes back to O(N), then, and they differ in kind: hold the density (grow the world with the population - a constraint on the simulation), or collapse the cell to a representative (a change to the query). The first keeps the exact nearest; the second trades a within-cell approximation for O(N) in a world that does not grow. Part II measures both on the simulator at scale.

29 - The wall at 10K → 1M

Concept node: see the DAG and glossary entry 29.

A simulator that runs cleanly at 10 000 creatures often grinds to a halt at 1 000 000. Not because the algorithm changed - because constant factors that were invisible at the smaller scale now bind.

This chapter is about finding the wall. The fixes are techniques you already have: narrower fields (§7), subscriptions (§26), working-set discipline (§27), spatial binning and locality (§28), pre-sized buffers, batched cleanup. The chapter’s job is to teach the reader to measure - to find which constant factors blew up.

Constant-factor bugs that bind at 10K → 1M:

• Reallocation. A to_insert: Vec<CreatureRow> that grew lazily was fine at 100 pushes per tick (10K creatures × 1% reproduction). At 10K pushes per tick (1M × 1%), the reallocations dominate. Fix: Vec::with_capacity(estimated_max).
• Linear scans. hungry.iter().any(|&s| s == target) was 0.1 ms at 10K, but 10 ms at 1M. Fix: the sparse set (§23) - O(1) membership, no per-creature flag.
• Cache spillover. creature working set at 10K is 200 KB (L2-resident). At 1M it is 20 MB (L3-resident). Per-element time triples. Fix: narrower fields (§7) and the spatial compaction for locality (§28); for a system that touches only a subset, a subscription (§26).
• HashMap iteration order. A HashMap<u32, _> iterated by systems that need deterministic order. At 10K the cost was tolerable; at 1M the bandwidth cost is high. Fix: BTreeMap or Vec<(K, V)>.
• Per-tick allocation. A system that allocates a fresh Vec per tick was fine when the Vec was 1 KB. At 1M it is 100 KB; allocation latency starts to matter. Fix: reuse buffers across ticks.
• Logging. A println! per creature was tolerable at 10K. At 1M it is the simulator’s bottleneck. Fix: buffered logging, periodic snapshots, or simply turn it off.

The pattern: any cost that was O(1) per creature, multiplied by 1M, is no longer free. Anything that was O(N) per tick at 10K is now O(N²)-equivalent in wall time. The fixes are local - each cost is a single-line change - but finding them requires measurement.

The right tool is a profiler. cargo flamegraph (or perf record + perf report) tells you where the time goes. The same simulator at 10K and 1M produces different flame graphs; the wall is the difference.

A useful exercise: run your simulator at 10K for 1000 ticks; time it. Run at 1M for 100 ticks (same total entity-ticks); time it. The 1M version should take ~10× longer, not 100×. If it takes 100×, something has crossed a constant-factor wall and the profiler will show you what.

The fix is structural. Apply the techniques: narrow fields, subscriptions, working set, spatial binning and locality, pre-sized buffers, batched cleanup, deterministic structures. Each is a chapter you have already read. The wall is the moment they all become non-optional.

Exercises

1. Calibration. Run your simulator at N = 10K for 1000 ticks. Time it. Note the wall-clock total.
2. Scale up. Run at N = 1M for 100 ticks (same total entity-ticks). Time it. Compute the ratio.
3. Profile. Use cargo flamegraph (or perf) on the 1M run. Identify the top three hottest functions.
4. Pre-size to_insert. Apply Vec::with_capacity to your cleanup buffers. Re-run; re-profile. Did the hot list change?
5. Subscribe a subset system. Take a system that acts on a fraction of creatures (starvation, reproduction) and give it a slot-keyed subscription table (§26) instead of scanning all 1M and branching. Re-run; re-profile. The scan-all frame should drop off the flame graph; the system’s work falls in proportion to the subscribed fraction.
6. Use index maps. Replace any linear iter().any() with the §23 id_to_slot lookup. Re-run; re-profile.
7. (stretch) Find one new wall. Pick any system in your simulator and find one constant factor that scales worse than expected. The fix is usually one of the techniques above; identifying which one is the lesson.

Reference notes in 29_wall_10k_to_1m_solutions.md.

What’s next

§30 - Moving beyond the wall takes the next step: when even your fastest, tightest, subscription-driven, compacted-for-locality simulator no longer fits in RAM, the architecture itself shifts.

Solutions: 29 - The wall at 10K → 1M

Exercise 1-2 - Calibration and scale-up

Run the simulator at 10K for 1000 ticks: typical wall-clock ~1-3 s.

Run at 1M for 100 ticks (same total entity-ticks): expect ~10-30 s if the simulator is well-tuned, ~100-300 s if it has unaddressed walls.

The ratio is the diagnostic. Anything above ~15× indicates that constant-factor walls are binding.

Exercise 3 - Profile

cargo flamegraph produces a flamegraph.svg. The wide frames at the top of the graph are the hottest functions. Common offenders at the 1M boundary:

• <Vec as Extend>::extend - uncapped reallocations
• core::iter::any over a Vec<u32> - linear scan that should be an indexed lookup
• std::collections::HashMap::iter - non-deterministic, slow at scale
• core::fmt::Write - println! in the hot path

Exercise 4 - Pre-size to_insert

#![allow(unused)]
fn main() {
let estimated_max = creatures.len() / 50; // 2% reproduction rate, with margin
let to_insert: Vec<CreatureRow> = Vec::with_capacity(estimated_max);
}

Re-profile: the Vec::extend frames should shrink dramatically. A typical fix removes 5-15 % of total wall time.

Exercise 5 - Subscribe a subset system

Give a subset system (say apply_starve, which acts only on the hungry) a slot-keyed subscription table (§26) instead of scanning all 1M creatures and branching on a flag. Re-profile: the scan-all frame disappears from the flame graph, and the system’s cost falls in proportion to the subscribed fraction - at 1 % subscribed the ebp_partition benchmark shows roughly a 14x drop versus scan-all-and-branch. Motion, which touches every creature, gains nothing from a subscription; its lever is narrower fields (§7) and the spatial compaction (§28).

Exercise 6 - Index maps

Replace hungry.iter().any(|&s| s == target) with the sparse-set test hungry.is_member(target) (one array read and a sentinel check, §23). A function that was O(N) per call is now O(1), with no per-creature boolean.

For a system that asks the question 100K times per tick at 1M creatures, this is the difference between 100 s and 0.005 s per tick.

Exercise 7 - Find one new wall

Open-ended. Common discoveries the first time a reader runs this exercise:

• A Vec<Box<T>> somewhere in the code, costing one allocation per element.
• A clone() inside a hot loop where a & would do.
• A String::from(...) in a logging path that runs millions of times.
• A HashMap::contains_key where a Vec<bool> mask would be O(1) and 100× faster.

In each case, the fix is a one-line change once the wall is found. The challenge is finding the wall, not removing it.

30 - Moving beyond the wall

Concept node: see the DAG and glossary entry 30.

At 100 million creatures with 24 bytes of hot data each, the working set is 2.4 GB. At a billion, 24 GB. Most desktops have 16-64 GB of RAM. The simulator can no longer hold its world and its history and the OS and whatever else and operate at speed.

The fix is streaming: only the relevant slice of the world is in memory at any one time; the rest lives on disk and is read on demand.

The shape:

#![allow(unused)]
fn main() {
struct StreamingWorld {
    in_memory: Window,    // a small contiguous range of recent state
    disk:      Archive,   // the rest, append-only on disk
}
}

A window of recent state lives in memory, indexed for cheap query. Older state lives on disk in append-only chunks; it is read into the window when a query needs it.

This pattern shows up wherever this scale matters:

• Time-series databases (Prometheus, InfluxDB): recent metrics in RAM; older series compressed and disk-resident.
• Game replay systems: the last 30 seconds replayable from a memory ring; the full match streamed from a server.
• Event-sourced systems: recent state cached; the full event log on disk; replay reconstructs.
• Database write-ahead logs: append to log; flush to data files; the data files become disk-resident; recent log + memory hold the active set.

For the simulator, streaming entails three architectural shifts:

The log is the canonical state. The world’s tables are derivable from the log. If the log is complete and durable, every other in-memory representation is reconstructible. This is the structural framing of §37 - The log is the world: the log is not a record of state, it is the state.

Persistence is serialisation of tables. A snapshot is the world’s current SoA, written as a stream of (entity, key, value) triples - the same shape it has in memory. Recovery is reading the triples back. There is no separate domain model; serialisation is transposition, not translation. This is §36.

Storage is a cost like any other. Reading from disk costs bandwidth and IOPS, just as reading from RAM costs cache-line loads. Storage systems with bandwidth (bytes per second) and IOPS (operations per second) limits must be counted against the tick budget. SQLite, network sockets, distributed file systems - all are storage systems with their own cost profiles. This is §38.

Cleanup amortises the write cost. The cleanup system you built in §22 already batches in-memory mutations to avoid mid-tick races. At streaming scale, the same pattern earns its keep again, for a second reason: it batches disk writes. Without batching, 10 000 individual mutations per tick would mean 10 000 disk writes - at 100 µs per write, a full second of I/O per tick, far over budget. With cleanup, those 10 000 mutations become one durable batch per tick: a handful of disk pages flushed sequentially to the log. One syscall, one trip through the block layer, one (or a few) DMA transfers - versus 10 000 of each. The cost is amortised across the batch, not paid per row. The mechanics - page cache, vectored I/O, fsync semantics - belong to §38; the gradient is what matters here. The architecture you assembled in §22 was already the streaming architecture in miniature; this section just lets you spell it out at scale.

The simulator at streaming scale is no longer a process running in memory; it is a pipeline between a memory window and a durable log, with the systems running on whatever slice of the world is currently mounted. Every read might fault to disk; every write is buffered into the next cleanup’s batch.

The transition from in-memory to streaming is the largest architectural shift in the book. Below this wall, the simulator is a single-process program with its working state in RAM. Above it, the simulator is closer to a database with its working state on disk and a small in-memory hot path. The techniques are different; the discipline is the same - layout, working set, ownership, determinism - applied at a different scale.

This wall is where most projects either re-architect or quietly accept slower-than-target performance. The book points at the wall and names the techniques; it does not pretend the techniques are free.

Exercises

1. Compute your streaming threshold. Estimate your simulator’s per-creature footprint at full SoA. Divide your machine’s RAM (the half you can spare for the simulator) by that footprint. The result is roughly the N at which the simulator hits the streaming wall.
2. Predict the cost. A disk read is ~100 µs (NVMe SSD), ~200-500 µs (SATA SSD), or ~10 ms (spinning disk). At a 33 ms tick budget, how many disk reads can a tick afford? How many might a system want to make?
3. Snapshot a small world. Write a function that serialises your simulator’s current state to a single file (one file, no schema gymnastics, just write the columns). Read it back into a fresh world. Confirm the simulator continues running indistinguishably.
4. A windowed log. Implement an append-only log with a fixed in-memory window. Older entries go to disk; new entries always go to memory. Verify queries inside the window are fast; queries outside the window pay the disk cost.
5. Log-as-world. With the windowed log from exercise 4, reconstruct creature state at an earlier tick by replaying the log over the most recent snapshot whose tick is ≤ the requested one. Compare query speed to the in-memory case.
6. (stretch) Document your bound. Write down, for your simulator, the largest N you can run while staying inside a 33 ms tick budget. Include footprint, cache regime, and any disk-bound cost. Above this N, the simulator needs the streaming architecture.

Reference notes in 30_streaming_wall_solutions.md.

What’s next

You have closed Scale. The next phase is Concurrency, starting with §31 - Disjoint write-sets parallelize freely. The simulator is about to start running on more than one thread.

Solutions: 30 - The wall at 1M → streaming

Exercise 1 - Streaming threshold

Per-creature footprint at full SoA: hot ~24 B, cold ~16 B, plus presence flags ~8 B = ~48 B. Plus index maps and indices into derived tables: round to ~64 B per live creature.

For a desktop with 32 GB RAM, allocating 16 GB to the simulator: 16 × 10⁹ / 64 = 250 million creatures. Above ~250M, the simulator must start streaming.

In practice the threshold is lower because logs, snapshots, and OS overhead consume RAM too. A safe budget might be 50-100M before the streaming architecture is needed.

Exercise 2 - Disk read cost

NVMe SSD: ~100 µs per read. A 33 ms tick budget is 33 000 µs / 100 µs ≈ 330 random disk reads. If a system would naturally make 10 000 disk reads per tick, the simulator is roughly 30× slower than budget unless reads are batched or sequential.

The fix is the same as for cache misses at the smaller scale: amortise the cost. Read a page of consecutive entries (4 KB = 64 cache lines = many rows) at one IOPS cost; touch all of them while the cost is paid.

Exercise 3 - Snapshot

#![allow(unused)]
fn main() {
fn snapshot_world(world: &World, path: &Path) -> std::io::Result<()> {
    let mut f = std::fs::File::create(path)?;
    use std::io::Write;
    let c = &world.creatures;
    f.write_all(&c.len().to_le_bytes())?;
    for &x in &c.px     { f.write_all(&x.to_le_bytes())?; }
    for &y in &c.py     { f.write_all(&y.to_le_bytes())?; }
    for &x in &c.vx     { f.write_all(&x.to_le_bytes())?; }
    for &y in &c.vy     { f.write_all(&y.to_le_bytes())?; }
    for &e in &c.energy { f.write_all(&e.to_le_bytes())?; }
    for &i in &c.id     { f.write_all(&i.to_le_bytes())?; }
    Ok(())
}

fn load_world(path: &Path) -> std::io::Result<World> {
    // Read the columns back in the same order. Sizes from the prefixed length.
    todo!()
}
}

After a snapshot + load round-trip, the simulator continues running indistinguishably if and only if the simulator is deterministic (§16). Determinism is the precondition for snapshot/load to mean what we expect.

Exercise 4 - Windowed log

#![allow(unused)]
fn main() {
struct WindowedLog<E> {
    in_memory: std::collections::VecDeque<E>,
    capacity:  usize,
    disk:      std::fs::File,
}

impl<E: Encode> WindowedLog<E> {
    fn push(&mut self, e: E) {
        if self.in_memory.len() == self.capacity {
            // Evict oldest to disk, append new in memory.
            let oldest = self.in_memory.pop_front().unwrap();
            oldest.encode(&mut self.disk).unwrap();
        }
        self.in_memory.push_back(e);
    }
}
}

Queries inside the in-memory window are O(1) per entry. Queries past the window pay one disk seek (~100 µs) plus a sequential read for the requested span. The cost difference is the streaming wall.

Exercise 5 - Log-as-world reconstruction

To recover state at tick T:

1. Find the most recent snapshot at tick S ≤ T.
2. Load the snapshot into a fresh world.
3. Replay log entries from S to T in order.

Replay speed depends on the events-per-tick rate. For most simulators, replay is 10-100× faster than original-tick rate (no rendering, no I/O outside the log read). Reconstructing 1000 ticks of history takes ~1 second of replay time.

Exercise 6 - Document your bound

A well-tuned simulator at 30 Hz on a typical desktop:

• ~10 M creatures comfortable (RAM-resident, sub-budget).
• ~100 M creatures possible with narrow fields and the spatial compaction (RAM-resident, near-budget).
• ~1 B creatures requires streaming (working set exceeds RAM).
• ~10 B+ requires distributed simulation across multiple machines (covered in the monograph).

Each step is a different architecture, with the techniques in the previous step still applying. The book ends below the streaming wall; the monograph picks up above it.

31 - Disjoint write-sets parallelize freely

Concept node: see the DAG and glossary entry 31.

Two systems can run in parallel if and only if their write-sets do not overlap. That is the rule. It is small. It is what node 25’s single-writer ownership buys you.

Concretely: in the simulator’s tick, motion writes creature.pos and creature.energy; food_spawn writes food. Their write-sets are disjoint. They can run on two different threads with no coordination - no locks, no atomics, no message-passing. The data layout makes the parallelism free.

#![allow(unused)]
fn main() {
use std::thread;

thread::scope(|s| {
    s.spawn(|| motion(&mut hot.px, &mut hot.py, &hot.vx, &hot.vy, &mut hot.energy, dt));
    s.spawn(|| food_spawn(&food_spawner, &mut food));
});
// both threads have completed before scope() returns
}

std::thread::scope is the Rust idiom that proves at compile time the two threads finish before the surrounding state is touched. The borrow checker enforces the disjoint-writes rule: if you tried to spawn two threads each holding &mut hot.px, the code would not compile.

The same shape works at finer grain. The simulator’s three appliers (apply_eat, apply_reproduce, apply_starve) all read pending_event and write disjoint things - apply_eat writes food, to_remove; apply_reproduce writes to_insert; apply_starve writes to_remove. Two of the three write the same table (to_remove). To parallelise them, give each its own segment of to_remove (one per thread), then merge at cleanup. The merge is Vec::extend_from_slice or equivalent - O(N) in the merged total, free relative to the work that produced it.

Three things this rule does for you:

No locks. A lock is a tax paid by every reader and writer of the locked thing. With single-writer ownership, locks are unnecessary; with disjoint write-sets, they remain unnecessary at the parallel boundary. The simulator at this scale has zero Mutex, zero RwLock, zero Atomic* in its inner systems.

Speedup is structural, not promised. N threads with disjoint work give N× speedup, modulo memory-bandwidth limits. That ceiling is real - at 50 GB/s of DDR5 bandwidth, eight threads cannot all do bandwidth-bound work in parallel; one thread saturates the bus. But for compute-bound work or for cache-resident loops, the speedup is close to N¹.

Tools without ceremony. The Rust ecosystem’s standard parallelism crate is rayon, which provides par_iter and par_chunks_mut for parallel iteration. With disjoint writes by construction, rayon::join and par_iter_mut work without changing the simulator’s design - they are conveniences over std::thread::scope, not new architectures.

The single-writer rule (§25) was the precondition. Disjoint write-sets is the rule applied across systems. Together, parallelism becomes a scheduling decision, not a design decision.

Measurements

Two disjoint systems run under thread::scope hit close to the ideal 2x on every machine - the write-sets do not collide, so there is nothing to serialise. Full output: code/README.md.

Exercises

You will need a multi-core machine. Most desktops and laptops qualify.

1. Two parallel systems. Wrap motion and food_spawn in std::thread::scope. Run a tick. Verify both completed and the world state is the expected combination.
2. Time the speedup. Run the same two systems serially. Run them in parallel via thread::scope. Compare. Speedup should be close to 2× when both systems are individually expensive and compute-bound; less if one dominates or the work is bandwidth-bound. Measured 1.8-2.0× across the four reference machines (scope_speedup).
3. A failing case. Try to run motion and apply_eat in parallel. Both write creature.energy. Rust’s borrow checker rejects the code. Note the error message - that is the architecture being enforced by the compiler.
4. rayon::join. Replace thread::scope with rayon::join((|| motion(...), || food_spawn(...))). Confirm the same behaviour. Adding rayon to Cargo.toml is a §42 dependency-pricing decision in miniature: read what the crate gives you, decide consciously.
5. Per-thread segments. Split to_remove into 8 thread-local Vec<u32>s. Run 8 threads of apply_starve, each producing its own segment. Merge at the end. Verify the merge produces the same result as a single-threaded run.
6. (stretch) Find the bandwidth ceiling. Time motion at 1, 2, 4, 8 threads on a 1M-creature world. Plot speedup vs thread count. The plot is roughly linear up to the memory-bandwidth limit, then flat.

Reference notes in 31_disjoint_writes_parallelize_solutions.md.

What’s next

§32 - Partition, don’t lock takes the next step: when one system must write a single table from multiple threads, you split the table, not the access.

Solutions: 31 - Disjoint write-sets parallelize freely

Exercise 1 - Two parallel systems

#![allow(unused)]
fn main() {
std::thread::scope(|s| {
    s.spawn(|| motion(&mut hot.px, &mut hot.py, &hot.vx, &hot.vy, &mut hot.energy, dt));
    s.spawn(|| food_spawn(&food_spawner, &mut food));
});
}

Both systems write disjoint tables. The borrow checker is satisfied; the threads cannot interfere. After the scope returns, both threads have finished and the world is consistent.

Exercise 2 - Time the speedup

At 1M creatures: motion alone ≈ 3 ms; food_spawn alone ≈ 0.1 ms. Serial total ≈ 3.1 ms. Parallel total ≈ 3 ms (food_spawn finishes first; motion dominates). Speedup is close to 1× because the workload is dominated by motion.

When both systems are individually expensive (e.g. food at 1M items as well), serial ≈ 6 ms, parallel ≈ 3.5 ms (memory bandwidth shared); speedup ≈ 1.7×.

Exercise 3 - A failing case

std::thread::scope(|s| {
    s.spawn(|| motion(&mut hot.px, &mut hot.py, &hot.vx, &hot.vy, &mut hot.energy, dt));
    s.spawn(|| apply_eat(&pending, &food, &mut hot.energy));
});

Rust rejects:

error[E0524]: two closures require unique access to `hot.energy` at the same time

The architecture’s safety is the language’s safety. Compile-time, not run-time.

Exercise 4 - rayon::join

#![allow(unused)]
fn main() {
use rayon::join;

join(
    || motion(&mut hot.px, &mut hot.py, &hot.vx, &hot.vy, &mut hot.energy, dt),
    || food_spawn(&food_spawner, &mut food),
);
}

Identical behaviour to thread::scope for two-system parallelism. rayon adds value at finer-grained parallelism (par_iter, work-stealing); for the simulator’s two-system pattern, join is sufficient.

Exercise 5 - Per-thread segments

#![allow(unused)]
fn main() {
const N: usize = 8;
let mut segments: Vec<Vec<u32>> = (0..N).map(|_| Vec::new()).collect();
let chunk = energy.len().div_ceil(N);

thread::scope(|s| {
    for (t, segment) in segments.iter_mut().enumerate() {
        let energy_chunk = &energy[t * chunk .. ((t+1) * chunk).min(energy.len())];
        let ids_chunk    = &ids[t * chunk    .. ((t+1) * chunk).min(ids.len())];
        s.spawn(move || apply_starve(energy_chunk, ids_chunk, segment));
    }
});

let to_remove: Vec<u32> = segments.into_iter().flatten().collect();
}

Each thread writes its own Vec<u32>. Merge at the end via flatten. The merge is O(total) - same cost as building the single-threaded vec, but distributed across threads.

Exercise 6 - Bandwidth ceiling

Above 4-6 threads, memory bandwidth becomes the bottleneck. The 8-core ceiling is around 5×, not 8×, because all cores pull from the same memory bus. Compute-bound work scales further; bandwidth-bound work hits this ceiling.

For your machine, the ceiling depends on the memory controller’s throughput. DDR5-5600 dual-channel tops out around 60 GB/s sustained; eight cores doing 50 GB/s of bandwidth-bound work each would need 400 GB/s - they cannot.

32 - Partition, don’t lock

Concept node: see the DAG and glossary entry 32.

§31 said “disjoint write-sets parallelise freely”. What if the system has to write one table from many threads? Motion at 1M creatures wants to update creature.pos for every creature; the table is one. Eight threads, one table - looks like a lock case.

It is not. The fix is to partition the data, not to lock the access.

Each thread takes a slice of the table. Thread t writes slots t * N/8 .. (t+1) * N/8 and only those slots. The slices are disjoint by construction; no thread can write where another is writing. Inside each slice, a single thread is the writer - node 25’s ownership rule still holds, just at the slice level instead of the table level.

#![allow(unused)]
fn main() {
use std::thread;

thread::scope(|s| {
    let chunk = pos.len().div_ceil(8);
    let pos_chunks    = pos.chunks_mut(chunk);
    let vel_chunks    = vel.chunks(chunk);
    let energy_chunks = energy.chunks_mut(chunk);

    for ((p, v), e) in pos_chunks.zip(vel_chunks).zip(energy_chunks) {
        s.spawn(move || {
            for i in 0..p.len() {
                p[i].0 += v[i].0 * dt;
                p[i].1 += v[i].1 * dt;
                e[i] -= burn * dt;
            }
        });
    }
});
}

chunks_mut is the standard library’s splitter for &mut [T] into disjoint sub-slices. Each chunk is a &mut [T] with its own ownership; the borrow checker is satisfied. No Mutex, no atomic, no contention.

The choice of partitioning matters.

By entity range (above): simple, works when access is uniform. Each thread does the same work on a different slice.

By spatial cell (after the spatial compaction, §28): each thread takes a region of the world. Useful when interactions are local - neighbours-only collisions, regional behaviours. Threads at boundary cells need a small synchronisation step (or a halo region copied into each thread’s input).

By hash: each thread takes ids whose hash modulo N matches its thread number. Useful when access is uniform but you want stable thread-to-data mapping across ticks.

By workload weight: each thread takes a number of rows weighted by expected work per row. Useful when rows differ in cost (e.g. some creatures have many neighbours, others have none). Requires a profiling pass or a heuristic.

The partition shape is the design choice; the partition mechanism (slicing) is trivial in Rust.

A subtlety: even with partitioning, false sharing (next section) can wreck the performance gains. If two threads write adjacent fields in the same cache line, the hardware coherency protocol forces them to take turns despite the logical independence. The fix is alignment, padding, or partitioning at cache-line boundaries - §33 develops it.

The pattern is the right answer to “but I have one big table”. You almost never need a lock; you need a partition.

Exercises

1. Partition motion. Use chunks_mut to split pos, vel, and energy into 8 chunks. Run motion across 8 thread::scope threads. Compare to single-threaded.
2. Speedup at scale. Time partitioned motion at N = 100K, 1M, 10M creatures with 1, 2, 4, 8 threads. Plot speedup. Note where the bandwidth ceiling kicks in.
3. Spatial partition. After running §28’s spatial compaction, partition by spatial region (e.g. 8 vertical stripes of the world). Each thread handles one stripe. Compare with the entity-range partition. Does the spatial version pay off for next_event?
4. Workload-weighted partition. Suppose 90 % of creatures are idle and 10 % are active. A naive partition gives most threads almost no work and one thread all the work. Implement a partition that balances active count, not total count. Time both.
5. (stretch) rayon::par_chunks_mut. Replace your manual thread::scope + chunks_mut with pos.par_chunks_mut(chunk_size). Same result, less code. Note that rayon’s work-stealing scheduler internally rebalances unbalanced workloads.

Reference notes in 32_partition_dont_lock_solutions.md.

What’s next

§33 - False sharing names the hardware-level pitfall that can sink the partition pattern: two threads writing different fields in the same cache line slow each other down despite logical independence.

Solutions: 32 - Partition, don’t lock

Exercise 1 - Partition motion

#![allow(unused)]
fn main() {
use std::thread;

const N_THREADS: usize = 8;
let chunk = pos.len().div_ceil(N_THREADS);

thread::scope(|s| {
    for ((p, v), e) in pos.chunks_mut(chunk)
        .zip(vel.chunks(chunk))
        .zip(energy.chunks_mut(chunk))
    {
        s.spawn(move || {
            for i in 0..p.len() {
                p[i].0 += v[i].0 * dt;
                p[i].1 += v[i].1 * dt;
                e[i] -= burn * dt;
            }
        });
    }
});
}

chunks_mut is the standard library’s slice splitter. Each thread receives its own &mut [T]; the borrow checker is satisfied; no Mutex is required.

Exercise 2 - Speedup at scale

At 10M, the working set is 200 MB, well past L3. The loop is memory-bandwidth bound; adding threads stops helping past 4 cores. At 1M (in L3), 8 threads gives ≈ 5× speedup.

Exercise 3 - Spatial partition

After §28’s spatial compaction, creatures in the same region are adjacent in memory. Assigning each thread a region means the cache lines a thread loads are the cache lines that thread uses - no cross-thread cache traffic.

For systems with neighbour reads (next_event’s collision check), spatial partitioning is roughly 10-30 % faster than entity-range partitioning at scale, depending on neighbour density.

Exercise 4 - Workload-weighted partition

A naive partition with 1M creatures and 100K active gives some threads all the work and others none. A weighted partition divides the active set:

#![allow(unused)]
fn main() {
let active: Vec<u32> = /* ids of active creatures */;
let active_per_thread = active.len().div_ceil(N_THREADS);

thread::scope(|s| {
    for chunk in active.chunks(active_per_thread) {
        s.spawn(move || drive_active(chunk, /* ... */));
    }
});
}

Each thread gets ≈ 12 500 active creatures. Cost-per-thread is balanced; total time ≈ total_active_work / N_THREADS. The naive version would have one thread doing all the active work → speedup ≈ 1×.

Exercise 5 - rayon::par_chunks_mut

#![allow(unused)]
fn main() {
use rayon::prelude::*;

pos.par_chunks_mut(chunk_size)
   .zip(vel.par_chunks(chunk_size))
   .zip(energy.par_chunks_mut(chunk_size))
   .for_each(|((p, v), e)| {
       for i in 0..p.len() {
           p[i].0 += v[i].0 * dt;
           p[i].1 += v[i].1 * dt;
           e[i]   -= burn * dt;
       }
   });
}

Rayon’s work-stealing scheduler handles unbalanced workloads automatically: a thread that finishes its chunk early steals work from a slower thread. For uniform work, performance matches manual chunks_mut; for unbalanced work, it can outperform.

The dependency cost: rayon brings in a small ecosystem (~50 KB binary impact, a global thread pool, a few transitive crates). For most simulators this is a clear win; for embedded targets or fully reproducible builds it may not be. The §41 from-scratch rule applies: write the manual thread::scope version first, then decide.

33 - False sharing

Concept node: see the DAG and glossary entry 33.

You partitioned the table. Each thread folds its own disjoint slice into its own accumulator. The work is balanced. The speedup is… 0.4× - the parallel version runs slower than a single thread (measured 0.26-0.42× across the four reference machines)¹. Where did the parallelism go?

Probably to false sharing.

The CPU cache works on 64-byte cache lines. When a thread writes to address X, the cache controller invalidates that line everywhere else - every other CPU’s cache must throw away its copy and reload. If two threads are writing to different addresses but in the same cache line, every write triggers an invalidation on the other thread’s cache. The threads slow each other down without ever logically conflicting.

A pathological case: eight threads each incrementing one entry in [u64; 8]. The array is exactly 64 bytes - one cache line. All eight threads write to that line. Every write invalidates the other seven caches. The threads run slower together than one thread alone - true negative scaling.

The fix is to put each thread’s data on its own cache line. Either pad the underlying value:

#![allow(unused)]
fn main() {
#[repr(align(64))]
struct CachePadded(u64);

let counters: [CachePadded; 8] = std::array::from_fn(|_| CachePadded(0));
}

Or split into separate allocations (each Vec lives in its own heap region, normally far apart). Or use thread-local storage. Or partition at cache-line granularity from the start.

The Rust idiom for the padding pattern is crossbeam_utils::CachePadded<T> from the crossbeam-utils crate, which exists for exactly this case.

False sharing is a hardware concern, not a Rust concern. The borrow checker sees no problem with eight &mut u64 references at disjoint addresses; the hardware sees one cache line and serialises the access. The bug is invisible at the language level. It shows up only as performance - the parallel version is mysteriously slow.

How to find it. Profile with perf stat -e cache-misses (or its equivalent on your platform). False sharing produces high cache-misses despite supposedly disjoint writes. If profiling shows your parallel system has surprisingly high cache traffic, false sharing is a likely cause.

How to avoid it without painful debugging. Make per-thread data structurally separate from the start. Each thread gets its own Vec<T> (separate allocation, separate cache lines). Merge at the end (§31’s pattern for to_remove with per-thread segments). The merge is cheap; the false-sharing avoidance is structural.

The takeaway: physical layout matters even for logically disjoint data. Two &muts pointing at different addresses do not parallelise freely if those addresses are within 64 bytes. The fix is alignment or separation. The detection is profiling.

Line size is not always 64 bytes

64 bytes is the size on the hardware this book measures, but it is not universal. The false-sharing unit by architecture, separated into what the book has run on versus what it cites:

1. Measured on the i7 / NUC / Ryzen boxes. Intel’s adjacent-line prefetch can make the effective unit 128 B - pad to 128 to avoid false sharing.
2. Measured on a Raspberry Pi 4 (Cortex-A72). Runtime query: CTR_EL0.
3. Implementation-defined; not fixed by the ISA.

Cache lines have grown with the hardware: 32 bytes on pre-Pentium-4 x86 and many embedded cores, 64 across most of today’s desktops and phones, 128 on Apple Silicon and POWER, 256 on IBM Z. This book assumes 64 and pads to 128 where false sharing demands it; on anything else, query the line size rather than trust the default: getconf LEVEL1_DCACHE_LINESIZE, sysconf(_SC_LEVEL1_DCACHE_LINESIZE), or /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size. The coherency granule can differ from the fetch line; the coherency unit is what governs false sharing.

If you have one of the unmeasured machines and run the suite, send the numbers and they go in.

Measurements

Below 1.0 means the “parallel” run is slower than a single thread - real negative scaling from false sharing. The partitioned-reduction row is the realistic case (disjoint input slices, a packed per-thread accumulator); padding each accumulator to its own cache line recovers the speedup. Full output: code/README.md.

Exercises

1. The pathological counter. Build the 8-thread case with eight AtomicU64 in one cache line:

   #![allow(unused)]
   fn main() {
   use std::sync::atomic::{AtomicU64, Ordering};
   let counters: [AtomicU64; 8] = std::array::from_fn(|_| AtomicU64::new(0));
   // ... 8 threads, each incrementing counters[t] in a tight loop
   }

   Time the parallel version against a single-threaded loop doing the same total work. The parallel version should be slower - true negative scaling.
2. The padded version. Pad each counter to its own cache line via #[repr(align(64))]. Re-run. The parallel version should now scale near-linearly with thread count.
3. A real example. In your simulator’s per-thread to_remove segments (§31 exercise 5), check whether the thread-local Vec<u32> allocations might land in the same cache line. They normally should not - separate Vecs have their data on the heap, which the allocator distributes - but if performance is unexpectedly poor, this is one place to look.
4. Adjacent struct fields. Build a struct with two u64 fields. Spawn two threads, one writing each field. They are at adjacent addresses, same cache line. Time vs. two u64 in separate allocations.
5. (stretch) Find your cache-line size. getconf LEVEL1_DCACHE_LINESIZE on Linux. Verify it is 64. Some chips use 128-byte lines (especially Apple Silicon at certain levels); if you are on one, #[repr(align(64))] is not enough - you need 128.

Reference notes in 33_false_sharing_solutions.md.

What’s next

§34 - Order is the contract ties parallelism back to the determinism rule from §16: parallelism is allowed inside a step, never across steps.

Solutions: 33 - False sharing

Exercise 1 - The pathological counter

#![allow(unused)]
fn main() {
use std::sync::atomic::{AtomicU64, Ordering};
use std::thread;
use std::time::Instant;

const ITERS: u64 = 10_000_000;
let counters: [AtomicU64; 8] = std::array::from_fn(|_| AtomicU64::new(0));

let t = Instant::now();
thread::scope(|s| {
    for c in counters.iter() {
        s.spawn(|| {
            for _ in 0..ITERS {
                c.fetch_add(1, Ordering::Relaxed);
            }
        });
    }
});
println!("8 threads on one cache line: {:?}", t.elapsed());
}

Compare with a single-threaded loop doing 8 × ITERS increments on a single counter. On most chips the single-threaded version is faster than the 8-thread version. True negative scaling.

Exercise 2 - The padded version

#![allow(unused)]
fn main() {
#[repr(align(64))]
struct Padded(AtomicU64);

let counters: [Padded; 8] = std::array::from_fn(|_| Padded(AtomicU64::new(0)));
}

Each Padded occupies a full cache line. The 8 padded counters span 8 cache lines. Re-time the parallel loop. It should now scale near-linearly - typically 6-8× faster than single-threaded.

Exercise 3 - Per-thread to_remove

The data inside each thread’s Vec<u32> lives on the heap, in regions the allocator distributes to be far apart (typically separated by at least one page = 4 KB). False sharing on the data is unlikely.

The Vec headers (the (ptr, len, cap) fields, 24 bytes each on 64-bit), if stored adjacent in a parent struct or array, can share a cache line. If you observe poor scaling, padding the parent struct fixes it:

#![allow(unused)]
fn main() {
#[repr(align(64))]
struct ThreadLocalVec(Vec<u32>);

let segments: [ThreadLocalVec; 8] = std::array::from_fn(|_| ThreadLocalVec(Vec::new()));
}

Exercise 4 - Adjacent struct fields

#![allow(unused)]
fn main() {
struct TwoCounters { a: AtomicU64, b: AtomicU64 } // 16 bytes, one cache line

let counters = TwoCounters { a: AtomicU64::new(0), b: AtomicU64::new(0) };

thread::scope(|s| {
    s.spawn(|| {
        for _ in 0..ITERS { counters.a.fetch_add(1, Ordering::Relaxed); }
    });
    s.spawn(|| {
        for _ in 0..ITERS { counters.b.fetch_add(1, Ordering::Relaxed); }
    });
});
}

Two threads, separate fields, same cache line. Performance is similar to two threads contending on one field - the line is invalidated on every write either way.

Fix: pad each field to its own cache line.

#![allow(unused)]
fn main() {
#[repr(align(64))]
struct PaddedAtomic(AtomicU64);

struct TwoCounters { a: PaddedAtomic, b: PaddedAtomic } // 128 bytes, two lines
}

Exercise 5 - Find your cache-line size

$ getconf LEVEL1_DCACHE_LINESIZE
64

x86 and most ARM Cortex-A: 64 bytes. Apple Silicon: 128 bytes for some cache levels. AArch64: variable; 64 or 128 depending on the chip. If padding to 64 bytes does not eliminate false sharing on a particular chip, try 128.

crossbeam_utils::CachePadded<T> checks the platform at compile time and pads to 128 on platforms that need it. For portable code, use it instead of hardcoded #[repr(align(64))].

34 - Order is the contract

Concept node: see the DAG and glossary entry 34.

§31, §32, and §33 unlocked parallelism. The natural temptation is to run everything in parallel - let the OS scheduler decide which system runs when, fan systems out across all available cores, push throughput up. This is wrong.

The system DAG (§14) is the contract for the simulator’s behaviour. Two systems with overlapping write-sets must run in a defined order. Two systems on the same DAG level may run in parallel - but they must both complete before any system that reads their outputs begins. Parallelism is allowed inside a step; it is never allowed across steps.

The reason is determinism (§16). Same inputs + same system order = same outputs. If apply_eat, apply_reproduce, and apply_starve run in undefined order - say, the first one to finish gets to write to_remove first - then cleanup sees a different to_remove ordering on different runs, and the world state at the end of the tick is non-reproducible. Replay breaks. Tests become flaky. Distributed simulation drifts apart.

The schedule looks like:

                  ┌── apply_eat ──┐
                  │               │
   next_event ────┼── apply_repro ┼─→ cleanup → inspect
                  │               │
                  └── apply_starve┘

next_event runs first (its writes are needed by all three appliers). The three appliers run in parallel - their writes are disjoint (or partitioned into thread-local segments, §31). cleanup runs after all three finish, never before any of them. inspect runs last.

The schedule is fixed by the DAG. Parallelism happens within the structure the DAG permits, not around it.

Two specific anti-patterns to avoid:

The “let the OS decide” anti-pattern. Spawning every system as a thread and letting them race is fast in the wrong way. Some runs will produce one result; some will produce another. The bug is intermittent, the cause is hard to find, and “fixing” it with locks reintroduces the costs §31-§33 worked to avoid.

The “early start” anti-pattern. Starting a system before its prerequisites have finished - even if the data “looks ready” - is a bet that the schedule will not change. The bet often pays off in practice, until the day a buffer fills slightly later than usual and the world’s state shifts in ways no test caught. Wait for the explicit completion of every prerequisite.

The discipline is enforced by a scheduler. A scheduler executes systems in topological order, parallelising those at the same DAG level, joining at every level boundary. Most production ECS engines (Bevy’s World::run_schedule, Unity DOTS’s JobHandle.Complete) implement exactly this. The pattern is the same as a parallel make: build dependencies in order, build independents in parallel, never start before prerequisites have finished.

A useful test: can you replay a tick to bit-identical output? If yes, your scheduler respects the contract. If no, it does not - somewhere a system runs in undefined order, and the bug will surface in the worst possible debugging window.

This rule closes Concurrency. The simulator can now use every core on the machine without sacrificing the determinism that §16 guaranteed. The DAG is both the parallel schedule and the deterministic execution order; one document, two readings.

Exercises

1. Build the schedule. Write a tick() that runs next_event, then a parallel block of the three appliers (using thread::scope plus per-thread to_remove segments), then cleanup, then inspect. Verify the boundaries: cleanup must not start before all three appliers complete.
2. Test for determinism. Run the simulator twice with the same seed. Hash the world after 100 ticks. The hashes must be identical even though the appliers ran in parallel.
3. Break the contract. Construct a schedule where cleanup starts before apply_starve finishes (e.g. via an unsafe shared buffer). Run twice. Hashes should differ - sometimes. The bug’s intermittency is the lesson.
4. Find your level boundaries. Sketch your simulator’s full DAG. Identify each level (set of systems with no transitive dependency on each other). Each level is a parallel batch; each boundary is a sync.
5. (stretch) A minimal scheduler. Write a function that takes a list of (name, read_set, write_set) and produces a Vec<Vec<&str>> - the systems grouped by level. The scheduler is just a topological sort plus level grouping. Around 50 lines of Rust.

Reference notes in 34_order_is_the_contract_solutions.md.

What’s next

You have closed Concurrency. The simulator now runs on multiple cores without losing determinism. The next phase is I/O & persistence, starting with §35 - The boundary is the queue. The simulator is about to begin talking to the world outside its tick.

Solutions: 34 - Order is the contract

Exercise 1 - Build the schedule

#![allow(unused)]
fn main() {
fn tick(world: &mut World, dt: f32) {
    next_event(&world.creatures.px, &world.creatures.py, &world.food, &mut world.pending);

    // Per-thread to_remove segments to avoid the appliers' shared write.
    let mut seg_eat:    Vec<u32> = Vec::new();
    let mut seg_repro:  Vec<CreatureRow> = Vec::new();
    let mut seg_starve: Vec<u32> = Vec::new();

    thread::scope(|s| {
        s.spawn(|| apply_eat(&world.pending, &world.food, &mut seg_eat, /* energy partition */));
        s.spawn(|| apply_reproduce(&world.pending, &world.creatures.energy, &mut seg_repro));
        s.spawn(|| apply_starve(&world.pending, &world.creatures.id, &mut seg_starve));
    });
    // All three appliers have completed before this line.

    world.to_remove.extend(seg_eat);
    world.to_remove.extend(seg_starve);
    world.to_insert.extend(seg_repro);

    cleanup(world);
    inspect(world);
}
}

The three appliers run in parallel; their write-sets are made disjoint by per-thread segments. cleanup runs after the scope returns, never before.

Exercise 2 - Test for determinism

#![allow(unused)]
fn main() {
let mut w1 = init_world(0xCAFE);
let mut w2 = init_world(0xCAFE);

for _ in 0..100 { tick(&mut w1, 0.033); }
for _ in 0..100 { tick(&mut w2, 0.033); }

assert_eq!(hash_world(&w1), hash_world(&w2));
}

If the parallel boundaries are correct, the hashes match. The merge of seg_eat, seg_starve, and seg_repro happens in the same order on both runs (the extend calls are sequential after the scope), so to_remove and to_insert end up identical between runs.

Exercise 3 - Break the contract

thread::scope(|s| {
    s.spawn(|| apply_eat(&world.pending, &world.food, &mut seg_eat, /* ... */));
    s.spawn(|| apply_reproduce(&world.pending, &world.creatures.energy, &mut seg_repro));
    s.spawn(|| apply_starve(&world.pending, &world.creatures.id, &mut seg_starve));
    s.spawn(|| cleanup(world)); // running concurrently with appliers
});

If you sidestep the borrow checker (e.g. via unsafe shared pointers), cleanup may start before the appliers have written seg_*. The two runs of the simulator produce different hashes - sometimes. Sometimes the same. The bug’s intermittency is the lesson; intermittent bugs are the worst kind to debug, and the contract exists to prevent them.

Exercise 4 - Level boundaries

For the simulator:

Six levels. Within each level, parallelism. Between levels, sync. Total throughput is bounded by the slowest level’s longest system.

Exercise 5 - A minimal scheduler

#![allow(unused)]
fn main() {
use std::collections::HashMap;

struct SystemDecl {
    name: &'static str,
    reads:  Vec<&'static str>,
    writes: Vec<&'static str>,
}

fn schedule<'a>(systems: &'a [SystemDecl]) -> Vec<Vec<&'a str>> {
    let mut levels: Vec<Vec<&str>> = Vec::new();
    let mut placed: HashMap<&str, usize> = HashMap::new();

    for s in systems {
        // The earliest level we can run is one after every system that
        // wrote a table we read.
        let mut my_level = 0;
        for sr in &s.reads {
            for prior in systems.iter().filter(|p| p.writes.iter().any(|w| w == sr)) {
                if let Some(&l) = placed.get(prior.name) {
                    my_level = my_level.max(l + 1);
                }
            }
        }
        if levels.len() <= my_level {
            levels.resize(my_level + 1, Vec::new());
        }
        levels[my_level].push(s.name);
        placed.insert(s.name, my_level);
    }
    levels
}
}

Around 30 lines. Topological sort + level grouping. Real schedulers add work-stealing, priority, GPU dispatch, dynamic re-balancing - but the core contract enforcement is exactly this. If your scheduler produces the right Vec<Vec<&str>>, you have a correct parallel ECS executor.

35 - The boundary is the queue

Concept node: see the DAG and glossary entry 35.

The simulator is a pure function. Given the world at tick start (world_t) and the inputs that arrived during the tick (inputs_t), it produces the world at tick end (world_t+1) and the outputs that should leave (outputs_t). Between those endpoints, no system touches the outside world. No system reads Instant::now(), sends a packet, writes to disk, or prints to stdout. Inside, the simulator is a transformation. Outside, it is a queue.

   ┌─────────────────────────────┐
   │      Simulator (pure)       │
   │  ┌──────────────────────┐   │
   │  │     systems run      │   │
   │  │   on world_t state   │   │
   │  └──────────────────────┘   │
   │     ↑                  ↓    │
   │ inputs_t           outputs_t│
   └─────↑──────────────────↓────┘
         │                  │
   ┌─────────┐        ┌─────────┐
   │ in queue│        │out queue│
   └─────────┘        └─────────┘
        ↑                  ↓
   environment        environment

Inputs arrive on the in-queue: events with timestamps, food-spawn requests from the policy, network packets in a multiplayer simulator, user input events. They wait in the queue until the next tick consumes them.

Outputs leave on the out-queue: state-change events for the log (eaten, born, dead), rendering data for the visualiser, packets for peers, replication updates for distributed nodes. They wait in the queue after the tick produces them, until the storage system or transport layer ships them.

What happens inside the boundary: pure transformation. Systems read from inputs_t (which is just another table by the time the systems start), update the world’s tables, queue mutations to to_remove/to_insert, and write to outputs_t (also just a table). No Instant::now(). No println!. No File::open. No TcpStream::connect. The inside is reproducible by construction; the outside is unpredictable, and the queue is the seam.

Why this matters:

Determinism. §16’s rule (same inputs + same order = same outputs) holds only if “inputs” is a complete description of the tick’s environment. The queue is that complete description. Any system reading from outside the queue is a source of non-determinism the queue cannot capture.

Replay. Record the in-queue. Replay the tick from world_t with the recorded queue. Get bit-identical world_t+1. The queue is what makes replay possible.

Testability. A test fills the in-queue with a synthetic input, runs one tick, asserts on the out-queue. The test does not need to mock File, TcpStream, or the system clock; the queue interface is the only thing the simulator sees.

Distribution. A distributed simulator with multiple nodes communicates via queues - each node’s out-queue feeds another node’s in-queue. The queue interface is the same on a single machine and across a network. The simulator’s design does not change.

Auditability. Every input that ever reached the simulator is in the in-queue’s history. Every output is in the out-queue’s history. The simulator’s full external interface is two append-only logs.

The cleanup pattern from §22 was the boundary at tick scope (mutations buffer, apply at boundary). The queue pattern at this scope is the same idea at run scope (I/O buffers, apply at the seam). The two compose: cleanup makes the tick atomic; the queue makes the run reproducible.

A useful test: can you run two simulators side-by-side from the same in-queue and get identical out-queues? If yes, the boundary holds. If no, somewhere a system reads the environment directly.

Exercises

1. Build the queues. Add in_queue: Vec<InputEvent> and out_queue: Vec<OutputEvent> to your simulator. Both fill at tick boundaries.
2. Refactor a system that reads time. Find any system that uses Instant::now() directly. Refactor: take current_time as a parameter. The caller (the tick driver) reads Instant::now() once and passes it down. The system itself is now deterministic.
3. Refactor a system that prints. Find any system that calls println!. Refactor: push the message to out_queue instead. The caller reads the queue after the tick and writes whatever’s there. Logging is now deterministic; tests can assert on the queue.
4. Replay test. Save the in-queue across a 100-tick run. Run the simulator a second time from the initial world state with the saved queue. Hash both worlds. They must match.
5. Two simulators from one queue. Run two simulators in parallel (or sequentially), feeding both from the same in-queue. After 100 ticks, hash both worlds. They must match. If they do not, somewhere a system reads from outside the queue.
6. (stretch) Audit a real simulator. Open any open-source simulator’s tick function. Find every place it reads from the environment (clock, file, network, env vars). Each is a place where determinism leaks; each could be queue-ified.

Reference notes in 35_boundary_is_the_queue_solutions.md.

What’s next

§36 - Persistence is table serialization takes the next step: when the simulator pauses and resumes, persistence is just writing the columns and reading them back. No translation, no impedance mismatch.

Solutions: 35 - The boundary is the queue

Exercise 1 - Build the queues

#![allow(unused)]
fn main() {
struct InputEvent  { tick: u64, kind: u8, payload: u64 }
struct OutputEvent { tick: u64, kind: u8, payload: u64 }

struct World {
    // ... tables ...
    in_queue:  Vec<InputEvent>,
    out_queue: Vec<OutputEvent>,
}

fn tick(world: &mut World, current_time: f64) {
    let inputs: Vec<_> = world.in_queue.drain(..).collect();
    // pure transformation - no other I/O
    next_event(/* ... */);
    motion(/* ... */, current_time);
    // ... etc; outputs accumulated into world.out_queue
}
}

The two queues are the boundary. Inputs arrive via in_queue.push(). Outputs leave via out_queue.drain(). Inside the tick, only the inputs and outputs cross the seam.

Exercise 2 - Refactor Instant::now()

Before: every system that needs time calls Instant::now() directly. Multiple systems → multiple non-deterministic readings.

After: the tick driver reads Instant::now() once, computes current_time: f64, passes it to every system as a parameter. The systems are pure functions of their inputs.

#![allow(unused)]
fn main() {
let now = Instant::now();
let current_time = (now - sim_start).as_secs_f64();
tick(&mut world, current_time);
}

Replay can substitute a recorded current_time instead of reading the wall clock. The simulator’s behaviour is identical.

Exercise 3 - Refactor println!

Before:

fn apply_starve(...) {
    if energy[i] <= 0.0 {
        println!("creature {} starved", id[i]);
    }
}

After:

#![allow(unused)]
fn main() {
fn apply_starve(..., out: &mut Vec<OutputEvent>) {
    if energy[i] <= 0.0 {
        out.push(OutputEvent { tick, kind: STARVED, payload: id[i] as u64 });
    }
}
}

The system pushes to out_queue instead of writing stdout. The tick driver reads the queue after the tick and prints whatever is there. Logging is now deterministic; tests assert on the queue.

Exercise 4 - Replay test

#![allow(unused)]
fn main() {
let saved_inputs: Vec<Vec<InputEvent>> = run_and_record(&mut world1, 100);
let mut world2 = init_world(seed);
for inputs in saved_inputs {
    world2.in_queue.extend(inputs);
    tick(&mut world2, /* recorded current_time */);
}
assert_eq!(hash_world(&world1), hash_world(&world2));
}

If the boundary is respected, world2 after replay matches world1 after the live run. If they differ, somewhere a system reads outside the queue.

Exercise 5 - Two simulators from one queue

Same structure as exercise 4, but feed two simulators (in parallel or sequentially) from the same recorded inputs. Hash both worlds at tick 100. They must match. If they don’t, the difference traces back to one (or more) system reading outside the queue.

Exercise 6 - Audit a real simulator

Common findings in production code:

• tokio::time::Instant::now() inside a request handler - pulls wall time into the per-request transform.
• tracing::info! with side-effecting log macros - couples the system to the tracing infrastructure.
• tokio::fs::File::open reads - couples the system to the filesystem.
• env::var calls - couples the system to the OS environment.
• rand::thread_rng() - pulls non-deterministic randomness into the per-tick transform.

Each is a place where determinism leaks. Each could be queue-ified.

36 - Persistence is table serialization

Concept node: see the DAG and glossary entry 36.

The simulator pauses. The world is in memory: the creatures columns (px, py, vx, vy, energy, birth_t, id, generation), a food table, presence tables (hungry, dead, etc.), the index map (id_to_slot), and the cleanup buffers. To pause durably, all of this must be written to disk; to resume, all of this must be read back.

The instinct the OOP world brings: design a “persistence format” with a schema, marshalling logic, version handling, and a translation layer between in-memory objects and on-disk records. This is wrong on the data-oriented side. There is no translation. There is only transposition.

A snapshot is the columns, written sequentially. A recovery is the columns, read sequentially. The on-disk format is the same shape as memory.

#![allow(unused)]
fn main() {
fn snapshot(world: &World, path: &Path) -> std::io::Result<()> {
    use std::io::Write;
    let mut f = std::fs::File::create(path)?;

    // Header: tick, schema version.
    f.write_all(&world.tick.to_le_bytes())?;
    f.write_all(&SCHEMA_VERSION.to_le_bytes())?;

    // Each column: [length: u32][raw bytes...]
    write_column(&mut f, &world.creatures.px)?;
    write_column(&mut f, &world.creatures.py)?;
    write_column(&mut f, &world.creatures.vx)?;
    write_column(&mut f, &world.creatures.vy)?;
    write_column(&mut f, &world.creatures.energy)?;
    write_column(&mut f, &world.creatures.birth_t)?;
    write_column(&mut f, &world.creatures.id)?;
    write_column(&mut f, &world.creatures.generation)?;

    // Presence tables: same shape, append.
    write_column(&mut f, &world.hungry)?;
    // ... etc.

    Ok(())
}
}

Recovery is the inverse: read the bytes back into Vecs. No type conversion, no field mapping, no schema discrimination at the row level. The file is exactly what the memory was; the memory is exactly what the file is.

The savings are concrete:

No schema design. The schema is whatever the columns are. Schema documentation is the column declarations.

No object marshalling. No serialize() per row, no deserialize() per row. The Vec is written as bytes; bytes are read as a Vec. At 1M creatures × 24 bytes hot, the snapshot is 24 MB; writing it is one bulk write - ~5-10 ms on NVMe.

No translation bugs. ORMs are a famous source of subtle correctness issues - fields renamed, types coerced, edge cases mishandled. Here, the in-memory and on-disk forms are bit-identical; the load is read_bytes_into_vec and that is all.

Deterministic recovery. A snapshot taken in a deterministic simulator round-trips exactly. The hashed world after snapshot → load is identical to the hashed world before.

What it does not save you from:

Platform versioning. Three things can break a snapshot across environments: the schema changed (you added a column or renamed a type), the byte order differs (you saved on a little-endian machine and loaded on a big-endian one), or the OS conventions differ (line endings, native type widths, path conventions in any string fields). All three have the same fix. Write a small header into every snapshot - schema version, endianness, OS - and at load time, if any field differs from the current platform’s value, run the matching migration. The migrations are one-directional (newer code reads older snapshots, x86 code reads big-endian snapshots, Linux code reads macOS snapshots) and they are the only translations in the system. Most simulators target a single architecture and OS, write the header fields anyway, and skip the migrations until they are needed; the mechanism is there from day one, the cost is the bytes for the header.

Compression. Raw bytes are rarely compact at runtime - many fields are sparse or small, and compression is sometimes worth a few milliseconds at snapshot time to save tens of milliseconds on the disk side. Apply only after measurement.

The pattern shows up everywhere this scale matters. Write-ahead logs in databases, save-game files in games, checkpoint files in HPC, frame snapshots in video editing. They all dodge the ORM trap by writing the columns directly.

The §0/§1 simulator’s snapshot is roughly twenty-five lines of Rust per direction. The OOP equivalent - define a CreatureRecord, derive Serialize/Deserialize, walk the world serialising one creature at a time - is ten times the code, slower at runtime, and prone to the translation bugs the column-direct version cannot have.

Measurements

Serialisation’s slow reputation is the text path; a binary per-row encoder is barely slower than the column snapshot, and its cost is mostly the per-row function call. Either way the column-direct snapshot wins and carries no per-row translation bugs. Full output: code/README.md.

Exercises

1. Snapshot the world. Implement a snapshot function for your simulator. Save to snapshot.bin. Note the file size: it should match bytes per column × N for hot tables, plus headers.
2. Load the snapshot. Implement the inverse. Load snapshot.bin into a fresh World. Verify by running the simulator from the loaded state and comparing the hash to the original at the same tick.
3. The OOP comparison. Define a CreatureRecord struct and write a per-row serialiser via serde_json or bincode. Time it against the column snapshot at 1M creatures. The cost depends entirely on the format: measured (row_vs_column_serialize), a per-row text encoder (the serde_json shape) is ~30-65× slower than the column snapshot across the four reference machines¹; a per-row binary encoder (the bincode shape) is only ~1-2× slower². The text path is where serialisation’s slow reputation comes from; the binary path’s cost is mostly the per-row function call. Either way the column snapshot wins, and it carries none of the per-row translation bugs.
4. Schema versioning. Add a new column (hunger_buildup: f32) to the simulator. Make the snapshot reader handle both old and new versions: old snapshots get the new column zero-filled; new snapshots get loaded directly. Verify both round-trip cleanly.
5. (stretch) Memory-mapped snapshot. Use memmap2 to map the snapshot file directly into memory. The Vec’s pointer is the file’s memory; loading is zero-copy. Compare load times for a 24 MB snapshot.

Reference notes in 36_persistence_is_serialization_solutions.md.

What’s next

§37 - The log is the world makes the structural argument explicit: the log of events and the world’s tables share a shape; one is a projection of the other.

Solutions: 36 - Persistence is table serialization

Exercise 1 - Snapshot

#![allow(unused)]
fn main() {
const SCHEMA_VERSION: u16 = 1;

fn snapshot(world: &World, path: &Path) -> std::io::Result<()> {
    use std::io::Write;
    let mut f = std::fs::File::create(path)?;
    f.write_all(&world.tick.to_le_bytes())?;
    f.write_all(&SCHEMA_VERSION.to_le_bytes())?;

    let n = world.creatures.px.len() as u32;
    f.write_all(&n.to_le_bytes())?;
    write_slice(&mut f, &world.creatures.px)?;
    write_slice(&mut f, &world.creatures.py)?;
    write_slice(&mut f, &world.creatures.vx)?;
    write_slice(&mut f, &world.creatures.vy)?;
    write_slice(&mut f, &world.creatures.energy)?;
    write_slice(&mut f, &world.creatures.id)?;
    Ok(())
}

fn write_slice<T>(f: &mut std::fs::File, v: &[T]) -> std::io::Result<()> {
    use std::io::Write;
    let bytes: &[u8] = unsafe {
        std::slice::from_raw_parts(v.as_ptr() as *const u8, std::mem::size_of_val(v))
    };
    f.write_all(bytes)
}
}

The unsafe is for direct byte access. In production, use bytemuck::Pod to get the same effect safely. For 1 M creatures, the snapshot is roughly 24 MB; writing it is one bulk syscall, ~5-10 ms on NVMe.

Exercise 2 - Load

#![allow(unused)]
fn main() {
fn load(path: &Path) -> std::io::Result<World> {
    use std::io::Read;
    let mut f = std::fs::File::open(path)?;
    let mut tick_bytes = [0u8; 8];
    f.read_exact(&mut tick_bytes)?;
    let tick = u64::from_le_bytes(tick_bytes);

    let mut sv = [0u8; 2];
    f.read_exact(&mut sv)?;
    let _schema_version = u16::from_le_bytes(sv);

    let mut n_bytes = [0u8; 4];
    f.read_exact(&mut n_bytes)?;
    let n = u32::from_le_bytes(n_bytes) as usize;

    let mut world = World::new(tick);
    world.creatures.px = read_vec::<f32>(&mut f, n)?;
    world.creatures.py = read_vec::<f32>(&mut f, n)?;
    // ... vx, vy, energy, id ...
    Ok(world)
}
}

After load, hash_world(&loaded) matches hash_world(&original) byte for byte. Determinism + transposition = round-trip safety.

Exercise 3 - OOP comparison

For 1 M creatures:

• Column snapshot via raw bytes: ~5-10 ms.
• Per-row serde_json::to_writer: ~500-1000 ms (text encoding, per-row overhead).
• Per-row bincode::serialize_into: ~50-100 ms (binary, per-row overhead).

The column-direct version is bound by sequential disk bandwidth. The per-row versions add CPU encoding cost on top.

Exercise 4 - Schema versioning

#![allow(unused)]
fn main() {
let schema_version = u16::from_le_bytes(sv);
if schema_version >= 2 {
    world.creatures.hunger_buildup = read_vec::<f32>(&mut f, n)?;
} else {
    world.creatures.hunger_buildup = vec![0.0; n]; // default for older snapshots
}
}

Old snapshots (v1) lack the hunger_buildup column; the loader supplies zeros. New snapshots (v2) include it. Both round-trip cleanly. Version migration lives at load time, in one place; the rest of the simulator does not know about it.

Exercise 5 - Memory-mapped snapshot

#![allow(unused)]
fn main() {
use memmap2::MmapOptions;

let f = std::fs::File::open(path)?;
let mmap = unsafe { MmapOptions::new().map(&f)? };
let bytes: &[u8] = &mmap;
// Parse columns directly from `bytes` - no copy.
}

For a 24 MB snapshot:

• Read-into-Vec: ~5 ms (one syscall + memcpy from kernel buffer to user heap).
• mmap: ~10 µs initial setup; access is page-faulted lazily.

If the simulator accesses the loaded data sequentially after load, mmap wins. If access is random, mmap pays page-fault costs each time. For load-once-then-stream patterns, mmap is the cleaner option.

37 - The log is the world

Concept node: see the DAG and glossary entry 37.

§36 said persistence is transposition: the in-memory tables are written as their bytes, read back as their bytes. This section makes the deeper structural claim. The log is the world, and the world is the log decoded.

In an event-sourced simulator, every state change is an event:

(tick=42, kind=become_hungry, creature_id=17)
(tick=42, kind=eat,           creature_id=23, food_id=8, energy_delta=+5.0)
(tick=43, kind=reproduce,     parent_id=14, offspring_id=400, offspring_energy=2.5)
(tick=43, kind=die,           creature_id=89)

The log is a sequence of such events. The world’s tables can be reconstructed from the log: start from an empty world (or a snapshot), replay events in order, and the resulting tables are bit-identical to the world the live simulator produced.

The structural fact: the log and the world have the same shape.

In memory a presence table like hungry is a list of slots (§17); in the log it is a stream of become_hungry and stop_being_hungry events keyed by the stable creature id - the boundary rule from §26, since a slot is meaningless once the world is reloaded into a different layout. Replaying that stream of (tick, creature_id) pairs reconstructs the membership.

A column energy: Vec<f32> is the result of starting from an empty Vec plus the events that wrote each entry. The log holds these writes; the column is the cumulative effect of replaying them.

In the most explicit form - the triple-store shape - the log is a sequence of (rid, key, val) triples:

• rid = which entity: the stable id, not the slot
• key = which cell: a code for table.column (e.g. creatures.energy)
• val = the value written there

Read one triple as a sentence: entity rid, cell table.column, becomes val. The key is best read as table.column - it names the table and the column, so (rid, table.column) is a fully-qualified address of one cell anywhere in the world. That table.column form is what makes the log uniform: every state change, in every table, is the same three fields, and replay is the mechanical world.table.column[id_to_slot[rid]] = val applied over the log in order. The codebook stores each distinct table.column string once and the per-event key as a small integer code, so the log never carries the string. (This is a write-ahead log: table.column, row-by-id, value.)

Three stable handles, one moving thing left out. The entity id is identity - it survives relocation and the save (§26). The table.column is the schema address - stable as long as the schema is. The value is the write. The slot - the entity’s momentary position in the columns - is never logged, because it is the one part that moves; replay re-derives it through id_to_slot (§23). The triples form the log; transposed, they form the columns. Transposition is the only translation. There is no impedance mismatch because there is no model gap.

A working specimen: code/logger

The crate code/logger implements this triple-store shape directly, dependency-free. Its design is worth walking through, because it meets three problems that recur whenever a simulator wants to log everything.

The IOPS problem → batching. A naive event logger calls write once per event. At a million events per minute, that is millions of disk operations per minute - bound by IOPS, not bandwidth (§38). The disk’s bandwidth sits mostly idle while it queues operations. The fix: collect events into an in-memory buffer; when the buffer fills, flush it as one large write. IOPS scales with “buffer flushes per second”; bandwidth absorbs the actual byte volume. Logging cost drops from disk-latency-bound to bandwidth-bound - typically 100-1000× faster.

The redundancy problem → codebook and type inference. Most fields in a simulator’s event records repeat: the same kind code thousands of times, the same set of activity strings, the same handful of entity types. Storing each event’s full payload wastes bytes. The fix: a codebook assigns each unique string a small integer code; the log stores the code, not the string. On read, the codebook reverses the mapping. The crate goes one step further with type inference - every value is stored as one f64 (8 bytes), whether it began as an integer, a float, or a string code. Integers up to 2⁵³ round-trip exactly; the union format eliminates per-field type tags. With only the populated fields stored per record, a sparse log uses far less memory than dense column arrays.

The write-blocking problem → the revolver. If the foreground blocks while the disk flushes, the simulator pauses on every flush. The fix: two buffers cycle between the foreground and a background writer thread. When one fills, the foreground hands it to the writer over a channel and takes a recycled empty one back; the writer flushes it and recycles it. When the foreground outruns the writer, that empty-buffer channel becomes the backpressure. From the simulator’s side, writing an event is a few pushes to a Vec, never a wait on disk - std::sync::mpsc does both the hand-off and the flow control.

The combined result: log() costs ~160 ns at 5 fields and ~310 ns at 11 on this machine, sub-microsecond and dominated by the codebook lookups and the Vec pushes (cargo run --release --bin benchmark times both widths; the number is on your machine, not on trust). The hot-path output is a sequence of raw little-endian column-byte chunks written sequentially by the background thread - the bytes on disk are the bytes in memory (§36), no .npz, no serde. Read-back rebuilds dense columns and presence masks (to_arrays), iterates decoded rows, and exports CSV; a SQLite export is left to a downstream converter, since SQLite is not in Rust’s std and a crate would forfeit the crate’s dependency-free property. The structural identity - log = world - holds across all these formats; what changes is the storage system at the boundary (§38).

The design is justified structurally, not by a microbenchmark: the sparse triple-store stores only what a record populates, the codebook deduplicates the strings, and the single f64 value stream erases per-field type tags. None of the three is exotic; together they are the compact shape a sparse simulation log wants. Three views of the same idea are sketched in the stretch exercise below.

The library does not need to know what an “event” is. It stores triples; the consumer interprets them. That separation is what makes the same code serve as a simulation logger, an audit trail, and a replay source - three uses, one structural pattern.

Why this matters in practice:

Replay is structural. Snapshot + log = pause/resume. To recover the world at any tick T, load the most recent snapshot at tick S ≤ T, then replay the log from S to T. The cost is bounded by T - S events, which is small if snapshots are taken regularly.

Auditability is free. Every change in the world is in the log. To answer “why is creature 17 dead?”, scan the log for events involving 17. The log is the system’s complete history, in order.

Testing is replay. A test fixture is an initial world plus a log. A test is “replay this log; assert this property of the result”. No mocks, no setup methods, no fixture builders.

Distribution is structural. Two nodes running identical code from the same log produce bit-identical worlds. Send the log; the worlds converge.

The log is the system of record. Snapshots are caches of the log’s state; they exist for performance, not correctness. If snapshots are lost, the log can rebuild them. If the log is lost, no snapshot can recover events that have not been logged.

The discipline that makes this work is structural, not stylistic. Every state change in the simulator is logged before being applied. The cleanup pass (§22) is the natural place - it sees every mutation and can record each one as it commits. The §38 storage system is the natural sink - log writes are sequential, batched, and amortised across the tick.

A simulator that respects this discipline is one whose history is the log, whose state is a projection of the log, and whose persistence is the log plus the most recent snapshot. Every other property the book has built - determinism, parallelism, EBP dispatch, snapshot serialisation - composes with this one.

Exercises

1. Log the simulator. Add an events: Vec<Event> table to your world. Modify the cleanup pass to push one event per applied mutation. After 100 ticks, the log has roughly active × ticks events.

2. Reconstruct from the log. Write a replay(initial: World, events: &[Event]) -> World that applies each event in order. Verify: starting from an initial world and applying the log produces a world identical to the live simulator’s output at the same tick.

3. Save and load the log. Persist the log via §36’s column serialisation. Reload. Replay. Confirm bit-identical state.

4. Snapshot + log. Save a snapshot at tick S; save the log from tick S onward. Reconstruct any tick T > S by loading the snapshot and replaying the log from S to T. Verify against the live simulator.

5. The triple-store form. Convert your events table to three parallel arrays: rids: Vec<u32>, keys: Vec<u8>, vals: Vec<f64>. Compare the storage size to the per-event-struct version. The triple-store form is typically 2-3× more compact for events with sparse fields.

6. (stretch) A logger, three ways. code/logger is the crate form, built and benchmarked. Read it, then sketch the other two shapes and compare what each gains and loses:

   • As a crate (the built one). Read code/logger/src/lib.rs: a log() over (name, Value) records, an evolving codebook, a double-buffered writer thread, reusable across simulators behind a stable public API. Note what that public boundary costs versus the two forms below.
   • As a module inside your simulator. Same shape, but accessing the simulator’s existing types (Event, World) directly without crossing a crate boundary. Less reusable, more efficient - no public API to keep stable.
   • As an ECS system. A logging system whose read-set is to_remove, to_insert, and any other commit-time tables, and whose write-set is the log buffer. It runs in the same DAG as cleanup, perhaps merged with it. The two halves of cleanup - committing mutations and logging them - become one system.

   Sketch the module and system forms; the crate form is already in code/logger to compare against. Weigh what each gains and loses: reusability, performance, ease of testing, distance from the simulator’s other concerns.

Reference notes in 37_log_is_world_solutions.md.

What’s next

§38 - Storage systems: bandwidth and IOPS names the cost of crossing the I/O boundary in concrete terms. The log lives there; so does the snapshot; so does every external connection.

Solutions: 37 - The log is the world

Exercise 1 - Log the simulator

#![allow(unused)]
fn main() {
struct Event {
    tick: u64,
    kind: u8,           // BORN, DIE, EAT, BECAME_HUNGRY, ...
    creature_id: u32,
    payload_a: u64,     // food_id, parent_id, ...
    payload_b: f32,     // energy_delta, ...
}

fn cleanup(world: &mut World, events: &mut Vec<Event>) {
    for &id in &world.to_remove {
        events.push(Event {
            tick: world.tick, kind: DIE,
            creature_id: id, payload_a: 0, payload_b: 0.0,
        });
        // ... apply removal ...
    }
    for row in &world.to_insert {
        events.push(Event {
            tick: world.tick, kind: BORN,
            creature_id: row.id, payload_a: row.parent_id as u64,
            payload_b: row.energy,
        });
        // ... apply insertion ...
    }
}
}

For a 100-creature simulator with steady birth and death, 100 ticks → roughly 200-1000 events depending on activity rate.

Exercise 2 - Reconstruct from the log

#![allow(unused)]
fn main() {
fn replay(initial: &World, events: &[Event]) -> World {
    let mut w = initial.clone();
    for e in events {
        match e.kind {
            BORN => insert_creature(&mut w, e.creature_id, e.payload_b),
            DIE  => remove_creature(&mut w, e.creature_id),
            EAT  => apply_eat(&mut w, e.creature_id, e.payload_a as u32, e.payload_b),
            BECAME_HUNGRY => mark_hungry(&mut w, e.creature_id),
            // ...
        }
    }
    w
}
}

Run the simulator live for 100 ticks → world A. Run replay(initial, &events) → world B. hash_world(&A) == hash_world(&B). The world is the log decoded.

Exercise 3 - Save and load the log

The log is a Vec<Event> - same shape as any column-based table. Use §36’s column serialisation pattern. Save the log; load it; replay it onto the initial world; compare hashes.

Exercise 4 - Snapshot + log

#![allow(unused)]
fn main() {
// At tick 0: snapshot.
snapshot(&world, "tick_0.snap")?;

// Run for 100 ticks, logging events.
let log = run_with_logging(&mut world, 100);

// Reconstruct any later tick T:
let snap_world = load("tick_0.snap")?;
let world_at_T = replay(&snap_world, &log[0..events_through_tick(T)]);
}

Snapshots are taken at convenient points; the log is appended continuously. Reconstruction at any T uses the most recent snapshot at S ≤ T plus the log slice from S to T.

Exercise 5 - Triple-store form

#![allow(unused)]
fn main() {
struct Triple { rid: u32, key: u8, val: f64 }

let triples: Vec<Triple> = events.iter().flat_map(|e| match e.kind {
    DIE  => vec![Triple { rid: e.creature_id, key: KEY_DEAD, val: 1.0 }],
    BORN => vec![
        Triple { rid: e.creature_id, key: KEY_PARENT, val: e.payload_a as f64 },
        Triple { rid: e.creature_id, key: KEY_ENERGY, val: e.payload_b as f64 },
    ],
    // ...
}).collect();
}

For events with sparse fields (a DIE event uses only creature_id; an EAT event uses three fields), the triple-store form is 2-3× more compact because empty fields don’t take space.

Exercise 6 - A working specimen

code/logger implements the triple-store shape directly:

• rids: Vec<u32> - which entity: the stable id, not the slot
• keys: Vec<u16> - which cell: a code for table.column (which table and column the value writes to)
• vals: Vec<f64> - the value, as 8 bytes (integers up to 2⁵³ round-trip exactly; strings are codebook-encoded to integers, then stored as the integer)

On read, these are densified into per-field Vecs plus presence masks. The same shape that was on disk is now in memory, ready for systems to iterate. The library does not need to know what an “event” is; it stores triples and lets the consumer interpret them. The §17/§37 structural pattern in working code.

38 - Storage systems: bandwidth and IOPS

Concept node: see the DAG and glossary entry 38.

A storage system is the part of the program that crosses the boundary into something that holds bytes for longer than RAM does. Disk, network, distributed file system, message queue, message broker - all are storage systems. They differ in technology; they share a cost model.

The cost has two dimensions.

Bandwidth - bytes per second. How fast bytes can move through the storage system. NVMe SSD is roughly 3-7 GB/s read, 2-5 GB/s write. SATA SSD: ~500 MB/s. Spinning HDD: 100-200 MB/s sequential. Gigabit network: 100 MB/s. 10 Gbit network: 1 GB/s. SQLite on local NVMe: 200-500 MB/s for bulk inserts.

IOPS - operations per second. How many separate read/write operations the storage system can complete per second. NVMe: 100K-1M random IOPS; sequential IOPS counts are much higher (the underlying flash can stream). SATA SSD: 50-100K IOPS. HDD: 100-200 IOPS (limited by seek time). Network connection: bounded by latency × concurrency.

A workload’s cost is bounded by both. A 1 MB sequential read on NVMe is one IOP and ~250 µs of bandwidth time. A million 1-byte random reads is a million IOPs and ~10 seconds of latency time. Same total bytes, three orders of magnitude apart.

The §22 batched-cleanup pattern at §30’s streaming scale gathers many small mutations into one large write. This converts a high-IOPS, low-bandwidth workload (1000 separate writes per tick) into a low-IOPS, bandwidth-friendly one (one batched write per tick)¹. The pattern is the natural fit for storage systems where IOPS is the binding constraint.

Three concrete examples worth keeping in mind:

SQLite. On local NVMe, SQLite handles ~50 K row inserts per second using one-by-one INSERT statements; ~500 K-1 M per second using prepared statements with batched transactions; ~5 M per second using INSERT INTO ... SELECT FROM ... over an in-memory table. What separates the tiers is the transaction boundary, not the driver (rusqlite, sqlx, and the C API all reach every tier): a bare INSERT is its own implicit transaction and its own fsync, a BEGIN/COMMIT around a batch amortises that fsync across every row in it, and a staged INSERT ... SELECT pays it once for the whole load. It is the same batch-the-commit move as the cleanup queue (§22) and the log (§37). The same database, three orders of magnitude in throughput, depending on whether the workload pushes IOPS or bandwidth.

Network sockets. A round-trip to a server is bounded by latency: ~0.1 ms LAN, ~10-100 ms internet, ~1 ms data centre. Each round-trip is one IOP from the workload’s perspective. Bandwidth is not the binding constraint until the response is many KB. The §22 pattern at this scale: batch many requests into one round-trip.

Distributed file systems. S3, EFS, CephFS - bandwidth scales with concurrency (many parallel reads from many objects = high aggregate bandwidth) but per-object IOPS is low (one operation per request). Workloads that want sequential bandwidth fan out across many objects; workloads that want low latency on small reads do not fit this storage system.

The lesson: when adding a storage system to the simulator, measure both bandwidth and IOPS of your workload - not just the system’s spec sheet. A 7 GB/s NVMe drive limited to 100 K IOPS is bottlenecked at ~30 KB per IOP for random workloads. Below that block size, IOPS bind.

The §4 budget framing applies here too. A 30 Hz tick has 33 ms of budget. A 100 µs disk read costs 0.3 % of the budget. Ten of them cost 3 %. A hundred cost 30 % - already a third of the tick. Bound the I/O per tick, batch where possible, and treat every cross-boundary operation as a real cost in the same ledger as cache misses and arithmetic.

The simulator inside the boundary is a pure function. The storage system at the boundary is the function’s connection to durable reality. The cost of that connection is the bandwidth × IOPS budget; the discipline is the batching pattern; the architecture is the queue.

Measurements

Batching many small writes into one trades a high-IOPS workload for a bandwidth-friendly one; how much it buys depends on the per-write overhead of the path (the 2012 laptop’s slow per-write syscall path is the outlier). Full output: code/README.md.

Exercises

1. Measure your bandwidth. On Linux: dd if=/dev/zero of=/tmp/test bs=1M count=1024 oflag=direct measures sequential write. Note your number.
2. Measure your IOPS. Time 10 000 separate File::write calls of 4 KB each, with sync_all() after the loop. Compute IOPS as 10_000 / time_in_seconds. Compare to the spec sheet.
3. Batched vs unbatched. Write 1 000 000 rows of 32 bytes each to a file: first as 1 000 000 separate writes; then as one bulk write. Compare times. The batched version is much faster - measured 14-256× across the four reference machines (batched_write), the spread driven by filesystem and write buffering. With an fsync per write (a durable log) the gap widens by orders of magnitude; the exercise here measures buffered writes, which is the floor.
4. SQLite throughput. Insert 1 000 000 rows into a SQLite table: first as separate INSERT statements; then in a single transaction; then via one INSERT INTO ... VALUES (...) with all rows. Note the three orders of magnitude.
5. Compute your tick budget. At 30 Hz with 1 000 mutations per tick, what is the largest acceptable per-mutation I/O cost? Below NVMe latency, you are fine; above it, you must batch.
6. (stretch) A second storage system. If you have a network filesystem handy (NFS, SSHFS), repeat exercise 3 against a remote file. Note the latency-vs-bandwidth tradeoff. The IOPS limit is your bandwidth-delay product divided by IO size.

Reference notes in 38_storage_systems_solutions.md.

What’s next

You have closed I/O & persistence. The simulator can now talk to durable storage and external systems without sacrificing determinism or layout discipline. The next phase is System of systems, starting with §39 - System of systems: the patterns for work that does not fit the standard tick model - long-running optimisation, time-sliced search, out-of-loop computation. After that, Discipline (§40-§43) closes the book with the design rules that keep the simulator working over time.

Solutions: 38 - Storage systems

Exercise 1 - Bandwidth

$ dd if=/dev/zero of=/tmp/test bs=1M count=1024 oflag=direct
1073741824 bytes (1.1 GB) copied, 0.395 s, 2.7 GB/s

Typical numbers:

• NVMe SSD (PCIe 4.0): 3-7 GB/s sequential write
• SATA SSD: ~500 MB/s
• Spinning HDD: 100-200 MB/s
• USB 3 external: depends on the device, often 200-500 MB/s

oflag=direct bypasses the OS page cache, giving you the device’s actual bandwidth, not what the page cache absorbs.

Exercise 2 - IOPS

#![allow(unused)]
fn main() {
use std::io::Write;
use std::time::Instant;

let mut f = std::fs::File::create("/tmp/iops_test")?;
let buf = [0u8; 4096];
let n = 10_000;

let start = Instant::now();
for _ in 0..n {
    f.write_all(&buf)?;
}
f.sync_all()?; // important - without this, writes sit in the OS buffer
let elapsed = start.elapsed();
println!("IOPS: {:.0}", n as f64 / elapsed.as_secs_f64());
}

Typical numbers:

• NVMe: 50-200 K IOPS for 4 KB writes (the device may report higher random IOPS in benchmarks; sequential same-block writes hit different caches)
• SATA SSD: 50-100 K IOPS
• HDD: 100-200 IOPS

Without sync_all, the kernel buffers writes; the apparent IOPS is much higher than the device’s actual rate. The actual disk-side IOPS is what sync_all exposes.

Exercise 3 - Batched vs unbatched

#![allow(unused)]
fn main() {
// Unbatched: 1M writes
let mut f = std::fs::File::create("/tmp/unbatched")?;
for _ in 0..1_000_000 {
    f.write_all(&[0u8; 32])?;
}
f.sync_all()?;

// Batched: 1 write
let mut f = std::fs::File::create("/tmp/batched")?;
let big_buf = vec![0u8; 32 * 1_000_000];
f.write_all(&big_buf)?;
f.sync_all()?;
}

Typical results on NVMe:

• Unbatched: 5-30 seconds (1 M writes × IOPS limit)
• Batched: 50-100 ms (one ~30 MB write at sequential bandwidth)

100-500× faster. The exact ratio depends on the OS page cache’s absorption behaviour; with sync_all to expose the actual disk-side cost, the gap is at the upper end.

Exercise 4 - SQLite throughput

#![allow(unused)]
fn main() {
// Per-row INSERT (no transaction): ~50K rows/sec
for row in &rows {
    conn.execute("INSERT INTO t VALUES (?, ?, ?)", params![/* ... */])?;
}

// Single transaction: ~500K-1M rows/sec
conn.execute("BEGIN", [])?;
for row in &rows {
    conn.execute("INSERT INTO t VALUES (?, ?, ?)", params![/* ... */])?;
}
conn.execute("COMMIT", [])?;

// Bulk INSERT VALUES: ~5M rows/sec
let mut sql = String::from("INSERT INTO t VALUES ");
for row in &rows {
    sql.push_str(&format!("({}, {}, {}),", row.0, row.1, row.2));
}
sql.pop(); // trailing comma
conn.execute(&sql, [])?;
}

The IOPS dimension binds the per-row version (each INSERT is one disk operation when not in a transaction). The transaction version reduces per-row to one shared commit. The bulk-VALUES version reduces 1M operations to one - bandwidth-bound, not IOPS-bound.

Exercise 5 - Tick budget

At 30 Hz: 33 ms / tick = 33 000 µs.

For 1000 mutations per tick:

• Per-mutation budget = 33 µs.
• NVMe latency = 100 µs (about 3× over budget per mutation).
• Batched: 1000 × 32 B = 32 KB, one ~5 µs write at NVMe sequential bandwidth (well under budget).

Unbatched mutations cannot fit a 30 Hz budget; batched ones easily can.

Exercise 6 - A second storage system

For SSHFS at LAN latency (~0.5 ms RTT):

• Per-statement INSERT: 2 RTT minimum ≈ 1 ms = ~1000 IOPS max
• Single-transaction with 1 M rows: 2 RTT for the transaction (commit) + bandwidth for the data ≈ 100 ms total
• The IOPS limit is the bandwidth-delay product divided by IO size: at 1 Gbit/s × 0.5 ms = 64 KB in flight, so ~16 K IOPS max for 4 KB I/Os, ~1 K for 64 KB.

The pattern is the same: batching converts a high-latency, low-bandwidth workload into a sequential one bounded by bandwidth. On a network filesystem the latency penalty is much larger; the batching imperative is correspondingly stronger.

39 - System of systems

Concept node: see the DAG and glossary entry 39.

The trunk so far has assumed every system runs every tick and completes within the tick budget. That covers most of what the simulator does - motion, EBP dispatch, cleanup, persistence - and the surrounding chapters earned the assumption. But the assumption is not universal. Practical simulators have at least three classes of work that do not fit it.

• Optimisation. A scheduler choosing which tasks each warehouse robot should take next. A combat AI choosing a counter-strategy. A constraint solver finding a feasible plan. These can take seconds or minutes; they cannot fit in a 33 ms tick.
• Search. The nearest-task scan for a warehouse operator. A path-finder over a large map. A neighbour query in a million-creature world. Even with §28’s spatial binning, some searches genuinely take longer than one tick can afford.
• Out-of-process work. A game AI evolving its strategy on a separate thread. A pricing model running on a remote server. A precomputation handed off to a worker pool. The simulator never blocks waiting; results arrive when they arrive.

This chapter names the three patterns that cover these cases without breaking any of the trunk’s previous rules. They are not new architecture. They are the trunk’s existing rules, applied to a wider set of cadences.

The unifying principle: a system has a cadence, and the cadence does not have to be one tick. A system can run every tick (motion). It can run every N ticks (the spatial compaction from §28/§24, which reorders the columns by cell every few dozen ticks). It can have a deadline and return its best current answer when the deadline arrives. It can be suspended and resumed across ticks, with its progress part of its state. It can be out-of-loop entirely, communicating with the simulator only through the queue. The DAG generalises naturally: edges still represent dependencies, but some dependencies wait for promises rather than synchronous returns.

Anytime algorithms

An anytime algorithm produces a valid answer at any time after it has started. The longer it runs, the better the answer. CP-SAT, Monte Carlo Tree Search, evolutionary algorithms, simulated annealing, branch-and-bound - all are anytime. They have a common shape: maintain a best so far; refine it as long as time permits; return best so far when the budget runs out.

For the simulator, the system call looks synchronous from the trunk’s perspective:

#![allow(unused)]
fn main() {
fn plan_route(world: &World, deadline: Instant) -> Route {
    let mut best = greedy_route(world);
    while Instant::now() < deadline {
        let candidate = improve(&best, world);
        if score(&candidate) > score(&best) { best = candidate; }
    }
    best
}
}

The deadline is the budget. The algorithm respects it. Quality is a function of how much time was available - at 5 ms it is mediocre but valid; at 50 ms it is good; at 500 ms it is near-optimal. The simulator can give it whatever budget the tick allows and never get blocked.

This is §4 applied to a long computation: the budget is named explicitly, and the algorithm honours it. The student who has internalised the budget calculus already knows how to design these algorithms; the only new vocabulary is the anytime contract.

Note - Soft real-time, not hard. The deadline here is a budget, not a guarantee. An anytime algorithm honours its deadline by returning the best answer it has when the deadline arrives; if that answer is mediocre, the system degrades gracefully but still ships a frame. That is soft real-time: a missed deadline costs quality, not correctness. Hard real-time is a different discipline. When a missed deadline is a fault - an avionics control loop, a surgical robot, motor control, an AMR’s emergency stop that fails to fire in time - you need worst-case execution time (WCET) analysis, bounded jitter, no allocation and no syscall in the inner loop, and a scheduler that can enforce deadlines (SCHED_DEADLINE, priority-inversion avoidance). The trunk’s tick budget is soft by construction. Building a hard-real-time controller on these ideas is real work the book does not do; for that frontier, start with the WCET and SCHED_DEADLINE literature.

Time-sliced computation

Some work cannot be made anytime - there is no “best partial answer” until the work is complete. A spatial search that has examined 20 % of the cells has a 20 % chance of having found the answer; otherwise it has nothing useful to report. For these, the pattern is time-slicing: divide the work across many ticks, with the system’s progress as part of its persistent state.

#![allow(unused)]
fn main() {
struct SpatialSearch {
    target_px: f32, target_py: f32,
    cursor:     usize,            // next cell to examine
    best:       Option<(u32, f32)>, // (creature_id, distance) so far
}

fn step_search(s: &mut SpatialSearch, world: &World, max_cells: usize) {
    let end = (s.cursor + max_cells).min(world.cells.len());
    for cell in s.cursor..end {
        for &id in &world.cells[cell] {
            let i = id as usize;
            let d = distance(world.creatures.px[i], world.creatures.py[i], s.target_px, s.target_py);
            if s.best.map_or(true, |(_, prev)| d < prev) {
                s.best = Some((id, d));
            }
        }
    }
    s.cursor = end;
}
}

Each call examines max_cells cells. The simulator runs step_search every tick (or every N ticks); progress accumulates in cursor and best; when cursor reaches the end, the search is complete and the result is delivered. From the simulator’s perspective, the search is one system that takes its budget every tick until done.

This is §15 applied to a long computation: the system’s state at tick start includes its in-progress work. The buffering rule that lets every system see consistent input also lets a system pick up where it left off.

Out-of-loop computation

For work that is genuinely too large for any tick budget - a game AI re-planning its grand strategy, an offline machine-learning model, a remote optimisation service - the pattern is out-of-loop: the work runs on a separate thread, process, or machine, completely outside the simulator’s tick. The simulator never blocks. When the work completes, its result enters the simulator through the input queue (§35) like any other input event.

#![allow(unused)]
fn main() {
// Out-of-loop, on a worker thread:
fn ai_planner_thread(snapshot_rx: Receiver<WorldSnapshot>, result_tx: Sender<InputEvent>) {
    while let Ok(snapshot) = snapshot_rx.recv() {
        let strategy = compute_counter_strategy(&snapshot); // could take seconds
        let _ = result_tx.send(InputEvent::StrategyUpdate(strategy));
    }
}

// Inside the simulator's tick:
fn dispatch_ai(world: &World, snapshot_tx: &Sender<WorldSnapshot>) {
    if world.tick % 30 == 0 { // every second at 30 Hz
        let _ = snapshot_tx.try_send(snapshot_of(world));
    }
}
}

The simulator dispatches a snapshot every second; the AI thread chews on it; the strategy update lands in the input queue some time later. The strategy might be three ticks late, or three seconds late - the simulator does not know and does not care. The result is just one more input event; the queue mechanism is the same.

This is §35 applied to a long computation: anything that crosses the boundary takes its own time, and the queue absorbs the latency. The discipline is not to wait - never block the tick on an out-of-loop result.

Hierarchical scheduling

Production simulators usually combine these patterns. Game engines run physics at 60 Hz (every-tick), AI at 5 Hz (every-12-ticks), save-game at 0.1 Hz (every-300-ticks), and a strategic planner out-of-loop on a worker. Industrial control loops run inner loops at 1 kHz and outer loops at 10 Hz. The DAG generalises: each system is annotated with its cadence; the scheduler runs each according to its frequency or trigger; the result is a system of systems - one architecture, many cadences.

The chapter is constructive: it names the three patterns and shows where each fits the simulator’s existing structure. The next phase, Discipline, addresses what comes after: how to keep the architecture working as it ages, as people leave, as requirements change. Making it work is this chapter; keeping it working is the four chapters that follow.

Exercises

1. Audit cadence. For each system in your simulator, name its cadence. Most are “every tick”; the ones that are not are candidates for the patterns in this chapter. Note any system whose work is currently capped or skipped because it would exceed the budget - these are unmet needs the patterns can serve.
2. Anytime path-finder. Implement plan_route(world, deadline) for one creature. The function returns the best path found within the deadline. With a 5 ms deadline, time how good the answers are; with 50 ms, how much better. Plot quality vs deadline.
3. Time-sliced spatial search. Implement SpatialSearch and step_search as in the prose. Run it across multiple ticks, advancing the cursor by a budget-bounded max_cells each tick. Verify the result is identical to a single-pass search done in one go.
4. Out-of-loop AI. Spawn a worker thread that receives world snapshots and returns strategy updates via channels. Dispatch a snapshot every second; let the worker take 5 seconds; observe that the simulator’s tick rate is unaffected and the strategy update lands at the queue when ready.
5. Mixed cadence. Run your simulator with motion at every tick, the spatial compaction every 50 ticks, snapshot every 1000 ticks, and a (mock) AI thread updating strategy out-of-loop. Verify that determinism still holds: same seed plus same input queue produces identical hashes after 1000 ticks.
6. (stretch) Anytime under varying budget. Modify the path-finder so its caller passes the remaining tick budget each time. Some ticks have plenty of budget; some have very little. The path-finder still returns a valid answer in every case, and the answers improve when the budget allows. Plot quality over time as the simulator runs.

Reference notes in 39_system_of_systems_solutions.md.

What’s next

§40 - Mechanism vs policy opens Discipline: the rules that hold the architecture together over time. Where this chapter was about making the system work for problems that don’t fit the standard tick, the next four chapters are about keeping it working as it ages.

Solutions: 39 - System of systems

Exercise 1 - Cadence audit

For a typical simulator the breakdown looks like:

The unmet-need column is what the chapter speaks to: any work currently being skipped or truncated is a candidate for one of the three patterns.

Exercise 2 - Anytime path-finder

#![allow(unused)]
fn main() {
use std::time::Instant;

fn plan_route(world: &World, start_px: f32, start_py: f32, goal_px: f32, goal_py: f32, deadline: Instant) -> Route {
    let mut best = greedy_route(world, start_px, start_py, goal_px, goal_py);
    let mut iter = 0;
    while Instant::now() < deadline {
        let candidate = local_search_step(&best, world);
        if cost(&candidate) < cost(&best) {
            best = candidate;
        }
        iter += 1;
    }
    eprintln!("plan_route: {iter} iterations, cost {}", cost(&best));
    best
}
}

Typical numbers (illustrative; depends on the map):

• 1 ms deadline: ~10 iterations, ~50% optimal
• 5 ms: ~50 iterations, ~75% optimal
• 50 ms: ~500 iterations, ~95% optimal
• 500 ms: ~5000 iterations, ~99.5% optimal

The shape - diminishing returns over time - is generic to anytime algorithms. The deadline is the budget; quality scales with the budget; the simulator never waits past the deadline.

Exercise 3 - Time-sliced spatial search

#![allow(unused)]
fn main() {
struct SpatialSearch {
    target_px: f32, target_py: f32,
    cursor:     usize,
    best:       Option<(u32, f32)>,
    done:       bool,
}

fn step_search(s: &mut SpatialSearch, world: &World, max_cells: usize) {
    let end = (s.cursor + max_cells).min(world.cells.len());
    for cell in s.cursor..end {
        for &id in &world.cells[cell] {
            let i = id as usize;
            let d = distance(world.creatures.px[i], world.creatures.py[i], s.target_px, s.target_py);
            if s.best.map_or(true, |(_, prev)| d < prev) {
                s.best = Some((id, d));
            }
        }
    }
    s.cursor = end;
    if s.cursor == world.cells.len() {
        s.done = true;
    }
}
}

To verify: run the time-sliced version across K ticks with max_cells = total_cells / K. Compare with a single-pass search. The results must be bit-identical because both visit the same cells in the same order.

Exercise 4 - Out-of-loop AI

#![allow(unused)]
fn main() {
use std::sync::mpsc::{channel, Receiver, Sender};
use std::thread;

fn spawn_ai(world_snapshot_rx: Receiver<WorldSnapshot>, ev_tx: Sender<InputEvent>) {
    thread::spawn(move || {
        while let Ok(snapshot) = world_snapshot_rx.recv() {
            let strategy = compute_counter_strategy(&snapshot); // takes seconds
            let _ = ev_tx.send(InputEvent::StrategyUpdate(strategy));
        }
    });
}
}

The simulator’s tick:

#![allow(unused)]
fn main() {
if world.tick % 30 == 0 {
    let _ = world_snapshot_tx.try_send(snapshot_of(&world));
}
// Drain any AI results from the input queue
for ev in world.in_queue.drain(..) {
    apply_input(&mut world, ev);
}
}

Time the simulator’s tick rate. With the AI computation taking 5 seconds and the simulator running at 30 Hz, the simulator should sustain its full 30 Hz throughout - the AI thread does not block the tick. The strategy update arrives 5 seconds after the snapshot was sent and lands in the input queue at the tick boundary it arrives at.

Exercise 5 - Mixed cadence with determinism

The key insight: each cadence is itself deterministic if its trigger is deterministic. “Every 50 ticks” is a deterministic trigger (if tick % 50 == 0). An “out-of-loop AI” is harder - its results depend on wall-clock timing and may not be reproducible.

For a deterministic system with out-of-loop work, treat the AI’s results as part of the input log. Replay re-feeds the same results at the same ticks they originally arrived. The simulator stays deterministic; the AI’s computation is no longer in the loop, but its inputs are.

Test:

#![allow(unused)]
fn main() {
let mut w1 = init_world(0xCAFE);
let mut w2 = init_world(0xCAFE);
for tick in 0..1000 {
    w1.in_queue.extend(recorded_inputs[tick].iter().cloned());
    w2.in_queue.extend(recorded_inputs[tick].iter().cloned());
    tick_with_mixed_cadence(&mut w1, recorded_time[tick]);
    tick_with_mixed_cadence(&mut w2, recorded_time[tick]);
}
assert_eq!(hash_world(&w1), hash_world(&w2));
}

If determinism holds, the cadences compose; if not, an out-of-loop result is leaking non-determinism that the input queue did not capture.

Exercise 6 - Anytime under varying budget

#![allow(unused)]
fn main() {
fn step(world: &mut World, plan_budget: Duration) {
    let deadline = Instant::now() + plan_budget;
    for creature_id in world.planning.iter() {
        let i = *creature_id as usize;
        let route = plan_route(world, world.creatures.px[i], world.creatures.py[i],
                               world.goal_px[i], world.goal_py[i], deadline);
        world.routes[i] = route;
    }
}
}

The remaining-tick budget is what is left after the higher-priority systems have run. Some ticks: 10 ms remaining, plenty for path-finding. Other ticks: 0.5 ms, only enough for greedy answers. The path-finder returns a valid path in both cases; quality varies; the simulator never blocks.

Plotting route_quality(t) over many ticks shows quality oscillating with budget, with steady-state quality reflecting the typical budget. The pattern is the simulator’s response to load - when the system is busy, planning gets less time; when idle, more time. No system is ever starved or stalled.

40 - Mechanism vs policy

Concept node: see the DAG and glossary entry 40.

Note - Building it to last. Sections 1-39 made the simulator work - deterministic, scaled, parallel, persisted. These four chapters make it last. They are still act one - building - but they answer the questions you ask of a thing you intend to keep: is it extendible (can it grow without a rewrite), is it maintainable (can someone who is not you change it safely), and does it hold its performance and its memory as it grows. Mechanism vs policy, deferred abstraction, dependency pricing, and tests-are-systems are the discipline for those four. The fifth question - can you operate it in service, with nobody watching - is the second act, and it opens at §45. Mechanism vs policy is where the discipline starts: a kernel you can keep is a kernel whose rules can change without it.

The kernel of a system exposes verbs. The rules - what’s allowed, what triggers what - live at the edges. Confusing the two is how systems calcify; once a kernel knows about a rule, the rule cannot change without rewriting the kernel.

The principle is older than ECS. It is named in operating-system kernel design (Mach, X11, Plan 9 all teach this rule), in network-protocol design (TCP is mechanism, congestion control is policy), and in file-system design (read/write/seek is mechanism, access control is policy). The same shape applies to ECS systems.

In the simulator:

• cleanup is mechanism. It takes to_remove and to_insert, applies them via swap_remove and push, and updates id_to_slot. It has no opinion about which creatures should be removed or why. It just commits the changes its callers asked for.
• apply_starve is policy. It reads creature.energy and pushes ids of creatures with energy <= 0 to to_remove. The rule “creatures die when energy reaches zero” lives here. Change the rule to energy < -10 or energy < threshold for 100 ticks and only apply_starve changes; cleanup stays the same.

The separation pays off in three places.

Replaceable rules. A new gameplay variant - “creatures don’t die, they hibernate” - is a new policy on top of unchanged mechanism. apply_starve becomes apply_hibernate; cleanup still works because cleanup does not know what these systems are doing. The kernel is stable; rules are mobile.

Composable rules. Two policies acting on the same kernel compose: one system marks “expired” creatures, another marks “predated” creatures. Both push to to_remove. Cleanup applies both batches without knowing why either was set.

Testable rules. A test fixture sets up to_remove and to_insert directly, runs cleanup alone, and asserts on the result. The mechanism is testable in isolation. Each policy’s test fixture sets up creatures and asserts on what the policy pushes to the buffer. Mechanism tests and policy tests don’t need each other.

The anti-pattern: a food_spawn that mutates food directly:

fn food_spawn(food: &mut Vec<Food>, /* ... */) {
    if some_condition {
        food.push(/* ... */); // BUG: bypasses to_insert
    }
}

Now food_spawn is doing both the deciding (when food appears) and the committing (writing to food). Two changes need rewriting it: a new spawn rule (policy change) and a new cleanup mechanism (mechanism change). They have become the same change. The kernel is married to its current rule.

The fix is to push to to_insert instead, letting cleanup commit. The two roles are separable because they were designed to be - through the buffering pattern from §22, which is itself a mechanism-vs-policy separation. The mechanism is “apply changes at the boundary”; the policy is “what changes to apply”.

Mechanism vs policy is therefore not a separate discipline. It is the rule that every previous chapter has been respecting implicitly. Naming it makes it visible.

Exercises

1. Find the mechanism. For each system in your simulator (motion, food_spawn, next_event, apply_eat, apply_reproduce, apply_starve, cleanup, inspect), classify: is this mechanism (committing what something else asked for), policy (deciding what to ask for), or both? Note where each role lives.
2. Replace a policy. Change apply_starve’s rule from energy <= 0 to energy < -10 && age > 100. Confirm: only apply_starve changes; cleanup stays untouched.
3. Add a new policy on the same mechanism. Write a new system apply_predation that pushes ids of “predated” creatures (some other rule) to to_remove. The two policies’ outputs both flow to cleanup, which applies them without distinction.
4. Spot the anti-pattern. Find any place in your simulator where a system writes directly to a “live” table instead of to to_insert or to_remove. Refactor.
5. (stretch) A second mechanism. Suppose you want a “soft delete” - creatures move to a dead table instead of being removed. Implement a new mechanism (cleanup_with_archive) without touching the existing policies. The same to_remove ids; different mechanism applied. Switch between them by swapping the system in the DAG, not by editing the systems that produce the data.

Reference notes in 40_mechanism_vs_policy_solutions.md.

What’s next

§41 - Deferred abstraction is the discipline for writing the kernel-and-policies in the first place: write three concrete cases before extracting any abstraction.

Solutions: 40 - Mechanism vs policy

Exercise 1 - Find the mechanism

cleanup is the cleanest mechanism in the simulator: no decisions, just commits. inspect is the cleanest read-only mechanism. apply_starve and apply_reproduce are mostly policy with a thin mechanism layer.

Exercise 2 - Replace a policy

#![allow(unused)]
fn main() {
fn apply_starve(/* ... */) {
    for i in 0..energy.len() {
        if energy[i] < -10.0 && age[i] > 100 {  // changed from `<= 0.0`
            to_remove.push(ids[i]);
        }
    }
}
}

cleanup does not change. The simulator behaves differently; the kernel is unmoved. This is the test of separation: did the change touch only one file?

Exercise 3 - Add a new policy

#![allow(unused)]
fn main() {
fn apply_predation(/* ... */) {
    for i in 0..creatures.len() {
        if is_predated(creatures[i], world) {
            to_remove.push(ids[i]);
        }
    }
}
}

Both apply_starve and apply_predation push to to_remove. Cleanup applies the union without distinction. The two policies compose because they produce the same shape of output (an id to remove); the mechanism does not care why.

Exercise 4 - Anti-pattern

Common offender: a logger system that writes directly to disk inside the system body, rather than pushing to an output queue. It mixes “decide what to log” (policy) with “write to disk” (mechanism). Refactor the disk write into a queue + cleanup-style flush.

Exercise 5 - A second mechanism

A cleanup_with_archive mechanism reads to_remove and, instead of swap_remove-ing the row, copies it into a dead archive table before removing it from creatures. Policies (apply_starve, apply_predation) are unchanged - they still push to to_remove. Switch between the two mechanisms by swapping which one is in the DAG, not by editing any policy.

41 - Deferred abstraction

Concept node: see the DAG and glossary entry 41.

The instinct most programmers acquire from training is abstract early. See a case; imagine the second case; design an interface that handles both. The early abstraction feels tidy. It also breaks down the moment the third or fourth case turns out not to fit.

The data-oriented discipline is the opposite. Write the concrete case three times before extracting anything. Then look at the three concrete versions and ask whether the abstraction that fits all three is obvious. Often it is, and the extraction is mechanical. Sometimes it is not - the three cases share less than expected, and the right move is to leave them concrete.

Walk through the failure mode. You write the simulator’s motion system. You can already see motion would also apply to food drift, particle effects, projectile trajectories. The instinct says: design a generic Movable interface. The discipline says: don’t yet. Write motion. Move on.

When the second case arrives - say, food drift - you write it concretely. Maybe it shares 80 % of motion’s structure. Maybe only 60 %. You see this clearly because both versions exist as concrete code, not as imagined cases.

When the third case arrives, look at all three. Now the shared structure is measured, not imagined. If the abstraction is obvious, extract it. If the three cases share only a vague shape, leave them. A bad abstraction is more expensive than three concrete versions of similar code.

The cost saving is in the avoided abstractions. A library of premature interfaces - Movable, Mortal, Hungry, Reproductive trait hierarchies - is a library of code-shaped scar tissue. Each interface fits some of its uses well and others poorly. The misfits add casts, downcasts, defaults, and special cases. Concrete code has none of these.

The Rust ecosystem demonstrates deferred abstraction repeatedly. std::iter::Iterator is the abstraction over many concrete iteration patterns; it earned its place because the concrete patterns existed first and were obviously shared. serde is the abstraction over serialisation; it earned its place because every serialisation library was writing the same boilerplate before serde existed. These abstractions feel inevitable because they are generalizations of patterns the community had already written by hand many times.

The discipline is structural, not stylistic. Generalize when you can see the shape, not before. The book’s own through-line uses it. The simulator was built one concrete piece at a time. The DAG was named after the systems were built, not before. The trunk vocabulary is the generalization of patterns that actually emerged.

A useful test: after extracting an abstraction, can the abstraction handle a fourth case without a special branch? If yes, the abstraction is real. If no - if the abstraction grew an if-else for the fourth case - the abstraction was wrong, and the fourth case is the case showing it.

The connection to the next chapter is concrete. A third-party library is somebody else’s abstraction - one they generalized from their concrete cases. If your three concrete cases match theirs, the library fits and adopting it saves real work. If they do not, the library is friction at every use. §42 develops this into the dependency-pricing discipline.