An Introduction to Programming using entity-component-systems & existence-based processing in python

written by Bjorn Madsen updated: 2026-06-21

Read online: Codeberg · GitHub Pages

Clone source (the public default branch is the rendered book; the runnable code lives on main):

git clone --branch main https://codeberg.org/root-11/intro-book-python.git
or
git clone --branch main https://github.com/root-11/intro-book-python.git

Issues: Codeberg · GitHub

This book teaches programming from first principles of data-oriented design, entity-component-systems (ECS), and existence-based processing (EBP). It uses Python and numpy as the only languages.

EBP is this book’s shorthand. The spelled-out term - existence-based processing - is Richard Fabian’s, from Data-Oriented Design; §17 builds it from the simulator. An acronym index will not list “EBP” because the source literature spells the term out rather than abbreviating it.

The book is structured around forty-three concepts (the DAG) and their canonical wording (the glossary). Sections are short - two to three pages of prose followed by four to twelve compounding exercises. Concepts are named only after they are built: every section earns its vocabulary through working code, not the other way around.

The through-line is a small ecosystem simulator built in stages from one hundred wandering creatures to a hundred million streamed ones. The simulator’s specification is at code/sim/SPEC.

This is the Python edition - a sister volume to the Rust edition of the same book. Same forty-four sections, same DAG, same simulator. The variation is per-chapter commentary on what Python’s defaults push the reader into, and why ECS and EBP win even in a slow language. The thesis the edition carries: ECS and EBP beat OOP because they process more efficiently (operations grouped over arrays), they extend more cleanly (data-oriented composition over class graphs), and they have smaller memory footprint (typed columns over object graphs).

What carries this edition is the evidence. Every load-bearing claim is backed by a measurement the reader can reproduce on their own laptop in under a minute. The exhibits live in code/measurement/ and run via uv run code/measurement/<file>.py.

This is a work in progress. Section ordering is by the DAG; reading order can be linear (front to back) or by following the cross-links wherever they lead.

Who this book is for

You used Python last week. You wrote a class, put instances in a list, iterated over them. Your code worked, but it was slower than you expected, and you have started wondering whether the standard idioms are the bottleneck.

This book is for people who want to find out. The premise is that they are - and that the architecture this book teaches is what Python is fast in, when Python is fast at all.

Many online books include a playground that runs the code in your browser. This one does not, on purpose: the measurements only mean something when they come from your hardware.

Background

You should be comfortable with high-school algebra and a command line - running a command, changing directories, reading error messages without panic. A laptop with internet is enough; the book uses Python 3.11+, numpy, and uv for environment management. Everything else is standard library.

You do not need prior expertise in numerics, parallel computing, or game development. The book teaches numpy and the simulator together; the language is a vehicle, not the subject.

A first taste

Before any vocabulary is named, here is what an ECS world looks like in fifteen lines of Python. One hundred creatures, each with a position and a velocity, moving for thirty ticks of simulated time. No classes, no instances, no method calls - four numpy arrays indexed in lockstep, and a function (the per-tick update) that advances every creature in one stride.

import numpy as np

n = 100
x  = np.arange(n, dtype=np.float32) * 0.1
y  = np.sin(np.arange(n, dtype=np.float32))
vx = ((np.arange(n) * 7) % 11).astype(np.float32) * 0.01 - 0.05
vy = ((np.arange(n) * 13) % 7).astype(np.float32) * 0.01 - 0.03

for tick in range(30):
    x += vx
    y += vy
    if tick % 10 == 0:
        print(f"tick {tick}: creature 17 at ({x[17]:.2f}, {y[17]:.2f})")

Run it locally. Three lines print, the script stops. That is the entire shape of what the rest of the book grows: tables (the four arrays), a tick (the outer loop), a system (the per-tick update). Everything that follows is the discipline that lets this same shape carry a hundred million creatures without falling apart.

The familiar Python shape - a Creature class, a list of instances, a step() method - works at this size too. It stops working at a million, and the reason is in §2: an order of magnitude more memory per creature, an order of magnitude slower per tick. The book teaches the layout that survives the next zero.

Running the code

Python has no equivalent of the Rust Playground - there is no browser-hosted runner that reproduces the numbers a chapter quotes. Every measurement and exhibit in this book runs locally, using uv to manage the Python toolchain and environment. To run anything, you will want a clone of the book’s repo:

git clone https://codeberg.org/root-11/intro-book-python.git
cd intro-book-python
uv run code/measurement/cache_cliffs.py

Each code/measurement/<name>.py file is one exercise group, runnable in isolation. The numbers it prints are yours - they come from your hardware. The exercise asks “how fast does your machine run this?”, and that question only has a real answer locally.

From the simulator chapters onward (§11+), the exercises stop being self-contained scripts. They build the through-line: a Python program that grows from one hundred wandering creatures to a hundred million streamed ones. That program holds state between runs, which is what uv run and the project layout buy you.

The companion edition

If you already know Python well and want compile-time enforcement of the discipline this book teaches by convention, the Rust edition (codeberg, github) covers the same forty-four sections in Rust. The architecture is identical; the language differs. Many readers find that watching the borrow checker enforce in Rust what this edition asks for as discipline is a useful calibration in the other direction too.

Nomenclature

Quick reference for symbols, notation, and abbreviations the book uses. Concept definitions live in the glossary; this page covers the shorthand only.

Symbols

Text formatting

Variables you will see across chapters

Python types and their numpy counterparts

This book uses numpy’s typed dtypes for hot data. The mapping the reader will see most often:

Abbreviations

EBP is this book’s shorthand. The spelled-out term - existence-based processing - is Richard Fabian’s, from Data-Oriented Design; §17 introduces it from the simulator. An acronym index will not list “EBP” because the source literature spells the term out rather than abbreviating it.

Conventions for code blocks

1 - The machine model

Concept node: see the DAG and glossary entry 1.

Most explanations of “how a computer works” use a diagram with a CPU and a single big block called memory. The diagram is wrong. Memory is many things at different speeds, and which one your data sits in decides whether your program is fast or slow.

Inside the CPU there is L1 cache - small, sometimes only 32 KB per core, but a read from it costs about one nanosecond. Around it sits L2 - a few hundred KB, around 3-4 ns. Then L3 - measured in megabytes, around 10 ns. Outside the CPU sits main memory (RAM) - gigabytes, around 100 ns per read. The numbers vary by chip; the ratios are stable. L1 is roughly a hundred times faster than RAM.

When your code reads arr[17], the CPU does not pull just byte 17. It pulls a whole 64-byte chunk - a cache line - and keeps that line in L1. The next read of arr[18] is then almost free. Reading sequentially is fast because every line that gets loaded is mostly used before it gets evicted. Reading at random is slow because every read costs a fresh trip to RAM.

A pointer is an address in memory. Following one is one memory read at an address the CPU does not get to predict. If the address is in cache, the read is fast; if not, you wait the full ~100 ns. A program with many objects and many pointers between them is a program with many of those waits.

Why you have not had to think about this

If you used Python last week, none of the above came up. The interpreter ran your code, the operating system handed it memory, and it worked. You felt no cliff at 100 KB or 100 MB. You wrote a for loop, the loop ran, and the cost per element was whatever it was.

That experience is real, and it is hiding the machine from you. The cost of one iteration of a Python for loop - PyObject_Add, the refcount increment, the PyLong boxing, the bytecode dispatch - is around 5 nanoseconds per element on this machine. That number is higher than an L3 cache miss. So when you iterate over a Python list, the cache hierarchy is invisible to you: you spend so long in the interpreter on every step that whether the next byte was in L1 or had to come from RAM is rounding error.

This is the missing piece of the machine model in Python. The hierarchy is still there; the bottleneck just moved. To see the machine, you have to look in places where the interpreter dispatch isn’t dominating. Two such places, both measurable on your laptop:

1. Sum a million int64s, three ways. code/measurement/cache_cliffs.py walks N from 10K to 100M and times: sum(lst) on a Python list, arr.sum() on a contiguous numpy array, and arr[idx].sum() where idx is a shuffled permutation. On this machine:

Read the columns (3-run medians; the sub-nanosecond seq column is noisy run-to-run, the gather column and the RAM ratios are the stable claims). The Python list is flat at ~4.6 ns/element across five orders of magnitude. From inside the interpreter the cache hierarchy does not exist. The numpy sequential column is 25-30× faster and reveals the bandwidth - the inner loop is C, the bytes are typed, the prefetcher works. The numpy gather column is the same data accessed in a shuffled order; while it fits in cache the gather penalty is small (~5-17×), and once the working set spills L3 (between 1M and 10M) the per-element cost climbs sharply, reaching 80× the sequential cost at 100M. That ratio is the L1-to-RAM cost gap on this machine, measured.

2. Take an exception once vs a million times. code/measurement/try_except.py compares try/except ZeroDivisionError against an explicit if value != 0 check, across hit rates from 0.0001% to 99.9999%. At 50/50 the try/except form is 4× slower; at 99.9999% (almost no exceptions raised) the try/except form is faster than the if. The difference is the CPU’s branch predictor: a taken branch with high frequency is essentially free; a mispredicted one costs ~10-20 cycles. The lesson is not “use try/except” or “use if” - it is that constant factors are rate-dependent, and even Python inherits this.

3. Constant factors leak through. code/measurement/string_methods.py compares %-format, f-strings, and .format for the same output. On this machine %-format is ~20% faster than f-strings, which are ~5% faster than .format. None of this matters in a one-off log line. All of it matters in a tight loop. The “modern idiomatic” choice is not automatically the cheap choice.

What this chapter is asking you to do

The dominant fact about modern CPUs is that arithmetic is virtually free; the cost is getting the data to the arithmetic. A program that respects this is fast. A program that ignores it can be a hundred times slower than a program that does the same work, with the same number of additions, in a layout the cache likes.

In Python this fact wears a disguise: the interpreter is so slow that the machine appears to have no cliff. The disguise comes off the moment you leave pure Python - and almost everything this book teaches involves leaving pure Python for typed contiguous columns where the cliff is right where it always was.

This is also what makes “complexity class” misleading on its own. An O(N log N) algorithm that hits the cache hard can outrun a “faster” O(N) algorithm that scatters reads across RAM. Big-O describes how cost grows with N; layout describes the constant factor that gets multiplied in. At the scales this book targets, the constant factor often wins.

Exercises

These exercises are calibrations. Run them on your machine and write the numbers down - the rest of the book references them.

1. Look up your cache sizes. On Linux, lscpu | grep -i cache lists L1d, L1i, L2, L3 per core. (On macOS: sysctl -a | grep cache.) Write them down. These are the budgets §27 will hold you to later.
2. Run the cache-cliffs exhibit. uv run code/measurement/cache_cliffs.py. Read the output. Note the size at which the numpy gather column starts climbing - that is where you spilled out of L1. Note where it climbs again - L2, L3.
3. Confirm the interpreter mask. Modify the exhibit to print arr.tolist() sum at every size step alongside the existing measurements. Confirm that the Python list cost is still flat - the cliffs do not appear, even though the data is the same.
4. Run the try/except exhibit. uv run code/measurement/try_except.py. Note the cross-over: at what hit-rate does try/except become faster than if? On most machines it lands above 99%.
5. Run the string-format exhibit. uv run code/measurement/string_methods.py. Note the ranking on your machine. The order can shift across CPython versions - measure, do not memorise.
6. A linked list of pointers. Build a chain of 1,000,000 nodes as class Node: __slots__ = ("value", "next"), then sum value by walking .next from the head. Compare against the same sum on a numpy int64 array of the same length. The ratio you see is roughly the L1-to-RAM ratio for one level of indirection in Python - note that this ratio compounds when objects nest deeper.
7. (stretch) Read your lscpu output to your benchmarks. With your cache sizes from exercise 1 and your timings from exercise 2, identify which level of cache each step in the gather column is leaving. The transitions are not always clean - annotate where they are noisy.

Note - Numbers in this chapter were measured on this author’s machine. The shape - flat Python list, staircase numpy, widening gather/seq ratio - is robust across hardware. The exact ratios shift with CPU generation: older or smaller chips (Raspberry Pi 4, 2012-era Intel) show a graded staircase across L1/L2/L3, while modern desktop chips often show one big cliff at the L3-to-RAM boundary. Measure on your own machine; reproduce shapes, not specific numbers.

Reference notes for these exercises in 01_the_machine_model_solutions.md.

What’s next

The cache sizes you wrote down in exercise 1 and the cliffs you found in exercise 2 are the constants behind the whole book. §2 - Numbers and how they fit takes the next step: how big is each unit of data, and how many fit in a cache line?

Solutions: 1 - The machine model

These exercises are about measuring your machine. Numbers vary; ratios are stable. Run them and write down what you see.

Exercise 1 - Cache sizes

Linux: lscpu | grep -i cache. macOS: sysctl -a | grep cache.

Typical desktop x86-64 in 2026: L1d 32-48 KB per core, L2 1-2 MB per core, L3 16-128 MB shared. Apple Silicon: larger L1, very large shared L2. Older or smaller chips (Pi 4, 2012-era Intel) show a graded L1 → L2 → L3 → RAM staircase; modern desktops often show one big cliff at L3 → RAM.

Write the numbers down. §27 refers back.

Exercise 2 - Run the cache-cliffs exhibit

uv run code/measurement/cache_cliffs.py

Source: code/measurement/cache_cliffs.py.

           N     Python list     numpy seq    numpy gather   gather/seq
-----------------------------------------------------------------------
      10,000         5.72 ns      0.420 ns         1.62 ns         3.9×
     100,000         6.01 ns      0.234 ns         2.24 ns         9.6×
   1,000,000         4.78 ns      0.203 ns         3.69 ns        18.2×
  10,000,000         4.46 ns      0.196 ns         7.60 ns        38.7×
 100,000,000         4.59 ns      0.152 ns         7.78 ns        51.3×

Read the columns:
  Python list - roughly flat across sizes; interpreter dispatch dominates.
  numpy seq   - staircase; cliffs reveal L1/L2/L3/RAM transitions.
  numpy gather - random access; gap to seq widens as working set spills caches.

The L1 → L2 step in the gather column is shallow (2-3×). The L3 → RAM step is the dramatic one. The Python list column is the chapter’s whole point: from inside the interpreter the cache hierarchy is invisible.

Exercise 3 - Confirm the interpreter mask

Add to the per-N loop in cache_cliffs.py:

lst = arr.tolist()
t0 = time.perf_counter()
sum(lst)
ns_lst = (time.perf_counter() - t0) * 1e9 / n
print(f"  list cost: {ns_lst:.2f} ns/elem")

Same data, same arithmetic. The number stays in the 4-6 ns/elem band at every N. The cliff is not in the data; it’s in what is touching the data.

Exercise 4 - Run the try/except exhibit

uv run code/measurement/try_except.py

Source: code/measurement/try_except.py. Four points along the rate axis from one author’s run:

At 0% hits (every call raises), try/except costs ~11× more. At 50/50, ~4×. Around 96% hits the two cross over. At ~100% hits, try/except is the cheaper form because no exception is raised and the path is straight-line; the if form pays the comparison every time. The branch predictor does the rest: a branch with a stable outcome predicts ~100% and costs ~0 cycles; a flipping one costs 10-20.

The lesson is not “use one or the other” - it is that constant factors are rate-dependent.

Exercise 5 - Run the string-format exhibit

uv run code/measurement/string_methods.py

Source: code/measurement/string_methods.py. Median over seven runs, one author’s machine:

%-format wins by ~14%. f-string and .format are within 1% of each other on this run; their order flips between CPython versions and between integer-only vs string-heavy payloads. Measure on yours; do not memorise.

Exercise 6 - A linked list of pointers

import time
import numpy as np

class Node:
    __slots__ = ("value", "next")
    def __init__(self, value, nxt=None):
        self.value = value
        self.next  = nxt

def build(n):
    head = Node(1)
    for _ in range(n - 1):
        head = Node(1, head)
    return head

def walk_sum(head):
    s = 0
    while head is not None:
        s += head.value
        head = head.next
    return s

n = 1_000_000
head = build(n)
arr  = np.ones(n, dtype=np.int64)

t0 = time.perf_counter(); walk_sum(head); t1 = time.perf_counter()
arr.sum();                                 t2 = time.perf_counter()

print(f"linked list: {(t1 - t0) * 1e9 / n:.1f} ns/elem")
print(f"numpy array: {(t2 - t1) * 1e9 / n:.2f} ns/elem")

linked list: 18.4 ns/elem
numpy array: 0.36 ns/elem

Ratio ~50×. That is not the full L1-to-RAM ratio, and the reason matters: nodes built in a tight loop land contiguously in memory because the allocator reuses freshly-freed slots. Walking the chain accidentally inherits some of the array’s locality; the prefetcher catches part of it.

To see the cost without that accident, link nodes in shuffled order:

import random, gc
nodes = [Node(1) for _ in range(n)]
order = list(range(n))
random.shuffle(order)
for i in range(n - 1):
    nodes[order[i]].next = nodes[order[i + 1]]
head = nodes[order[0]]
del nodes; gc.collect()

linked list: 107.7 ns/elem
numpy array: 0.36 ns/elem

Now each head.next is an unpredictable jump - close to a full RAM round-trip per node, ~300× slower than the numpy sum.

The structural label “linked list” doesn’t tell you the cost. The layout in memory does. __slots__ is the floor here, not the ceiling - without it, every Node carries a __dict__ and the numbers worsen further.

Exercise 7 - Reading lscpu against your benchmarks

The transitions are noisy because:

• Cache levels overlap (a hot line stays in L1 after spilling to L2).
• Hardware prefetchers help even shuffled accesses up to a point.
• The OS may evict pages between runs.
• The shuffle is fixed across runs; some indices land near recently-touched lines and amortise.

If your noise is worse than your signal: median of five runs. If transitions still don’t line up with lscpu (e.g. L2 is 1 MB but the cliff appears at 200 KB), convert byte budgets to elements - the gather array is 8 bytes per int64, so 1 MB of L2 holds 128K elements, not 1M.

2 - Numbers and how they fit

Concept node: see the DAG and glossary entry 2.

A cache line is 64 bytes on x86 and most ARM chips - the unit of memory the CPU loads at a time. (A few designs differ: some Apple Silicon cache levels use 128; §33 has the details.) This book assumes 64 throughout. Everything you do with data is, in part, a question of how many things fit in one cache line.

What an int actually costs

You wrote x = 1 last week and that was the end of the question. What sat in memory was a PyLong object: a header, a refcount, a length, and one or more 32-bit “digit” limbs holding the value. The minimum size, even for 0, is 28 bytes - and that covers every value below 2³⁰ (about a billion). Past that it grows four bytes per limb, one limb per 30 bits (≈ nine decimal digits), not four bytes per digit. From code/measurement/number_footprint.py on this machine:

int 0                          28 bytes
int 1                          28 bytes
int 256 (last interned)        28 bytes
int 257                        28 bytes
int 1_000                      28 bytes
int 2**31                      32 bytes
int 2**63                      36 bytes
int 2**127                     44 bytes
float 0.0                      24 bytes
float 3.14                     24 bytes
float 1e300                    24 bytes

A PyFloat is 24 bytes, fixed. A PyLong is at least 28 bytes and grows with magnitude. A bool is also a PyLong. A complex is 32 bytes. The header alone is bigger than the value in every case.

This is the first part of the chapter’s question. Picking the narrowest type that holds your range - the discipline that defines whether a cache line packs 8 things or 64 things - does not exist in pure Python. There is no uint8. There is no int32. Every Python int is the same costly object regardless of whether it holds the value 0 or 2**63. You cannot trade range for cache lines, because you cannot pick the range.

Note - CPython caches small integers in [-5, 256] as singletons (the small-int cache). A list of zeros does not allocate a million PyLong(0) objects - it allocates a million pointers, all to the same one. Once the values escape that range, every value is a fresh allocation. Confirm this with id(0) == id(0) (true) versus id(257) == id(257) (sometimes true, sometimes not, depending on the parser’s caching of literal constants in the same compilation unit - but never reliable). Treat the small-int cache as a CPython implementation detail you cannot lean on. Terry Jan Reedy, a CPython core developer, confirmed that 3.15.0b1 and newer raise that upper limit from 256 to 1024 - the boundary moved while this book was being written. The durable claim is the mechanism, not the number.

What numpy gives you back

numpy makes the width budget exist again. np.int8 is one byte, range -128 to 127. np.int16 is two bytes, np.int32 is four, np.int64 is eight. np.float32 is four bytes (~7 decimal digits of precision); np.float64 is eight (~15 digits). The signed/unsigned and integer/float variants compose freely.

A np.zeros(N, dtype=np.uint8) is N bytes - flat, contiguous, no per-element header. A cache line packs 64 of them. A np.zeros(N, dtype=np.int64) is 8N bytes; one cache line packs 8. Walk the whole array and the int64 version pulls in 8× as many cache lines as the uint8 version: the same element count, eight times the bytes. The width budget is back.

Same exhibit, the data column tells the story at N=1,000,000:

The Python-list-vs-numpy ratio at this scale: 40× more bytes in the list compared to numpy int8, 20× vs int16, 10× vs int32, 5× vs int64. Choosing the narrowest numpy width that holds your range gives you up to 8× additional shrink on top of the list-to-numpy step. Sum times collapse from milliseconds to fractions of a millisecond - two orders of magnitude.

Pick the narrowest type that holds your range, and write down why. A 52-card deck’s suits need 4 values, ranks need 13, locations need maybe 8 - all fit in np.uint8. A creature’s pos needs about ten kilometres of grid resolved to centimetre precision; that fits in np.float32. A timestamp in microseconds for a year-long simulation needs something like 3×10¹³, which does not fit in np.uint32 (4×10⁹) but fits comfortably in np.uint64. Choose, and write the choice down.

Floats are not real numbers

They look like real numbers but are not. There are only about 4 billion float32 values; there are only about 18 quintillion float64 values; that is finite. Operations have edges: 1.0 / 0.0 = inf, 0.0 / 0.0 = nan, and nan != nan - yes, equality is broken on purpose for nan, because there is no reasonable answer. But == is also unreliable for ordinary floats: 0.1 + 0.2 == 0.3 is False, because 0.1 and 0.2 cannot be represented exactly in binary and the rounding error happens to land just past 0.3. This is why math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0) exists - it is the standard library’s acknowledgement that == is the wrong tool for floats and that comparing them needs a tolerance you choose deliberately. Subtracting two nearly equal floats loses most of their precision (this is catastrophic cancellation). Adding a tiny float to a large one quietly drops the tiny one (this is absorption). None of this is a problem if you know it is there; all of it is a problem if you assume floats are mathematics.

code/measurement/sums.py demonstrates the consequences across five pathological datasets - random balanced, large-plus-many-small, alternating signs, tiny increments, and arrays containing NaNs - using six summation algorithms (sum, math.fsum, Kahan, Neumaier, pairwise, decimal reference). Run it; read the discrepancies. The same input data summed in different orders gives different answers, and the “naive” answer is sometimes off by orders of magnitude. The fix is not “use float64 instead of float32” - it is picking a summation algorithm aware of the data shape. math.fsum and Neumaier are usually the right defaults for a single-pass sum where you cannot bound the input.

Most of this book uses np.uint8, np.uint16, np.uint32, np.float32, and np.uint64 for time. int* and float64 appear when the range or precision genuinely demands it. The choice is documented at every column declaration.

Exercises

1. Per-value cost. Print sys.getsizeof(0), sys.getsizeof(2**31), sys.getsizeof(2**127), sys.getsizeof(0.0), sys.getsizeof(True). Confirm that even a bool costs 28 bytes (bool is a subclass of int). Now print np.array([0, 2**31, 0], dtype=np.int64).nbytes. Three int64s = 24 bytes total, no headers, no per-value pointers.
2. Cache-line packing. For each numpy dtype - int8, int16, int32, int64, float32, float64 - compute how many fit in a 64-byte cache line. A np.array(_, dtype=np.int32) of 16 elements is exactly one line; a np.array(_, dtype=np.float64) of 8 elements is exactly one line.
3. Width and speed. Sum a np.ones(100_000_000, dtype=np.int8), then a np.ones(100_000_000, dtype=np.int64). The ratio in time should be smaller than the ratio in bytes (8×) because compute is not the bottleneck - memory bandwidth is. Note also that the int8 sum overflows; this is a hint about why the book picks widths with the maximum value in mind.
4. Float weirdness. Compute 0.0 / 0.0, 1.0 / 0.0, (-1.0) ** 0.5, math.sqrt(-1.0). Print them. Then nan = float("nan"); assert nan != nan - confirm it does not raise.
5. == is the wrong tool. Print 0.1 + 0.2 == 0.3. Observe False. Print 0.1 + 0.2 to see the rounding error: 0.30000000000000004. Now use math.isclose(0.1 + 0.2, 0.3) and observe True. Read the math.isclose docs - note that the default rel_tol=1e-9 is a choice you should be making explicitly when the problem demands a tighter or looser tolerance. The standard library has isclose because the language designers know == is unreliable here; lean on it.
6. Catastrophic cancellation. Compute np.float32(1e10) - (np.float32(1e10) - np.float32(1.0)). The result should be 1.0; on float32 it usually is not. Repeat with np.float64 and observe it gets closer (but not always exactly 1.0).
7. Run the summation exhibit. uv run code/measurement/sums.py. Read the discrepancies between the algorithms across the five datasets. Note the dataset where the spread is largest. That dataset is the one that decides which summation routine you should reach for in production.
8. Choose a width. For each of these columns, write down the dtype you would pick and why: a creature’s age in ticks at 30 Hz over a year-long simulation; a card’s suit; the pixel count of a 4K screen; the user id in a system with up to 100 million users; an audio sample value in 16-bit PCM.
9. (stretch) The eps of a float. np.finfo(np.float32).eps is the smallest x such that 1.0 + x != 1.0 in float32. Compute the value, then compute np.float32(1.0) + np.float32(0.5) * np.finfo(np.float32).eps - is the result 1.0 or 1.0 + eps/2? What does this say about a sum of small numbers added one at a time to a large running total?

Reference notes in 02_numbers_and_how_they_fit_solutions.md.

What’s next

§3 - The np.ndarray is a table takes the next step: now that you know how big the elements are, what does an np.array do with them, and what shape does the rest of the book expect them to be in?

Solutions: 2 - Numbers and how they fit

Exercise 1 - Per-value cost

import sys, numpy as np
print(sys.getsizeof(0))         # 28
print(sys.getsizeof(2**31))     # 32
print(sys.getsizeof(2**127))    # 44
print(sys.getsizeof(0.0))       # 24
print(sys.getsizeof(True))      # 28  - bool is a subclass of int
print(np.array([0, 2**31, 0], dtype=np.int64).nbytes)  # 24

A single bool costs 28 bytes - same as a small int. Three int64s in a numpy array cost 24 bytes total: no per-element header, no per-element pointer. That ratio (28 bytes per Python int each, vs 8 bytes per int64 in a column) is the size budget the rest of the book leans on.

Exercise 2 - Cache-line packing

A cache line is 64 bytes:

A np.array(..., dtype=np.int32) of 16 elements is exactly one cache line. A np.array(..., dtype=np.float64) of 8 elements is exactly one. A Python list of anything is one pointer (8 bytes) per element plus the elements as separate objects elsewhere - at most 8 list pointers per line, with the actual values at unpredictable addresses.

Exercise 3 - Width and speed

import numpy as np, time
n = 100_000_000
a8  = np.ones(n, dtype=np.int8)
a64 = np.ones(n, dtype=np.int64)

t0 = time.perf_counter(); int(a8.sum());  t1 = time.perf_counter()
t2 = time.perf_counter(); int(a64.sum()); t3 = time.perf_counter()
print(f"int8  sum: {(t1-t0)*1000:.1f} ms")
print(f"int64 sum: {(t3-t2)*1000:.1f} ms")

int8  sum: 20.8 ms
int64 sum: 14.8 ms

The result is counterintuitive: int64 is faster than int8 despite reading 8× more bytes. The reason is how numpy reductions work. arr.sum() does not accumulate in the array’s dtype; it widens to a 64-bit accumulator by default to avoid silent overflow. That widening means each int8 is read as one byte then promoted to eight bytes inside the loop, so the int8 case pays bandwidth-savings plus per-element widening - and on this machine the widening cost dominates.

To force overflow (the chapter’s “hint about why the book picks widths with the maximum value in mind”), pin the accumulator:

a8.sum(dtype=np.int8)  # now wraps; result is some int8 value, not 100_000_000

The book’s discipline therefore has two parts. Pick the narrowest dtype that holds your values (storage) and be explicit about the accumulator (arithmetic). The two are different choices.

Exercise 4 - Float weirdness

In pure Python, three of the five prompts raise - Python’s defaults protect you from the IEEE 754 edges:

>>> 0.0 / 0.0
ZeroDivisionError: float division by zero
>>> 1.0 / 0.0
ZeroDivisionError: float division by zero
>>> math.sqrt(-1.0)
ValueError: math domain error
>>> (-1.0) ** 0.5
(6.123233995736766e-17+1j)        # promoted to complex, not nan
>>> nan = float("nan"); nan != nan
True

The IEEE 754 behaviour the chapter prose describes - nan and inf from division - surfaces through numpy:

>>> import numpy as np, warnings
>>> with warnings.catch_warnings():
...     warnings.simplefilter("ignore")
...     print(np.float64(0.0) / np.float64(0.0))   # nan
...     print(np.float64(1.0) / np.float64(0.0))   # inf
...     print(np.sqrt(np.float64(-1.0)))           # nan
nan
inf
nan

nan != nan works in pure Python because float("nan") constructs the IEEE bit pattern directly; the generation of nan from arithmetic is what numpy provides and pure Python guards against. Both views matter: when you leave the interpreter for numpy columns you trade exception protection for IEEE behaviour, and you need to know the rules of the side you’re on.

Exercise 5 - == is the wrong tool

>>> 0.1 + 0.2 == 0.3
False
>>> 0.1 + 0.2
0.30000000000000004
>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True

0.1 and 0.2 are not exactly representable in binary; their sum lands one ulp past 0.3. math.isclose exists because the standard library acknowledges == is the wrong tool for floats. The default rel_tol=1e-9 is a choice - make it deliberate when the problem demands a tighter or looser tolerance. The pattern you’ll learn to reach for:

math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0)   # near-zero values need abs_tol too

Exercise 6 - Catastrophic cancellation

import numpy as np
a32 = np.float32(1e10); b32 = a32 - np.float32(1.0)
print(a32 - b32)                          # 0.0  - should be 1.0

a64 = np.float64(1e10); b64 = a64 - np.float64(1.0)
print(a64 - b64)                          # 1.0

float32 has ~7 decimal digits of precision; 1e10 already exhausts them, so 1e10 - 1.0 cannot be distinguished from 1e10 and the subtraction returns 0.0. float64 has ~15 digits, room to spare for this size. The lesson is not “use float64” - it is that the right precision depends on the dynamic range of the values you’ll subtract. A simulation that subtracts two large nearly-equal positions to compute a small velocity needs the wider type even if the final answer fits in a narrower one.

Exercise 7 - Run the summation exhibit

uv run code/measurement/sums.py

Source: code/measurement/sums.py. Five datasets × three orders × five algorithms. The dataset where the spread is largest is large_plus_small (a few values of size 10⁶ added to many values of size 1):

=== DATASET: large_plus_small (N=2000002) ===
-- Order: original --
Reference: 2000000
builtin_sum        | time_s: 0.0085 | result: 2000000     | abs_err: 0
math_fsum          | time_s: 0.0077 | result: 2000000     | abs_err: 0
kahan_sum          | time_s: 0.0918 | result: 2000000     | abs_err: 0
neumaier_sum       | time_s: 0.1427 | result: 2000000     | abs_err: 0
pairwise_sum       | time_s: 0.3901 | result: 1999998     | abs_err: 2

pairwise_sum - usually the recommended general-purpose stable summation - is off by 2 absolute on this dataset. Two of the three large values get absorbed during a partial-sum step where they are paired with a million-and-something accumulated 1s. builtin sum, math.fsum, kahan_sum, and neumaier_sum all return the exact integer answer. The lesson: stability across reorderings is not the same as stability across magnitude mixtures. math.fsum is the safest single-pass default when you cannot bound the data; pairwise wins only when magnitudes are uniform.

Exercise 8 - Choose a width

The discipline is to write down why. Two years later, when someone changes the budget (10M users → 1B users), the column type’s reasoning is the diff that matters.

Exercise 9 - The eps of a float

import numpy as np
eps = np.finfo(np.float32).eps            # 1.1920928955078125e-07
print(np.float32(1.0) + np.float32(0.5) * eps)   # 1.0
print(np.float32(1.0) + eps)                     # 1.0000001

Half an eps added to 1.0 is absorbed - the result is still exactly 1.0. One full eps added to 1.0 produces the next representable float above 1.0. This is the unit in the last place rule: floats near 1.0 have a spacing of eps; smaller additions cannot be represented and are silently dropped.

The implication for summation: adding 10⁹ values each of size 0.5 * eps to a running total of 1.0 produces a final total of 1.0, not 1.0 + 5×10². Every step rounds away the contribution. This is the failure mode kahan_sum and neumaier_sum correct: they keep a compensation term that accumulates the dropped bits across iterations and folds them back in. The book uses math.fsum (which keeps full precision via a list of exact partials) when input magnitudes are unbounded.

3 - The np.ndarray is a table

Concept node: see the DAG and glossary entry 3.

A list in Python is a header object on the heap that stores three things: a length, a capacity (over-allocated by a small fraction), and a pointer to a contiguous run of PyObject* pointers. That last word is the lesson. The list does not contain your integers; it contains pointers to integer objects, each allocated separately on the heap. lst[i] reads a pointer from the contiguous run, then dereferences it to find the actual PyLong (28 bytes per int, 24 per float) somewhere else in memory.

If you used Python last week, this is the container you reached for, and it is the right shape for some problems. It is also the wrong shape for almost everything the trunk of this book teaches, which is “process all the rows of a table.” A list of N rows-as-tuples is one big jump table sitting in front of N+10N small objects scattered across the heap. Walking it is pointer-chasing, not sequential reading.

A numpy array - np.array(..., dtype=...) - is the same three-things-on-the-heap shape, but the contiguous run holds values, not pointers. Ten million int64s in a numpy array is 80 MB of contiguous bytes; ten million ints in a list is 280 MB of PyLong objects plus 80 MB of pointers, scattered. arr[i] computes base + i * 8 and reads - once. No object dereference. No allocation per element.

The trunk of this book uses two containers: list for the small bookkeeping (the names of your tables, the schedule of your systems) and numpy.ndarray for the rows. There are no dicts of objects, no class hierarchies, no dataclasses with __slots__ for the things that need to scale. Not because they don’t exist, but because every container that wraps a PyObject per row pays the pointer-chase tax on every read, and the rest of the book is about not paying that tax.

The flip, measured

Take the same data - N rows, K integers per row - and lay it out five ways. The first two are what the official tutorial teaches. The middle two are stdlib-only flips. The fifth is the disciplined endpoint.

code/measurement/aos_vs_soa_footprint.py builds each, in a fresh subprocess so resident set size (RSS) readings don’t bleed, with N=1,000,000 and K=10. Values past the small-int cache so PyLong objects aren’t shared singletons across rows. Three numbers per layout: peak RSS, construction time, time to sum column 0.

Note - Measured on this author’s machine; reproduce on yours with uv run code/measurement/aos_vs_soa_footprint.py. Order-of-magnitude is the durable claim. Numbers will shift with K, N, value range, and CPython version, but the shape - that the AoS-to-SoA flip and the boxed-to-typed flip and the Python-loop-to-C-loop flip are three independent wins - is stable across machines.

The five rows separate three independent decisions that the four-row version conflated.

The mutable AoS is worse than the immutable AoS. Replacing the inner tuples with lists costs ~60 MB of additional list-header overhead at this scale. The “list of lists” pattern is the most-taught layout in introductory Python and the most-expensive one in this comparison.

Step one - AoS → SoA - is the speed flip. Tuple-of-lists is the same code an intermediate Python programmer might write without ever touching numpy. It saves only ~12% on memory but sums column 0 about 10× faster than the AoS forms. The win is the access pattern: walking one contiguous list of 1M PyLong pointers instead of walking 1M tuple objects and dereferencing through each one to reach row[0]. Storage is barely better; the loop is dramatically better.

Step two - boxed list → typed bytes - is the memory flip. Going from list[int] to array.array('q', ...) shrinks each column from ~38 MB of pointers-and-PyLong-objects to ~8 MB of contiguous int64 bytes. The whole structure drops to ~77 MB total, smaller than numpy in this run (numpy carries ~20 MB of one-off import overhead). But the column-sum slows down - 2.5 ms → 11.6 ms - because Python has to unbox each int64 into a temporary PyLong before adding it. The unboxing tax buys back about a third of the SoA speed win. Typed storage saves bytes; it does not save the inner loop.

Step three - Python loop → C loop - is the order-of-magnitude move. np.sum walks the same typed bytes that array.array stored, but the loop is in C and the interpreter is stepped out of the way. 11.6 ms → 0.4 ms; about 30× speedup on the same bytes, no further memory saving (and a small import-overhead cost). This is the layout the simulator (§11+) and every system after it depends on.

Read the three steps together: the SoA flip is the speed move, the typed-storage flip is the memory move, the C-vectorisation flip is the speed move again at a larger scale. Each is a separate decision; each can be taken without the others. Numpy happens to bundle the second and third into one library, which is why most teaching collapses them into “use numpy.” The exhibit shows they are separate wins.

The Python-default trap, named

The official tutorial is not wrong. It’s optimised for teaching the language, not for teaching layout. The path it teaches looks like this:

1. Make a class for the row.
2. Put instances in a list.
3. Reach for dataclass when the class gets noisy.
4. Reach for __slots__ when memory pressure shows up.

Each step is a local improvement and a global trap. Step 1 commits you to AoS. Step 2 puts pointers between the rows. Step 3 makes the AoS more ergonomic. Step 4 saves a per-instance __dict__ but does nothing about the fundamental shape - every row is still its own heap object reached through a pointer. The __slots__ win is real and small; the SoA win is the same data costing 4-5× less memory, and you don’t need a class at all.

There is no such thing as a cost-free abstraction. Every pointer has a cost, and in a list of rows that cost multiplies linearly with the row count. The four-step path stacks pointers: an outer list of N row-pointers, each row pointing to K field objects, each field a separately allocated value somewhere else on the heap. __slots__ removes one layer (the per-instance __dict__); the SoA flip removes the rest. The next several phases of this book teach the alternative.

Exercises

1. Pointer-chase or value-read. Print sys.getsizeof(0), sys.getsizeof(1000), sys.getsizeof(10**100). Note that even a small Python int costs 28 bytes. Now print np.array([0, 1000, 10**18], dtype=np.int64).nbytes. Three int64s = 24 bytes, and there are no per-element headers.
2. The interning trap. Repeat exercise 1 with values 0 and 1, then again with values 257 and 1000. Use id() to confirm that [0] * 1_000_000 shares one PyLong object across all positions, but [1000 + i for i in range(1_000_000)] does not. The “list of small ints is cheap” intuition only holds inside CPython’s small-int cache [-5, 256].
3. Capacity vs length. Build lst = []. In a loop, append 0..1000 and print len(lst) and sys.getsizeof(lst) after each step. Observe the over-allocation pattern - list grows in chunks, like Vec::push, but the chunks are CPython implementation detail (currently ~1.125 × growth). Now try to grow a numpy column the same way: there is no in-place append. np.append(arr, i) returns a new array and copies the old one on every call, so arr = np.append(arr, i) across range(20_000) is O(N²). Time it against the list loop and watch the curve bend. The list grew for you; the numpy column will not. §10 pays this bill: a table that produces rows at runtime sizes a buffer ahead and tracks a logical length itself.
4. Run the §3 exhibit. uv run code/measurement/aos_vs_soa_footprint.py. Read the output. The sum-c0 column matters: even if you ignore the memory line, the column-sum cost gap between layouts 1 and 4 is two orders of magnitude on the same data.
5. The dict trap. Build d = {i: i*i for i in range(1_000_000)} and time looking up 100,000 random keys. Build arr = np.arange(1_000_000) ** 2 and time the same access pattern via arr[idx]. Note that you have replaced “look up by integer” with “index by integer,” and the structures cost different amounts.
6. swap-remove vs remove. Build lst = list(range(1_000_000)). Time removing 100 elements from the middle by lst.pop(500_000) (slow - every pop shifts ~half the list). Time the equivalent via lst[i] = lst[-1]; lst.pop(). Note the orders-of-magnitude difference. This trick will earn its keep at §21.
7. (stretch) Read your own array. Use np.frombuffer(arr.tobytes(), dtype=np.int64) and confirm that arr.data.tobytes() is exactly arr.size * 8 bytes long. The bytes you would write to disk are the bytes already in memory. This is what §36 - persistence means by “tables serialise themselves.”

Reference notes in 03_the_vec_is_a_table_solutions.md.

Applied reference

If you want to see this discipline carried through a real piece of code, read .archive/simlog/logger.py. It is a 700-line columnar logger that parks dict payloads into pre-allocated numpy columns, with a double-buffered design that lets the simulation write to one buffer while a background thread dumps the other to disk. The book does not require you to read it now. It’s the destination this chapter and the next several point at.

What’s next

§4 - Cost is layout, and you have a budget takes the layout reasoning into per-tick territory: how many bytes can you actually move per tick on your machine, and what does that buy you in entities? After that, §5 - Identity is an integer is where the through-line simulator gets its first concrete shape.

Solutions: 3 - The np.ndarray is a table

Exercise 1 - Pointer-chase or value-read

>>> import sys, numpy as np
>>> sys.getsizeof(0)            # 28
>>> sys.getsizeof(1000)         # 28
>>> sys.getsizeof(10**100)      # 72  - large ints grow per limb
>>> np.array([0, 1000, 10**18], dtype=np.int64).nbytes
24

Three int64s in a numpy array: 24 bytes, no per-element headers. Three PyLongs in a list: 28 + 28 + 72 = 128 bytes for the values plus 8 × 3 = 24 bytes of pointers in the list’s backing array plus the list header. The numpy column is the values; everything else in the list version is bookkeeping.

Exercise 2 - The interning trap

>>> a = [0] * 1_000_000
>>> b = [1000 + i for i in range(1_000_000)]
>>> len(set(id(x) for x in a[:100]))         # 1   - all the same object
>>> len(set(id(x) for x in b[:100]))         # 100 - every value is its own PyLong

[0] * 1_000_000 does not allocate a million PyLong(0)s; it allocates a million pointers, all to one shared 0 object. The list weighs 8 MB of pointers + one 28-byte int. The intuition “a list of small ints is cheap” is true inside CPython’s small-int cache ([-5, 256]) and false everywhere else.

id(257) == id(257) and id(1000) == id(1000) may both return True within a single statement because the parser caches literal constants in a compilation unit. Across statements, identity is not guaranteed for values outside [-5, 256]. Don’t lean on that - it’s an implementation detail of how the bytecode compiler stores literals, not a runtime property of integers.

Exercise 3 - Capacity vs length

import sys
lst = []
prev = sys.getsizeof(lst)
sizes = []
for i in range(1001):
    lst.append(i)
    s = sys.getsizeof(lst)
    if s != prev:
        sizes.append((len(lst), s))
        prev = s
print(sizes[:8])
print(f"growth points up to N=1000: {len(sizes)}")

[(1, 88), (5, 120), (9, 184), (17, 248), (25, 312), (33, 376), (41, 472), (53, 568)]
growth points up to N=1000: 28

list over-allocates and re-allocates in chunks, like Rust’s Vec::push. The growth pattern (currently ~1.125 × capacity) is a CPython implementation detail - different versions and pypy/micropython will pick different multipliers. The principle is identical to Vec: amortised O(1) push, occasional copy. The takeaway is the same as in any growable container: if you know the final size, pre-allocate (np.zeros(N, ...) for numpy; [None] * N then assign for lists) instead of pushing.

Exercise 4 - Run the §3 exhibit

uv run code/measurement/aos_vs_soa_footprint.py

Source: code/measurement/aos_vs_soa_footprint.py. N=1,000,000 rows, K=10 ints per row, values past the small-int cache, each layout in a fresh subprocess:

Ratios vs layout 5 (numpy SoA), 3-run medians. The memory column is stable run-to-run; build and sum are timing-noisy (±~50%), so they are round figures:

  layout                                    memory   build   sum-c0
  ------------------------------------------------------------------
  1. list of tuples       (AoS)              4.7×     ~10×     ~85×
  2. list of lists        (AoS)              5.3×     ~10×     ~82×
  3. tuple of lists       (SoA stdlib)       4.1×      ~7×      ~8×
  4. tuple of array.array (SoA typed)        0.8×     ~10×     ~27×
  5. tuple of numpy arrays (SoA numpy)       1.0×      1.0×     1.0×

The five rows separate three independent wins:

• AoS → SoA (1/2 → 3): the speed flip. ~12% storage win, 10× speedup on column-sum. Walking one contiguous list of pointers beats walking N tuples and dereferencing through each one to reach row[0]. No numpy required.
• SoA-list → SoA-typed (3 → 4): the memory flip. 5× storage win (~383 MB → ~77 MB) from dropping the PyLong boxes. But the sum slows down ~3-4× because Python unboxes each int64 to a PyLong before adding it. Typed storage saves bytes; it does not save the inner loop.
• SoA-typed → SoA-numpy (4 → 5): the C-vectorisation flip. Same bytes, ~27× speedup on the same sum. The loop moves into C; the interpreter is stepped out.

The four-row form of this exhibit collapsed steps 2 and 3 into “use numpy.” The five-row form shows they are separate. Numpy happens to bundle them; array.array lets you take the memory win without the C-loop win, which is sometimes the right trade for a project that wants stdlib-only deps.

Exercise 5 - The dict trap

import time, random, numpy as np

d   = {i: i*i for i in range(1_000_000)}
arr = np.arange(1_000_000) ** 2
idx = np.array([random.randrange(1_000_000) for _ in range(100_000)])

t0 = time.perf_counter()
for k in idx: d[int(k)]
t1 = time.perf_counter()
arr[idx]
t2 = time.perf_counter()

print(f"dict 100K lookups:  {(t1-t0)*1000:.1f} ms")
print(f"numpy 100K gather:  {(t2-t1)*1000:.2f} ms")
print(f"ratio: {(t1-t0)/(t2-t1):.0f}×")

dict 100K lookups:  34.6 ms
numpy 100K gather:   0.75 ms
ratio: 46×

Both look up “by integer.” The dict pays a hash, a probe, and a PyObject* dereference per access - all in pure Python. The numpy gather is one indirection through a typed buffer in C. Same operation, 46× cost gap. When the keys are dense integers, dicts are not the right tool - the only thing they buy you is sparse indexing, and a dense column gets you indexing for free.

Exercise 6 - swap-remove vs remove

import time
lst1 = list(range(1_000_000))
t0 = time.perf_counter()
for _ in range(100):
    lst1.pop(500_000)
t1 = time.perf_counter()
print(f"100 pop(middle):    {(t1-t0)*1000:.2f} ms")

lst2 = list(range(1_000_000))
t0 = time.perf_counter()
for _ in range(100):
    i = 500_000
    lst2[i] = lst2[-1]; lst2.pop()
t1 = time.perf_counter()
print(f"100 swap_remove:    {(t1-t0)*1000:.3f} ms")

100 pop(middle):    3.9 ms
100 swap_remove:    0.019 ms

~200× difference. lst.pop(i) for i in the middle costs O(N) because every element after i shifts down one slot; 100 mid-list pops at N=1M is ~50M element moves. The swap-remove pattern is O(1): overwrite the gap with the last element, then truncate. It changes the order of remaining elements, which is fine wherever order doesn’t carry meaning. §21 builds the rest of the discipline around it.

Exercise 7 - Read your own array

>>> import numpy as np
>>> a = np.arange(10, dtype=np.int64)
>>> raw = a.tobytes()
>>> len(raw)                                  # 80 - exactly 10 × 8 bytes
>>> b = np.frombuffer(raw, dtype=np.int64)
>>> (a == b).all()                            # True

The bytes you would write to disk are the bytes already in memory. There is no serialization step. A typed numpy column is its own on-disk format up to byte-order and dtype. §36 builds on this directly: the persistence layer stores (N, dtype, raw_bytes) and round-trips losslessly, with no encoder/decoder pair to maintain.

4 - Cost is layout - and you have a budget

Concept node: see the DAG and glossary entry 4.

A system is not handed a target rate; it chooses one. Work arrives as a stream - frames to draw, packets to route, sensor samples to fold in, requests to answer - and the only real decision is how finely to cut that stream into batches. Each batch is one tick, and the rate is the grain of the cut. Cut at one operation per tick and nothing batches: every operation carries its fixed overhead alone, and at a 1 GHz tick the budget is a few nanoseconds, too little to work in. Cut at one tick for all pending work and efficiency is maximal but nothing is answered until everything is: a tick a minute has no perceptible responsiveness. Every useful rate sits between those ends, balancing responsiveness against the efficiency of batching.

Two different things bound that band. Whether you can keep up at all is fixed by the per-item cost against the arrival rate: if work lands at rate λ and each item costs c, you survive only when λ · c ≤ 1, and c is a layout fact (§3), not a scheduling one. The rate itself is the second, separate choice: a faster tick means smaller batches and lower latency with less to amortise each fixed cost over; a slower tick means larger batches, better amortisation, and more latency. Batching only ever pays because there are fixed costs to spread - a dispatch, a cache warmup, a syscall, a kernel launch - which is the same amortisation §8 names over data, here run along time.

The responsiveness floor is set by whoever consumes the output. Roughly 24 to 30 frames a second is where discrete frames read as continuous motion for passive viewing, which is why film sits there; interactive rendering wants 60, and head-mounted VR wants 90 to 120 to stay comfortable. A control loop runs as fast as its plant needs a correction, often 1 kHz; an audio loop is pinned to its 48 kHz sample rate; a web request handler answers as fast as a human is willing to wait. Different consumers, different floors, one calculus. The rate you choose is the coarsest batch its floor will tolerate, and it sets a budget - the time available for one tick of work. What you then spend against that budget is governed by layout, which is where the rest of this chapter goes.

Every operation the program does in one tick spends from that budget. Operations have very different costs. From the numbers you measured in §1:

The bolded row is the one most explanations leave out. Inside a Python for loop, every step pays for PYTHON_NEXT_INSTR, refcount work, PyObject boxing - about 5 ns even when you do nothing. That cost is higher than an L1 read and competitive with an L3 read. It is the dominant fact about pure-Python performance, and it does not appear in any C-style cost table.

Three regimes - and a fourth

A loop is compute-bound when its cost is dominated by arithmetic - typically when the data fits in L1 and the inner work is heavy (dot products, transcendentals, integer divides). It is bandwidth-bound when its cost is dominated by how fast the memory subsystem can deliver bytes - typically when the working set is bigger than L3 but the access pattern is sequential, so the prefetcher can fill lines ahead of demand. It is latency-bound when its cost is dominated by individual memory round-trips - typically when the access pattern is random, so the prefetcher cannot help.

Python adds a fourth: interpreter-bound. From the §1 cache-cliffs exhibit, summing 100 million int64 values cost 4.59 ns per element in a Python list and 0.15 ns per element in a numpy array. The Python list run was not bandwidth-bound, nor latency-bound - the bytes were the same bytes. It was interpreter-bound. The CPU spent most of its cycles inside the bytecode dispatcher and the PyLong arithmetic path, not on the data. The fix is not “buy faster RAM”; the fix is leave pure Python for the inner loop.

The four regimes have very different time budgets:

A loop processing 1,000,000 entities in a 30 Hz tick costs 0.6% of the budget if it is bandwidth-bound, 36% if it is latency-bound, and 14% if it is interpreter-bound. The same algorithm, the same data, four ways of running it, four orders of magnitude apart. Complexity-class reasoning cannot tell these regimes apart.

Cost is layout, not just complexity

The same algorithm that costs 0.2 ms on a sequential numpy column may cost 27 ms on a list-of-tuples carrying the same data, because every row read is a pointer chase to a separately allocated tuple, and every column read inside the row is another pointer chase to a PyLong. From the §3 exhibit, summing column 0 of one million ten-int rows took 30 ms as a list of tuples and 0.4 ms as a numpy SoA - a 75× spread on the same payload. Two programs with the same big-O, same input data, and the same machine differ by almost two orders of magnitude on the inner loop, just because of where their data sits.

This gives you a design rule. Decide your target rate before you decide anything else. That sets the budget. Then when you choose data structures, ask whether the resulting working set fits in cache; ask how many memory loads per row your inner loop does; ask whether any single operation in the loop dominates the budget; ask whether you are running inside the interpreter or outside it. Most decisions become forced once the budget is named.

The reverse direction is also useful. If you find yourself wanting to add something to the inner loop - a dictionary lookup, a getattr against a class, a Python-level callback, an exception handler - count its cost in microseconds against the budget. Often the answer is “this single addition uses 80% of my tick”, and the right move is not to optimise it but to lift it out of the inner loop entirely.

The engineering analogy

The shape of this thinking is familiar to engineers in other domains. An electrical engineer designs a circuit by counting milliamps against a current budget. A structural engineer counts kilonewtons against a load budget. The data-oriented programmer counts microseconds against a tick budget. Good design is measured in millivolts and microamps - and in nanoseconds and microseconds. Pick the unit, write the budget down, count against it. Programming has no special exemption from accounting.

Note - Time is one budget. Power is another. Cache hits are energetically nearly free - the data is already next to the arithmetic units. Cache misses fire up the memory controller, the bus drivers, sometimes a DRAM refresh; that is where the watts go. A loop that fits in L2 spends most of its time on cheap arithmetic; a loop that pointer-chases through RAM spends most of its time waiting, and during the waiting the CPU drops clocks and the chip stays cool. The same SoA-and-sequential-access discipline that fits the time budget also fits a power budget. For embedded, mobile, control, and battery-powered work, power is the primary budget; time is downstream of it. The “millivolts and microamps” line above is literal, not metaphor. One Python-specific addendum: an interpreter-bound loop is also relatively power-hungry per useful operation, because the CPU is running flat-out doing dispatch work instead of arithmetic. Moving to numpy improves time and energy at the same time. There is no trade-off here - the disciplined choice is also the cheap one.

The budget is a curve, not a cliff

So far the budget has been a single number: name the rate, get the time per tick. But the work in a tick is rarely fixed. It grows with the problem - more entities to step, more rows to fold - and if the per-item cost holds, the tick time grows with it. So the rate you can actually sustain is not a constant either; it falls as the work rises, roughly as one over the size for a loop that costs O(N). Thirty hertz is not a wall you meet at some population and shatter against. It is one point on a slope that reads thirty, twenty-five, twenty, fifteen as the work climbs.

That moves the engineering question. It is seldom “does it hit thirty hertz” and almost always “where does the curve fall, and is that fall tolerable”. A control loop specified at thirty may be well served by twenty under a heavier load; a visualisation at fifteen is still watchable. So the useful design conversation names two numbers, not one: the target rate, and the tolerance - the slowest rate the consumer will accept - then reads off the scale at which the curve crosses the tolerance. You characterise the budget around the target instead of slamming into it.

In Python the slope is the same shape but it crosses sooner. An interpreter-bound tick pays its ~5 ns per element before it does any useful work, so the same O(N) loop reaches the budget at a population a compiled or vectorised version would shrug off. The cure is the one this chapter keeps naming - leave the interpreter for the inner loop - which does not change the shape of the curve; it slides the crossing rightward to a larger N. The simulator puts numbers on this slope once it runs at scale.

Exercises

1. Pick your rates. For each of these systems, name a plausible target rate and the resulting per-tick budget: a card game; a real-time strategy game; a market data feed; an embedded sensor controller; a web API endpoint a user is waiting for; an offline batch job that processes a billion rows.

2. Count an operation. Time a single dict[k] lookup on a dict of 1,000,000 entries (use timeit for a million repeats and divide). Note its cost in microseconds. How many can you fit in a 30 Hz tick (33 ms)? In a 1 kHz tick (1 ms)?

3. The layout difference. Sum 1,000,000 int64 values in a numpy array. Sum 1,000,000 ints in a Python dict with integer keys (use sum(d.values())). What is the per-element time difference (in nanoseconds)? Where did it go? Map the answer back to the regime table above.

4. The cliff. With your numbers from §1 exercise 2, pick a numpy array size that just fits in L2 and one that just doesn’t. Time a arr.sum() at each size. The cliff is real.

5. Working backwards from the budget. You target 60 Hz; your inner loop runs over 100,000 entities; each entity touches one cache line of state. Estimate the cost of the loop in microseconds in each of the four regimes (compute, bandwidth, latency, interpreter). Compare to your 60 Hz budget (16,666 µs). Note which regime gives you headroom and which blows the budget.

6. A bad design. Construct a Python design that is “obviously fast” by big-O reasoning but blows the 30 Hz budget on a million entities. (Hint: list of dataclass instances with a per-tick for entity in entities: entity.update() is the canonical example. Estimate its cost from the interpreter-bound row of the regime table.)

7. Find your CPU’s TDP. Look up your CPU’s rated thermal design power on the manufacturer’s spec sheet, or read it locally on Linux with sudo dmidecode -t processor | grep -i 'power\|TDP'. Note the value. TDP is what the chip can dissipate sustained without thermal throttling - burst can be 1.5-2× higher for tens of seconds; sustained settles back to TDP.

8. Battery budget. A typical laptop battery holds about 50 Wh. Your simulator runs at 30 Hz and draws an average of 8 W (mostly memory bandwidth on the inner loop). How many hours of simulation does a full charge buy? If a layout change pushes more loads to RAM and raises the average draw to 14 W, how many hours then? Express the cost of the layout change as a percentage of battery life.

9. Measure delta power. In one terminal, run a sustained sequential numpy sum loop:

   import numpy as np
   arr = np.arange(10_000_000, dtype=np.int64)
   while True: _ = int(arr.sum())
   

   In another terminal: sudo perf stat -a -e power/energy-pkg/ -- sleep 30 reads the package-energy counter over 30 seconds. Run the same measurement with a random gather version (arr[idx].sum() with a shuffled idx) and an idle baseline. Convert each to average watts. The random-access run should draw more watts than the sequential one, which should draw more than idle. The gap between them is the energy cost of breaking the prefetcher.

10. (stretch) Joules per access. Approximate energies per memory read: L1 hit ≈ 0.1 nJ, L2 ≈ 1 nJ, RAM ≈ 30 nJ (rough; published numbers vary by chip and process). Estimate the total energy of summing 10 million int64s sequentially (mostly prefetched, near-L1 cost) versus by random indices (mostly RAM misses). Convert both to milliwatt-hours and express as a fraction of a 50 Wh battery. The absolute numbers are tiny; the ratio is what your battery life and your data-centre electricity bill care about.

11. The budget is a curve. Take the loop from exercise 5 (100,000 entities, one cache line each, 60 Hz). Hold the per-entity cost fixed at the interpreter-bound ~5 ns/element from the regime table and sweep the count: 100,000, 300,000, 1,000,000, 3,000,000. Compute the tick time and the sustainable rate at each. At what size does the rate cross 30 Hz? 15 Hz? Plot rate against size and confirm the one-over-N shape. Then recompute the crossing scales for the bandwidth-bound (numpy) regime, and note how far leaving the interpreter slides the curve rightward.

Reference notes in 04_cost_and_budget_solutions.md.

What’s next

You now have the machine model (§1), the data widths (§2), the table primitive (§3), and the budget calculus (§4). The next section is the conceptual heart of the book: §5 - Identity is an integer. The card game is waiting.

Solutions: 4 - Cost is layout, and you have a budget

Exercise 1 - Pick your rates

The point of writing these down is that “this should be fast” is not a budget. “33 ms” is. The instant you have a number, every line of code in the inner loop is either spending bytes of that budget or it isn’t.

Exercise 2 - Count an operation

import timeit
d = {i: i*i for i in range(1_000_000)}
t = timeit.timeit("d[42]", globals={"d": d}, number=10_000_000)
print(f"dict[k] lookup: {t/10_000_000*1e9:.1f} ns")

dict[k] lookup: 15.1 ns

At 30 Hz (33 ms): ~2.2 million lookups per tick. At 1 kHz (1 ms): ~66,000 lookups per tick.

A 1-million-entity update that does one dict lookup per entity would cost 15 ms - half a 30 Hz budget on a single bookkeeping op. Two dict lookups per entity blows the budget on bookkeeping alone, with no actual simulation work done yet.

Exercise 3 - The layout difference

import time, numpy as np
n = 1_000_000
arr = np.arange(n, dtype=np.int64)
d   = {i: i for i in range(n)}
arr.sum(); sum(d.values())                     # warmup

t0 = time.perf_counter(); int(arr.sum());    t1 = time.perf_counter()
t2 = time.perf_counter(); sum(d.values());   t3 = time.perf_counter()

print(f"numpy sum:        {(t1-t0)*1e9/n:.2f} ns/elem")
print(f"sum(d.values()):  {(t3-t2)*1e9/n:.1f} ns/elem")

numpy sum:        0.20 ns/elem
sum(d.values()):  3.6 ns/elem
ratio: 18×

The dict version is interpreter-bound: the inner loop is a pure-Python for v in values: total += v, which pays bytecode dispatch + PyLong arithmetic + refcount work per element - about 3-6 ns. The numpy version is bandwidth-bound: a tight C loop reading int64s sequentially, the prefetcher loaded ahead, the L1 line warm. Same 1M int64 payload, two regimes apart, 18× cost gap.

Exercise 4 - The cliff

import time, numpy as np
for size in [100_000, 200_000, 1_000_000, 10_000_000]:
    a = np.ones(size, dtype=np.int64)
    a.sum()                                       # warmup
    best = float("inf")
    for _ in range(3):
        t0 = time.perf_counter(); a.sum(); t1 = time.perf_counter()
        if t1 - t0 < best: best = t1 - t0
    print(f"  N={size:>10,} ({size*8/1024:>7.0f} KB): {best*1e9/size:.2f} ns/elem")

  N=   100,000 (    781 KB):  0.11 ns/elem
  N=   200,000 (   1562 KB):  0.12 ns/elem
  N= 1,000,000 (   7812 KB):  0.11 ns/elem
  N=10,000,000 (  78125 KB):  0.20 ns/elem

On this machine the cliff between L2-fitting (200 KB - 1 MB) and L3-spilling (10 MB+) is shallow on the sequential sum (0.11 → 0.20 ns/elem, less than 2× slowdown). The prefetcher is doing its job: even with the working set in RAM, sequential-access numpy hovers near memory bandwidth limits. The dramatic cliff is on the gather column from §1; sequential numpy is forgiving.

This is why the chapter distinguishes bandwidth-bound from latency-bound: same N, same array, very different cliff depending on access pattern. The cliff exists; sequential numpy hides most of it.

Exercise 5 - Working backwards from the budget

Target 60 Hz (16.67 ms = 16,666 µs); 100,000 entities; one cache line touched per entity.

100K is small enough that even a Python-loop version fits comfortably. Scale to 10M:

At 10M entities, latency-bound and interpreter-bound layouts blow the budget by 3-7×. Bandwidth-bound finishes with 88% headroom. Same algorithm, same data, same machine.

Exercise 6 - A bad design

from dataclasses import dataclass

@dataclass
class Entity:
    x: float
    y: float
    vx: float
    vy: float

entities = [Entity(0.0, 0.0, 0.1, 0.1) for _ in range(1_000_000)]

# per tick:
for e in entities:
    e.x += e.vx
    e.y += e.vy

This is the canonical “obviously fast” Python design. Big-O is O(N); the inner work is two floating-point adds. Estimating from the regime table: interpreter-bound at ~5 ns × 4 attribute touches ≈ 20 ns/entity × 1M = 20 ms per tick, ~60% of a 30 Hz budget on simulation work alone. The exhibit tick_budget.py confirms this empirically:

  1,000,000  Python dataclass list      27.525 ms     30 Hz: 82.6%     60 Hz: 165% OVER
  1,000,000  numpy SoA                   0.278 ms     30 Hz:  0.8%     60 Hz:  1.7%

100× cost gap. The big-O is the same. The constant factor - the per-element interpreter dispatch through four attribute accesses on a heap-allocated dataclass - is what blows the budget.

Exercise 7 - Find your CPU’s TDP

Linux:

sudo dmidecode -t processor | grep -i 'power\|TDP'

Or look up the CPU model on the manufacturer’s spec sheet (Intel ARK, AMD product page, Apple silicon spec). Typical 2026 figures:

Burst can run 1.5-2× higher for tens of seconds; sustained settles back to TDP. The number matters because it’s the ceiling for energy per tick on your machine - useful when budgeting battery life or cooling.

Exercise 8 - Battery budget

50 Wh laptop battery, simulator at 30 Hz:

• 8 W draw: 6.25 hours runtime.
• 14 W draw (after a layout change): 3.57 hours runtime.

The layout change cost 2.68 hours, or 43% of battery life. A change that adds memory loads to the inner loop is a change that shortens battery life by nearly half. For mobile, embedded, or any battery-powered work, this matters more than the wall-clock tick time.

Exercise 9 - Measure delta power

# Terminal 1: sustained sequential numpy sum
python3 -c "
import numpy as np
arr = np.arange(10_000_000, dtype=np.int64)
while True: _ = int(arr.sum())
"

# Terminal 2: read package energy over 30 seconds
sudo perf stat -a -e power/energy-pkg/ -- sleep 30

Repeat for the random-gather version (arr[idx].sum() with shuffled idx) and for an idle baseline. Convert each to average watts (J/30s = W).

Expected ordering: idle < sequential < gather. The gap between sequential and gather is the energy cost of breaking the prefetcher - same arithmetic, same data volume, but more memory-controller and DRAM activity per useful operation.

This exercise needs root for perf access to RAPL counters, and works on x86 Linux. On macOS, powermetrics is the analog. On bare-metal embedded, an external power meter is the honest answer.

Exercise 10 - Joules per access

Approximate energies per memory read:

For 10M int64 reads:

• Sequential (mostly prefetched): assume mostly L1-equivalent cost. 10⁷ × 0.1 nJ = 1 mJ.
• Random gather (mostly RAM misses): 10⁷ × 30 nJ = 300 mJ.

300× more energy. Convert: 1 mJ = 0.28 µWh; 300 mJ = 83 µWh. As a fraction of a 50 Wh battery: 5.6 × 10⁻⁹ vs 1.7 × 10⁻⁶ - both tiny in absolute terms. The ratio is what compounds across millions of ticks per day across millions of laptops, or across the lifetime power bill of a data centre. The disciplined layout is also the cheap one, twice over: faster and cooler per useful operation.

Exercise 11 - The budget is a curve

Hold the per-element cost fixed and the tick time is linear in the count: tick = c · N, so the sustainable rate is 1 / (c · N) - it falls as one over N, a hyperbola. At the interpreter-bound c ≈ 5 ns:

The rate crosses 30 Hz at 33.3 ms / 5 ns ≈ 6.7M entities and 15 Hz at ≈ 13M. In the bandwidth-bound numpy regime (c ≈ 0.2 ns) the same crossings move out to ~166M and ~333M: leaving the interpreter slides the curve roughly 25× rightward without changing its shape. That is the Python lesson in one number - the curve is the same hyperbola whatever the regime; the regime decides how far out it crosses.

Two honest caveats. The per-element cost here is a single bookkeeping loop; the simulator’s real tick does far more per entity (a forage query, motion, the lifecycle), so its crossing sits at a much smaller N - hundreds of thousands, not millions. And c is your machine’s, not a universal constant. The shape is the durable claim; the crossing scale is something you measure.

5 - Identity is an integer

Concept node: see the DAG and glossary entry 5.

Hand a Python programmer fifty-two cards and tell them to write code that shuffles, sorts, and deals. Ask how long.

Most will start drawing classes. The “official” Python tutorial path leads here: define class Card with __init__(self, suit, rank), then class Deck holding a list[Card], then class Hand, then probably class Player and class Game. By the time the type hints are right and the __repr__ methods print nicely, an evening has passed. There will be debates about whether Hand should contain Card instances or hold references to a shared Deck, whether Deck.shuffle() should mutate or return a new deck, whether Card should be a @dataclass(frozen=True) for hashability. None of these debates are wrong; all of them are work that has nothing to do with cards.

The whole problem fits in three lines of numpy. The way it fits is the lesson of this section.

A deck of cards has three pieces of information per card: its suit (♠ ♥ ♦ ♣), its rank (A, 2, …, K), and its current location (in the deck, in someone’s hand, in the discard pile). That is three columns. The deck itself is fifty-two rows.

import numpy as np

suits     = np.zeros(52, dtype=np.uint8)  # 0..3
ranks     = np.zeros(52, dtype=np.uint8)  # 0..12
locations = np.zeros(52, dtype=np.uint8)  # 0=deck, 1..N=hands, 255=discard

That is the deck. The whole thing is 156 bytes - three contiguous columns of 52 unsigned bytes. There is no Card class. There is no Deck class. The card at index 17 has its suit at suits[17], its rank at ranks[17], and its current location at locations[17]. The card is the index.

Filling the columns with a fresh, ordered deck is one assignment per column:

suits[:] = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks[:] = np.tile(np.arange(13, dtype=np.uint8), 4)
locations[:] = 0

Dealing card 17 to player 1 is one element write:

locations[17] = 1

Asking what’s in player 1’s hand is one numpy primitive:

hand = np.where(locations == 1)[0]

hand is a numpy array of indices into the deck - a list of card identities - not a copy of any card data. Asking how many cards are in each location is also one primitive:

counts = np.bincount(locations, minlength=2)  # counts[0] = deck, counts[1] = player 1, ...

Shuffling - the move students expect to be hard - is shuffling the order of indices. 0..52 becomes [7, 32, 1, 19, ...], and you read your way through the cards in that order:

order = np.random.permutation(52)

Look at what just happened. Nothing about the cards changed. suits[17], ranks[17], and locations[17] are exactly the values they were before. The shuffle moved indices, not data.

Sorting works the same way. To sort by suit then rank, you sort the indices by (suits[i], ranks[i]):

order = np.lexsort((ranks, suits))  # last key is primary; sort by suit first, then rank

The cards do not move. Their identifiers are reordered.

That’s the deck of cards in maybe fifteen lines of Python. It includes shuffle, sort, deal, and several queries. It is not a stylistic shortcut; it is what a deck of cards is. The class-hierarchy version’s evening of work was the cost of pretending a card was an object that owned its suit and rank, when actually a card is one number - an index - and its suit and rank are values stored in arrays at that index.

We call this identity-is-an-integer, and it is the precondition for every economy the rest of this book buys you. Persistence will work because tables are easy to serialise - three np.save calls. Parallelism will work because indices are cheap to partition. Replay will work because a deck is just three arrays in a state. None of it works if you reach for class Card.

Even which integer matters

Not every integer is the same integer for performance. From code/measurement/float_or_int_tuple.py, looking up keys in a Python dict of 10,000 entries:

A two-tuple of ints hashes and compares 2.4× faster than a three-tuple of floats. Identity-is-an-integer is not just “use a number”; it is “use a small unsigned integer, ideally in a contiguous typed array.” A np.uint8 index packs 64 to a cache line and hashes in one CPU instruction. A (float, float, float) “identity” - the kind a Python tutorial might suggest for a 3D point in a dict - pays the price three times: more bytes, slower hash, slower compare.

The card-deck columns above use np.uint8 deliberately: 0..255 covers everything (4 suits, 13 ranks, up to 254 locations), one byte per value, 64 cards per cache line. The width budget from §2 meets the identity choice from §5: a np.uint8 column is the cheapest possible identity, the cheapest possible storage, and the cheapest possible lookup, all in one decision.

Note - The strong form, which we will return to later: sometimes you do not even need the index. The pair (suit, rank) already uniquely identifies a playing card - there are only fifty-two such pairs. The index is a surrogate key; the pair is a natural key. For variable-quantity tables (creatures that come and go) you usually need a surrogate, because two creatures can be identical. For a constant-quantity 52-card deck, you do not. The book uses surrogates throughout because the simulator is variable-quantity, but knowing when you can drop the index is its own discipline.

Exercises

The first time through, write everything from scratch in deck.py. Resist the urge to add a Card class or helper methods. Three numpy arrays.

1. Build the deck. Write def new_deck() -> tuple[np.ndarray, np.ndarray, np.ndarray] that returns the suits, ranks, and locations for a fresh, ordered deck (all 52 in location 0 = deck). All three arrays are dtype=np.uint8.
2. Print a card. Write def card_to_string(suit: int, rank: int) -> str that returns strings like "A♠", "10♥", "K♦". Use it to print the whole deck.
3. Shuffle. Use np.random.default_rng(seed).permutation(52) to produce a shuffled order. Print the deck in shuffled order. Confirm by inspection that the suits, ranks, and locations arrays are unchanged.
4. Sort by suit then rank. Use np.lexsort((ranks, suits)) to produce an order such that suits come out grouped, ranks ascending within each suit. Print again. Once again, the deck arrays are unchanged.
5. Deal a hand. Move the first 5 cards from the deck (location 0) to player 1 (location 1). Print player 1’s hand using card_to_string.
6. Hand query. Write def cards_held_by(locations: np.ndarray, player: int) -> np.ndarray returning all card indices currently held by a given player. The body is one line.
7. Count by location. Write a function that returns counts grouped by location using np.bincount. Confirm counts[0] + counts[1:].sum() == 52.
8. Deal four hands. Deal 5 cards to each of players 1, 2, 3, 4. Print all four hands.
9. (stretch) Drop the index. Rewrite cards_held_by to return an (N, 2) numpy array of (suit, rank) pairs directly - no indices. What does this make easier? What does it make harder? (Hint: you cannot move the cards back to the deck without knowing which i they were.)
10. (stretch) The sort hazard. While player 1 is holding indices [3, 17, 21, 28, 41], sort the deck arrays themselves in place by suit (order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]). What does player 1 think they hold now? Print the cards at the indices [3, 17, 21, 28, 41] after the sort. This is the bug §9 - sort breaks indices was written for. Don’t fix it yet - observe it.

Reference solutions for exercises 1-3 in 05_identity_is_an_integer_solutions.md. Solutions for the rest follow the same shape.

What’s next

Exercise 10 leaves you with a bug. The next several sections build the discipline that prevents it: §6 - A row is a tuple is the next vocabulary lesson, and §9 - sort breaks indices is the fix - keep a stable id alongside the position so external references survive reordering.

Solutions: 5 - Identity is an integer

The exercises ask you to write three columns and a handful of small functions. The whole deck - shuffle, sort, deal, query - fits in about 50 lines. No Card, no Deck, no Hand.

Exercise 1 - Build the deck

import numpy as np

def new_deck() -> tuple[np.ndarray, np.ndarray, np.ndarray]:
    suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)   # 0,0,...,1,1,...,3,3
    ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)     # 0,1,..,12,0,1,..,12,...
    locations = np.zeros(52, dtype=np.uint8)                  # all in 'deck' (=0)
    return suits, ranks, locations

Total bytes: 156. The deck is three contiguous arrays of 52 unsigned bytes.

Exercise 2 - Print a card

SUIT = ['♠', '♥', '♦', '♣']
RANK = ['A','2','3','4','5','6','7','8','9','10','J','Q','K']

def card_to_string(suit: int, rank: int) -> str:
    return f"{RANK[rank]}{SUIT[suit]}"

suits, ranks, _ = new_deck()
for i in range(52):
    print(card_to_string(suits[i], ranks[i]))

The string-rendering layer is outside the deck. It looks up into two small lookup tables. The deck itself never deals in symbols.

Exercise 3 - Shuffle

rng = np.random.default_rng(seed=42)
order = rng.permutation(52)
for i in range(52):
    j = order[i]
    print(card_to_string(suits[j], ranks[j]))

order is a permutation of [0, 1, ..., 51]. Reading the deck through order reads the cards in shuffled order. suits and ranks are byte-for-byte unchanged after the shuffle - (suits == new_deck()[0]).all() is True. The shuffle moved indices, not data.

Exercise 4 - Sort by suit then rank

order = np.lexsort((ranks, suits))   # last key is primary; suit groups, ranks ascending within
for i in range(52):
    j = order[i]
    print(card_to_string(suits[j], ranks[j]))

np.lexsort returns indices that would sort by the keys (last key dominates). (ranks, suits) means: primary sort by suit, secondary by rank. Once again, suits and ranks are unchanged.

Exercise 5 - Deal a hand

locations[:5] = 1                                   # first 5 cards → player 1
hand = np.where(locations == 1)[0]                  # indices held by player 1
for i in hand:
    print(card_to_string(suits[i], ranks[i]))

One element write per card moved. The card data does not move; only the location markers change.

Exercise 6 - Hand query

def cards_held_by(locations: np.ndarray, player: int) -> np.ndarray:
    return np.where(locations == player)[0]

One line. Returns indices, not card data. The caller looks up the card data through those indices.

Exercise 7 - Count by location

def location_counts(locations: np.ndarray) -> np.ndarray:
    return np.bincount(locations, minlength=2)

counts = location_counts(locations)
assert counts[0] + counts[1:].sum() == 52
print(f"in deck: {counts[0]}, in hands: {counts[1:].sum()}")

np.bincount is the right primitive for “count by integer category” - one C-level pass over the locations array. For 52 cards the cost is negligible; the same primitive scales to 100M creatures with hunger states without changing shape.

Exercise 8 - Deal four hands

suits, ranks, locations = new_deck()
order = rng.permutation(52)

for player in range(1, 5):
    take = order[(player - 1) * 5 : player * 5]
    locations[take] = player

for player in range(1, 5):
    hand = cards_held_by(locations, player)
    cards = [card_to_string(suits[i], ranks[i]) for i in hand]
    print(f"player {player}: {cards}")

Twenty cards dealt; four arithmetic slices into a permutation; one assignment per slice. No object construction, no per-card branching.

Exercise 9 - Drop the index (stretch)

def cards_held_by_pairs(suits: np.ndarray, ranks: np.ndarray,
                        locations: np.ndarray, player: int) -> np.ndarray:
    mask = locations == player
    return np.column_stack([suits[mask], ranks[mask]])     # shape (N, 2)

What this makes easier: returning a self-contained snapshot of the hand. The caller can inspect (suit, rank) without holding a reference to the deck arrays. For constant-quantity tables (a 52-card deck never grows), this is fine.

What it makes harder: putting the cards back. To move a card from a hand to the discard pile you need to know the index, not the value - there are 52 distinct cards but no general way to invert from (suit, rank) to “which row in the deck arrays held this.” For variable-quantity tables (creatures that are born and die), the index is what survives mutations to the table; the (suit, rank) “natural key” is brittle to anything that adds rows.

The book uses indices throughout because the simulator is variable-quantity. For constant-quantity domain (a fixed 52-card deck), dropping the index is a real option.

Exercise 10 - The sort hazard (stretch)

import numpy as np
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)

# Shuffle the arrays in place so positions are non-trivial
rng = np.random.default_rng(42)
order = rng.permutation(52)
suits[:] = suits[order]
ranks[:] = ranks[order]

# Player 1 holds indices [3, 17, 21, 28, 41]
held = [3, 17, 21, 28, 41]
locations[held] = 1
print("Player 1 holds at indices", held, "→",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# → ['K♦', '3♣', 'A♦', '4♥', 'A♠']

# Now sort the deck arrays in place by suit
order2 = np.argsort(suits, kind='stable')
suits[:]     = suits[order2]
ranks[:]     = ranks[order2]
locations[:] = locations[order2]

print("After in-place sort, player 1 looks at the SAME indices", held, "→",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# → ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Player 1 holds at indices [3, 17, 21, 28, 41] → ['K♦', '3♣', 'A♦', '4♥', 'A♠']
After in-place sort, player 1 looks at the SAME indices [3, 17, 21, 28, 41] → ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Player 1 recorded indices [3, 17, 21, 28, 41] and stashed them somewhere outside the deck arrays. The sort moved cards around. Player 1’s stored indices now point at whichever cards happened to land at those positions. They are not the cards player 1 was holding.

The locations column was reordered alongside suits and ranks, so internally np.where(locations == 1) correctly identifies player 1’s cards at their new positions ([8, 20, 27, 32, 44]). The bug is in the external index list - the one the player code held outside the table. Indices are not stable across reorderings.

This is the bug §9 - sort breaks indices addresses. The fix is to issue every card a stable id (a number that travels with the card across reorderings) and let external code refer to cards by id, not by current position. The deck arrays then carry an id column whose contents are reordered along with the card data; np.where(ids == card_id) finds a card no matter how the rows have been shuffled.

6 - A row is a tuple

Concept node: see the DAG and glossary entry 6.

In §5 you built a deck of 52 cards as three numpy columns. The card at index 17 is the triple (suits[17], ranks[17], locations[17]). Together those three values are the row. There is no Card class. There is not even a tuple object - the row exists implicitly in the alignment: the same index, used in every column, recovers all the data about one card.

This is what we call a row throughout the rest of the book - a coherent set of values that belong to the same entity. In a creature table the row is (pos[i], vel[i], energy[i], birth_t[i], id[i], gen[i]). In a food table it is (pos[i], value[i], id[i]). The fields belong to the same entity by virtue of all sharing index i. There is no dataclass holding them; there is no NamedTuple instance; there is no dict. There is only the discipline that whatever index i you used to read one column, you also use to read every other column of the same table.

Why “implicit” matters in Python

Python’s tutorial reflex when it sees the word row is to reach for a class - @dataclass class Row or class Row(NamedTuple) or, if performance is mentioned, class Row: __slots__ = (...). Each of these constructs the row as an object, with a header, a refcount, and field pointers. None of them are free. From code/measurement/classes_or_tuples.py, the time to materialise 1,000,000 two-field “rows” on this machine, ordered fastest to slowest:

Two readings of this table.

First reading: the bare tuple is ~16× faster than a slotted class and ~23× faster than a frozen+slots dataclass for per-row construction. The named alternatives all pay for an object header and per-field descriptor lookup that the tuple skips. From code/measurement/simple_namespace.py, even a dict ({'x': 10.0, 'y': 20.0}) constructs faster than any of the named-class options - about 0.036 s for the same million. Naming the row is the cost; the tuple is the cheapest row that is still recognisable as a row.

Second reading - and the one this book cares about - is the top line: two bulk numpy column allocations construct 1,000,000 rows-worth of data faster than a million individual tuple literals. Bulk allocation is roughly 30× faster than the named alternatives and is not even slower than the cheapest per-row option. The shape that lets you do this - pre-allocate a column once, fill it with values, and treat row i as the implicit tuple (col0[i], col1[i], ...) - has no per-row construction cost at all. The tuple at index i only exists when you ask for it explicitly; until then it lives in contiguous bytes inside numpy columns. From the §3 footprint exhibit, one million ten-field rows cost 99 MB as numpy SoA columns and 437 MB as a list of tuples - and the SoA version pays zero per-row construction cost on top of that, because there are no row objects.

A row is a tuple, but in Python the most useful version of that statement is: a row is a tuple you do not have to build.

Alignment is the discipline

The cost of implicit binding is that you must keep the indices aligned. If you sort suits without also sorting ranks and locations, the row at every index is corrupted - the deck still has 52 entries in 52 slots, but each slot now holds the suit of one card, the rank of another, the location of a third. This is not a hypothetical bug; you produced it deliberately in §5 exercise 10, and §9 will hand you the structural fix. The rule is simple: every operation that reorders any column of a table must reorder all columns of that table together.

The discipline that makes alignment maintainable is single-writer-per-column. If only one function writes to locations, and that function writes consistently, alignment is never violated. Multiple writers to the same column race against each other and produce inconsistent rows. This is what §25 (one writer, many readers) enforces: each table has exactly one writer, and a row is a tuple precisely because that one writer kept all its columns in step.

A row is a tuple - assembled from columns indexed by the same entity, kept aligned by discipline rather than by any container holding it together.

Exercises

These extend your deck.py from §5.

1. Print row 17. Write def row(suits, ranks, locations, i) returning (int(suits[i]), int(ranks[i]), int(locations[i])). Use it to print the suit, rank, and location of card 17.
2. Mishandle the alignment. Sort only suits in place: suits.sort(). Print row 17 again. The values are now from three different cards - exactly the bug.
3. Lockstep sort. Reset the deck. Now sort all three columns together using an order array: order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]. Print row 17 again. The values are from one card. (The [:] matters - it is an in-place assignment that keeps the same backing array; suits = suits[order] would rebind the name to a new array and break aliases held elsewhere.)
4. Add a fourth column. Add dealt_at = np.full(52, 255, dtype=np.uint8) (when a card is dealt at tick t, write t into dealt_at[i]; the sentinel 255 means “not yet dealt”). Modify your lockstep sort to also reorder this column. Verify by spot-check that a row is still consistent after a sort.
5. The single-writer rule. Write def reorder_deck(suits, ranks, locations, dealt_at, order). This function is the only one that should ever reorder any column of the deck. Document that contract in a docstring above the function. Refactor your shuffle and sort to call it.
6. The construction cost, your machine. Run uv run code/measurement/classes_or_tuples.py on your machine. Note the ratios. Confirm that the slotted-dataclass row, the canonical “right” answer in modern Python, is the slowest of the named options at construction.
7. (stretch) When alignment is moot. A query that uses only (suits[i], ranks[i]) to identify a card - for instance, “is this the Ace of Spades?” - does not depend on locations or dealt_at. Write such a query (one line, using np.where). The natural-key view from §5’s strong form means this query survives reorderings of unrelated columns; only suits and ranks need to be aligned with each other.

Reference notes in 06_a_row_is_a_tuple_solutions.md.

What’s next

§7 - Structure of arrays (SoA) names the layout choice you have been making implicitly: each field its own column. The next section defends that choice against its alternative.

Solutions: 6 - A row is a tuple

These exercises extend the deck.py from §5. They demonstrate one rule: every operation that reorders any column must reorder all columns together.

Exercise 1 - Print row 17

def row(suits, ranks, locations, i):
    return (int(suits[i]), int(ranks[i]), int(locations[i]))

print(row(suits, ranks, locations, 17))
# (1, 4, 0)   - card 17 is suit 1 (♥), rank 4 (5), in deck (0)

The row is the implicit tuple (col0[i], col1[i], col2[i]). Casting to int strips the numpy dtype wrapper for cleaner printing - the underlying data is unchanged.

Exercise 2 - Mishandle the alignment

suits.sort()                                         # sorts only `suits`
print(row(suits, ranks, locations, 17))
# (1, 4, 0)  - but the (1, ...) is now from one card and (4, 0) from another

After suits.sort(), position 17 contains the 17th-smallest suit value but ranks[17] and locations[17] still hold the rank and location of whichever card originally sat at index 17. Row 17 is now a Frankenstein composite of three different cards. Reading any row gives nonsense; the per-column data is internally consistent, but the table no longer has rows.

Exercise 3 - Lockstep sort

suits, ranks, locations = new_deck()                 # reset

order = np.argsort(suits, kind='stable')             # one permutation, used for all
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

print(row(suits, ranks, locations, 17))
# (1, 4, 0)  - values from one card again

A single order array, applied identically to every column, preserves alignment. The row at any new index is still a coherent tuple from one card.

The [:] matters. suits = suits[order] rebinds the local name suits to a new array; any other code holding the original suits array (a function parameter, an attribute, an element of a tuple) keeps the unsorted array. suits[:] = suits[order] writes through the existing buffer, so all aliases see the sort. Aliasing pitfalls live or die on the difference.

Exercise 4 - Add a fourth column

suits, ranks, locations = new_deck()
dealt_at = np.full(52, 255, dtype=np.uint8)          # 255 = not yet dealt

# example: deal card 17 at tick 7
locations[17] = 1
dealt_at[17]  = 7

# lockstep sort, now over four columns
order = np.argsort(suits, kind='stable')
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]
dealt_at[:]  = dealt_at[order]

# spot-check: find where card 17 ended up via dealt_at = 7
moved_to = int(np.where(dealt_at == 7)[0][0])
print(row(suits, ranks, locations, moved_to), dealt_at[moved_to])
# (1, 4, 1) 7   - same card, new index, all four columns aligned

Adding a column adds one line to every place that reorders the table. That repetition is exactly what the next exercise factors out.

Exercise 5 - The single-writer rule

def reorder_deck(suits, ranks, locations, dealt_at, order):
    """The ONLY function permitted to reorder any column of the deck.

    Applies `order` (a permutation array) identically to every column,
    in place, so external references to these arrays continue to see
    aligned rows.
    """
    suits[:]     = suits[order]
    ranks[:]     = ranks[order]
    locations[:] = locations[order]
    dealt_at[:]  = dealt_at[order]


def shuffle(suits, ranks, locations, dealt_at, rng):
    reorder_deck(suits, ranks, locations, dealt_at,
                 rng.permutation(len(suits)))


def sort_by_suit_then_rank(suits, ranks, locations, dealt_at):
    reorder_deck(suits, ranks, locations, dealt_at,
                 np.lexsort((ranks, suits)))

The contract is in the docstring; future-you (or any other reader) sees in one place what every reordering must do. Adding a fifth column means editing one function. Forgetting to update one column at the call site stops being possible - there is only one call site.

This is the §25 ownership-of-tables discipline applied at the smallest scale: one writer per column, one reorder function per table.

Exercise 6 - The construction cost, your machine

uv run code/measurement/classes_or_tuples.py

Source: code/measurement/classes_or_tuples.py. One million two-field rows, ordered fastest to slowest:

0.004 s  numpy SoA: two np.full(1_000_000, 10.0) calls (bulk)
0.011 s  bare tuple (10.0, 20.0) × 1M individual constructions
0.117 s  class with __slots__
0.157 s  typing.NamedTuple subclass
0.167 s  collections.namedtuple
0.178 s  @dataclass

Two readings:

• The slotted dataclass - the canonical “right” answer in modern Python - is the slowest of the named options. The slots win is real but small (it removes the per-instance __dict__); the dataclass overhead at construction (descriptor lookup, __init__ call) dominates.
• Bulk numpy column allocation finishes 1M rows-worth of data in 3 ms, half the time of a million bare-tuple constructions. The shape with no per-row construction cost is the cheapest shape even when measured against the cheapest per-row option.

A row is a tuple. The most useful version of that statement is: a row is a tuple you do not have to build.

Exercise 7 - When alignment is moot (stretch)

def is_ace_of_spades(suits, ranks):
    return np.where((suits == 0) & (ranks == 0))[0]

# returns the index (or indices, if duplicates) of the Ace of Spades
print(is_ace_of_spades(suits, ranks))

This query reads only suits and ranks. It is correct as long as those two columns are aligned with each other. It does not care about the alignment of locations or dealt_at. If a future reorder swaps two columns alongside suits and ranks - but for some reason fails to update dealt_at - this query still finds the Ace of Spades correctly.

This is the strong-form observation from §5: a (suit, rank) natural key uniquely identifies a card without an index. For constant-quantity tables (52 cards, fixed) this alternative works. For variable-quantity tables (creatures coming and going) you usually need a stable surrogate id, because the natural key may collide or fail to identify a row that has been re-issued. The book uses surrogates throughout because the through-line simulator is variable-quantity; this exercise is a reminder that not every table needs one.

7 - Structure of arrays (SoA)

Concept node: see the DAG and glossary entry 7.

Your deck has three numpy columns: suits, ranks, locations. Each field lives in its own array, indexed by entity. This layout is called Structure of Arrays - SoA. The opposite layout - a single list[Card] where each element is a dataclass holding all three fields - is called Array of Structs - AoS. They are different choices about where the same data lives.

# SoA: three columns, indexed in lockstep
suits     = np.zeros(52, dtype=np.uint8)
ranks     = np.zeros(52, dtype=np.uint8)
locations = np.zeros(52, dtype=np.uint8)

# AoS: one list of objects
@dataclass
class Card:
    suit: int
    rank: int
    location: int

cards: list[Card] = [...]  # 52 instances

Most Python programmers reach for AoS by default. It is what every introductory tutorial teaches: define a class for the entity, put instances in a list. The trouble is that in a real loop “the entity” is whatever the inner loop reads, not whatever the data model says belongs together. A system that counts cards in player 1’s hand reads only the location column - it does not need suits or ranks at all.

What “reads only one column” actually costs

With SoA, that count is one numpy primitive:

held_by_p1 = int(np.sum(locations == 1))

That call walks N bytes of locations, generates an N-byte boolean mask, and sums it - all inside C, no Python-level iteration. At N = 1,000,000 cards on this machine, the call takes ~0.5 ms.

With AoS, the same count is a Python for loop:

held_by_p1 = sum(1 for c in cards if c.location == 1)

That loop pays for one bytecode dispatch per card, one getattr per card, one comparison per card, and one increment per card. From §1, interpreter dispatch is ~5 ns/element, and getattr adds more. At N = 1,000,000 the same count takes 30-50 ms - two orders of magnitude slower for the identical answer on the identical data.

This is the bandwidth-bound vs interpreter-bound regime distinction from §4. SoA pushes the inner loop into C and walks contiguous bytes; AoS keeps the inner loop in the interpreter. The SoA call can run inside a 30 Hz tick (33 ms budget) at 1 million entities and use under 2% of the budget. The AoS call uses the entire tick budget at 1 million entities, leaving no room for the rest of the simulation.

The Python AoS penalty does not shrink with width

In a Rust AoS layout, the cost grows with the size of the struct: a 19-byte Card fills a cache line with three cards instead of sixty-four bytes of locations. A reader who does not need suits and ranks pays for them anyway because they ride in on the same cache line. Add a 16-byte nickname field and the gap widens.

In Python the story is different. Every field of a dataclass is a PyObject* pointer, so a “wider” Card does not put more bytes in the same cache line - it puts more pointers. The cost of c.location is not “extra cache traffic”; it is the fixed overhead of the Python attribute lookup. Adding fields you do not read makes each Card heavier in absolute terms (more allocation, more refcounts) but does not slow down the per-attribute access. The penalty is fixed by interpreter dispatch and getattr.

This makes the SoA win in Python categorical, not just quantitative. The numpy primitive escapes the interpreter entirely; the AoS loop does not. No amount of @dataclass(slots=True) discipline removes the per-attribute dispatch cost. From §6, slots reduce construction cost and per-instance memory, but every read of c.location still goes through Python’s attribute machinery.

SoA is the default

SoA is therefore the default in this book. AoS is sometimes the right choice - for example when every system reads every field of every entity on every tick (rare), or when N is so small that the loop overhead dominates regardless of layout (think dozens of items, not millions). But this is a tradeoff to earn by measurement, not to assume by habit. Write SoA first; switch to AoS only when a benchmark forces you to.

The §3 exhibit (code/measurement/aos_vs_soa_footprint.py) is the reference measurement for this chapter. Re-read its sum-column-0 row: list-of-tuples (the AoS twin) summed column 0 of one million ten-field rows in 30 ms; numpy SoA did the same in 0.4 ms. 75× faster for the canonical “system reads one column” operation. That is the regime your inner loops will live in for the rest of this book.

Note - numpy stores rows; pandas stores columns. numpy arrays are row-major (C order) by default, with column-major available via order="F". pandas is the other way - column-oriented under the hood (each column stored contiguously), which is why a DataFrame is fast down a column and slow across a row. Neither layout is optimal for all uses: SoA wins when loops read a few fields across many rows; row storage wins when loops touch whole records one at a time.

Exercises

You will need time.perf_counter() for some of these.

1. Build both layouts. Take your deck.py from §5 and add an AoS twin: a list[Card] of 52 entries, where Card is a @dataclass with three int fields. Build both and verify they encode the same logical content.
2. Count cards in a player’s hand, both ways. Write count_held_soa(locations, player) using np.sum(locations == player) and count_held_aos(cards, player) using a Python generator expression. Confirm they return the same number on the same deck.
3. Time the count at 10,000 entries. Replicate your deck to length 10,000. Time both functions with timeit (e.g., number=1000 for the numpy one, number=100 for the AoS one). Note the ratio in nanoseconds per element.
4. Scale to 1,000,000 entries. Repeat at length 1,000,000. The SoA version reads 1 MB of bytes; the AoS version walks a million pointer-chases through Python’s attribute machinery. Note the ratio. On most machines it is in the 50-200× range.
5. The unused-field case, Python edition. Extend Card with a nickname: str = "" field and a dealt_at: int = -1 field - five fields total instead of three. Rebuild both. Time the count again. Note that the SoA time is unchanged (the count still walks only locations) and the AoS time is also roughly unchanged (interpreter dispatch dominates either way). Compare to the Rust version of this chapter, where the AoS time grows with row size - Python’s penalty is fixed differently.
6. A case where AoS does not lose. Write a function that updates every field of one specific card. SoA writes to three (or five) different columns; AoS writes to one Python object. For the case “update every field of one card” - single entity, no loop - AoS is competitive or better. Time it. Note that this case has no inner loop, which is why the regime distinction from §4 doesn’t apply.
7. Construct, then read. From §6 you know constructing dataclass instances is slow. Time building a million-entry AoS list once, then summing the location query 1000 times. Compare to building a million-entry SoA once, then summing 1000 times. The construction cost amortises over many reads; for short-lived data, even SoA construction time becomes a factor. (Hint: this is a foreshadowing of §22 - mutations buffer.)
8. (stretch) A from-scratch SoaDeck class. Wrap the columns (suits, ranks, locations, dealt_at) in one Python class that owns them all. Provide reorder(self, order) as the only public mutator. What do you gain in correctness? What do you lose in flexibility? (Hint: you have just rebuilt the contract from §25 - one writer, many readers, four chapters ahead of schedule.)

Reference notes in 07_structure_of_arrays_solutions.md.

What’s next

§8 - Where there’s one, there’s many is the universalising principle. The deck taught it implicitly; the next section names it.

Solutions: 7 - Structure of arrays (SoA)

Exercise 1 - Build both layouts

import numpy as np
from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

def make_soa():
    return (np.repeat(np.arange(4, dtype=np.uint8), 13),
            np.tile(np.arange(13, dtype=np.uint8), 4),
            np.zeros(52, dtype=np.uint8))

def make_aos():
    return [Card(i // 13, i % 13, 0) for i in range(52)]

The two layouts encode the same logical content. The SoA version costs 156 bytes of data plus three numpy header objects. The AoS version costs 52 Card instances (each ~88 bytes including header, refcount, and three int fields) plus a list pointing at them - close to 5 KB total. 30× memory difference at N=52, before you’ve added any operations.

Exercise 2 - Count cards in a player’s hand

def count_held_soa(locations, player):
    return int(np.sum(locations == player))

def count_held_aos(cards, player):
    return sum(1 for c in cards if c.location == player)

# both return the same number on the same logical deck

np.sum(locations == player) produces a boolean mask in C, sums its True entries in C, returns an int - no Python iteration. The generator-expression form pays per-element interpreter dispatch plus getattr (.location) on each Card.

Exercise 3 - Time the count at N = 10,000

import timeit
n = 10_000
cards = [Card(i%4, i%13, i%5) for i in range(n)]
suits = np.tile(np.arange(4, dtype=np.uint8),  n//4+1)[:n]
ranks = np.tile(np.arange(13, dtype=np.uint8), n//13+1)[:n]
locations = np.tile(np.arange(5, dtype=np.uint8), n//5+1)[:n]

t_soa = timeit.timeit(lambda: count_held_soa(locations, 1), number=1000) / 1000
t_aos = timeit.timeit(lambda: count_held_aos(cards, 1),     number=100)  / 100
print(f"SoA: {t_soa*1e6:.2f} µs   AoS: {t_aos*1e6:.1f} µs   ratio: {t_aos/t_soa:.0f}×")

SoA:    5.89 µs   AoS:   181.9 µs   ratio:   31×

At N=10,000 the SoA version is 31× faster for the same answer.

Exercise 4 - Scale to 1,000,000 entries

SoA:  226.07 µs   AoS: 12,008.3 µs   ratio:   53×

The ratio widens with N because the SoA call stays bandwidth-bound (a tight C loop reading int8s sequentially) while the AoS call stays interpreter-bound (one Python step per row). Doubling N doubles both costs, but they live in different regimes - at 1M, SoA finishes in 0.2 ms, AoS in 12 ms. AoS uses 36% of a 30 Hz tick budget on a single count-by-attribute query.

Exercise 5 - The unused-field case, Python edition

Add nickname: str = "" and dealt_at: int = -1 to Card, rebuild, time again:

SoA:  226.07 µs   AoS5: 12,524.5 µs   (vs AoS3: 12,008.3)

The SoA time is unchanged (the count still walks only locations). The AoS time is also roughly unchanged (~4% slower from slightly larger objects, but not the multiplicative blowup the Rust edition’s chapter shows).

This is the Python-specific shape of the SoA win the chapter prose names: in Rust, AoS pays for unread fields by dragging them into the cache line. In Python, AoS pays a fixed-per-attribute interpreter cost regardless of how wide the row is - adding fields you don’t read makes each Card heavier in memory but does not slow the per-attribute access. The penalty is set by getattr and bytecode dispatch, not by cache-line traffic.

The categorical SoA win in Python is the escape from the interpreter. The numpy primitive runs in C; the AoS loop runs in CPython. No slots=True, no __slots__, no @dataclass(frozen=True) removes that gap.

Exercise 6 - A case where AoS does not lose

import time, numpy as np

# AoS: update one card, all five fields
cards = [Card(0, 0, 0) for _ in range(1_000_000)]
target = cards[42]
t0 = time.perf_counter()
target.suit = 1; target.rank = 5; target.location = 2
t1 = time.perf_counter()
print(f"AoS 1-card update: {(t1 - t0) * 1e9:.0f} ns")

# SoA: same update, three columns
suits = np.zeros(1_000_000, dtype=np.uint8)
ranks = np.zeros(1_000_000, dtype=np.uint8)
locations = np.zeros(1_000_000, dtype=np.uint8)
t0 = time.perf_counter()
suits[42] = 1; ranks[42] = 5; locations[42] = 2
t1 = time.perf_counter()
print(f"SoA 1-card update: {(t1 - t0) * 1e9:.0f} ns")

For a single row update, AoS and SoA are within noise of each other - both pay one or three Python attribute accesses, no inner loop, no scaling. The regime distinction from §4 doesn’t apply because there is no loop to be inside or outside of. AoS is competitive whenever your access pattern is “touch one row, read or write all its fields” - for example, a UI inspector showing details of a selected entity.

The book’s argument is not that AoS is always worse. It is that the inner loop of every system in the simulator reads one or two columns across many rows - exactly the case where SoA wins by the order of magnitude. AoS for the bookkeeping (the list of system names, the schedule); SoA for the rows.

Exercise 7 - Construct, then read

import time, numpy as np
from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

n = 1_000_000

# AoS: build once, query 1000 times
t0 = time.perf_counter()
cards = [Card(i%4, i%13, i%5) for i in range(n)]
t1 = time.perf_counter()
build_aos = t1 - t0
t0 = time.perf_counter()
for _ in range(1000):
    sum(1 for c in cards if c.location == 1)
t1 = time.perf_counter()
read_aos = t1 - t0

# SoA: build once, query 1000 times
t0 = time.perf_counter()
suits = np.tile(np.arange(4, dtype=np.uint8), n//4+1)[:n]
ranks = np.tile(np.arange(13, dtype=np.uint8), n//13+1)[:n]
locations = np.tile(np.arange(5, dtype=np.uint8), n//5+1)[:n]
t1 = time.perf_counter()
build_soa = t1 - t0
t0 = time.perf_counter()
for _ in range(1000):
    int(np.sum(locations == 1))
t1 = time.perf_counter()
read_soa = t1 - t0

print(f"AoS: build {build_aos*1000:.1f} ms, read 1000× {read_aos*1000:.1f} ms, total {(build_aos+read_aos)*1000:.1f} ms")
print(f"SoA: build {build_soa*1000:.1f} ms, read 1000× {read_soa*1000:.1f} ms, total {(build_soa+read_soa)*1000:.1f} ms")

Build cost amortises across many reads. For long-lived data (a deck that exists for the duration of a game session), the construction cost is a one-off. For short-lived data (a list of “cards dealt this hand” rebuilt every tick), construction can dominate - and even SoA pays a non-trivial construction time for million-element columns. This foreshadows §22 - mutations buffer: pre-allocate once, mutate in place, never reconstruct in the inner loop.

Exercise 8 - A from-scratch SoaDeck class (stretch)

class SoaDeck:
    """The single owner of the deck columns. The only mutation entry point is `reorder`."""

    def __init__(self):
        self.suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
        self.ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
        self.locations = np.zeros(52, dtype=np.uint8)
        self.dealt_at  = np.full(52, 255, dtype=np.uint8)

    def reorder(self, order: np.ndarray) -> None:
        """Apply `order` to every column in lockstep - the only function permitted to do so."""
        self.suits[:]     = self.suits[order]
        self.ranks[:]     = self.ranks[order]
        self.locations[:] = self.locations[order]
        self.dealt_at[:]  = self.dealt_at[order]

    def shuffle(self, rng: np.random.Generator) -> None:
        self.reorder(rng.permutation(len(self.suits)))

    def sort_by_suit_then_rank(self) -> None:
        self.reorder(np.lexsort((self.ranks, self.suits)))

    def deal(self, indices: list[int], player: int, tick: int) -> None:
        self.locations[indices] = player
        self.dealt_at[indices]  = tick

What you gain: one writer per column. Adding a fifth column means editing one method (reorder) and one constructor; every existing call site keeps working. Forgetting to reorder a column at a call site is impossible - there is only one site.

What you lose: explicit access to the columns from outside. Code that wants to read suits directly has to either reach through deck.suits (still allowed; reads are not the issue) or go through a method. For mostly-read systems this is fine; for diagnostic code that wants to peek at internals, the indirection adds friction.

The pattern is the §25 ownership-of-tables discipline at the smallest scale. The simulator’s actual tables are larger and have more callers, but the contract is identical: one writer, one reorder, columns read freely.

8 - Where there’s one, there’s many

Concept node: see the DAG and glossary entry 8.

Code is written for the array. A function that operates on one entity is just the special case of N = 1; it does not need its own abstraction. A card game with 52 cards is three arrays - suit, rank, location - not 52 objects. A simulation with 100 creatures is six arrays of length 100, not 100 instances of Creature. The plural is the primary unit; the singular is the trivial case.

The pattern is simple. Write the array version first. The singleton drops out as a one-element slice. To shuffle one card you swap two indices in the order array - same as shuffling the whole deck. To find the highest-rank card in player 1’s hand you scan the (small) hand array - same shape as scanning all 52. To deal one card you write one cell in locations - same shape as dealing many cells.

The OOP instinct, named

This stands against an instinct most Python programmers acquire on day one: the urge to write card.shuffle() or creature.update() and then puzzle over how to do it for many. Almost every Python tutorial models behaviour as methods on objects, then introduces lists of objects as the natural way to have many, then introduces for c in creatures: c.update() as the natural way to do something for each. Three steps, each locally sensible, that together build the pattern this chapter is asking you to drop.

The puzzle does not exist when you write for arrays from the start. shuffle(deck) is one function that works for any deck, including a deck of one. update(creatures) - taking the columns as numpy arrays - is one function that works for any population, including a population of one. The method-on-object form is strictly more code than the function-over-slice form: it requires a class, an __init__, a self argument that does nothing useful at the array level, and a calling convention that prevents the inner loop from ever leaving the interpreter.

A useful test: when you find yourself writing a method on a class, ask what does this look like over an array? If the array version is shorter, drop the method. If the array version is the same length, keep it as a free function over numpy arrays - def shuffle(suits, ranks, locations, order), not class Deck: def shuffle(self): .... Either way, the singleton was never the right unit of code.

The performance argument

There is also a performance reason - sharper in Python than in any compiled language. A method that operates on one entity at a time forces the system that uses it to call the method N times. From code/measurement/cache_cliffs.py, Python per-element work cost ~5 ns regardless of the size of the data; numpy bulk work cost ~0.2 ns/element. The ratio is roughly 25× at any size, and that is just the dispatch cost - before you add the cost of getattr(creature, 'energy') once per call, the refcount work on every return, and the lost opportunity for numpy to use SIMD instructions on contiguous bytes.

In a compiled language, an “obvious” inner loop over creatures.iter().for_each(|c| c.update()) is something the optimizer can usually rescue - inline the method, fuse the body into the loop, autovectorize the result. In Python the optimizer is the bytecode dispatcher and it cannot do any of that. The per-method-call form is essentially the worst case the language offers. Writing for arrays first is a request the interpreter can fulfil - it can hand the work to numpy and step out of the loop entirely. Writing for singletons-and-iterate is a request that pins the work inside the interpreter for every element.

“Where there’s one, there’s many” is therefore not an architectural slogan but a daily practice. It costs nothing the first time. It costs everything the first time you forget.

Exercises

These extend deck.py once more. The aim is to feel the array-first pattern in your fingertips before Part 3 turns into the rest of the book.

1. The function over a slice. Write def highest_rank_in_hand(hand, ranks) where hand is a numpy array of card indices and ranks is the deck’s rank column. Body should be one line: int(ranks[hand].max()). Use it on a 5-card hand. Then use it on a 1-card hand. Then use it on an empty hand. Same function, three N values.
2. Reverse the urge. Given an OOP-style def is_face_card(self) -> bool that lives on a hypothetical Card class, rewrite it as def face_cards(ranks) returning a numpy boolean mask of shape (N,). Apply it to all 52 cards in one call: mask = face_cards(ranks); face_count = int(mask.sum()).
3. The N = 0 case. What does highest_rank_in_hand do when hand is empty? arr.max() on an empty array raises. Pick a behaviour - return None, return a sentinel, raise - and justify the choice. (Hint: most uses can short-circuit with if hand.size == 0: return None.)
4. Predicate over a single value. Suppose you want is_red(suit) for a single card (suits 0 and 1 are hearts/diamonds). Write the array version def red_mask(suits) first - one line: (suits < 2). Then convince yourself the singleton case is red_mask(np.array([suit]))[0] - the array version covers it.
5. Count overhead. Time sum(is_face_card_per_row(suits[i], ranks[i]) for i in range(52)) against int(face_cards(ranks).sum()). The array version should be measurably faster at 52, much faster at 100,000. Document the ratio. (Repeat at N = 100,000 by replicating the deck.)
6. The dataclass twin, revisited. Take your list[Card] from §7 exercise 1. Write face_count_aos(cards) as a generator-expression sum and face_count_soa(ranks) as the numpy version. Time both at 1,000,000 entities. The ratio you measure here is the same ratio §7 measured for count_held - it is not specific to one query, it is the per-element dispatch cost of any inner loop you write in pure Python.
7. (stretch) From a tutorial. Find any Python tutorial that uses a class Card with methods (__init__, is_face, __repr__, etc.). Rewrite their full card game as three (or four) numpy arrays plus free functions. Compare line counts. Compare clarity. Compare what happens when you want to query “all face cards across the table” - one numpy call versus a loop over per-card method calls.

Reference notes in 08_where_theres_one_theres_many_solutions.md.

What’s next

You have closed Identity & structure. Cards behave; rows align; layouts are SoA; the singleton drops out. The next phase is Time & passes, starting with §11 - The tick. The ecosystem simulator from code/sim/SPEC.md is about to start running.

Solutions: 8 - Where there’s one, there’s many

These exercises ask you to write the array version first and let the singleton fall out as the trivial case.

Exercise 1 - The function over a slice

def highest_rank_in_hand(hand, ranks):
    return int(ranks[hand].max())

ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
print(highest_rank_in_hand(np.array([0, 13, 26, 39, 12]), ranks))   # 12 (K)
print(highest_rank_in_hand(np.array([12]), ranks))                  # 12 (K)
print(highest_rank_in_hand(np.array([], dtype=np.int64), ranks))    # raises - see ex 3

One function, three N values. The function does not branch on N; numpy’s indexing primitive handles all three identically (modulo the empty case).

Exercise 2 - Reverse the urge

def face_cards(ranks):
    return ranks >= 10                  # J=10, Q=11, K=12

mask = face_cards(ranks)
print(int(mask.sum()))                  # 12 - three face cards × four suits

The OOP-shaped def is_face_card(self) -> bool would force every caller to write for c in cards: if c.is_face_card(): ... - back to the interpreter-bound regime. The array version face_cards(ranks) is one numpy primitive that returns a mask, costs ~25 µs at N=100K, and also answers the singleton case via face_cards(np.array([rank]))[0].

Exercise 3 - The N = 0 case

def highest_rank_in_hand(hand, ranks):
    if hand.size == 0:
        return None                    # explicit "no answer" - caller decides
    return int(ranks[hand].max())

arr.max() on an empty array raises ValueError: zero-size array ... has no identity. Three reasonable resolutions:

• Return None. Forces the caller to handle the empty case explicitly. Best when “no cards” is a normal state.
• Return a sentinel (e.g., -1 for ranks). Cheap; risks confusing data with metadata. Avoid unless the type already has a natural sentinel.
• Raise. Right when “empty hand” is a programming error in this code path (e.g., a function that should only be called when at least one card is held).

The book leans toward returning None for “no answer” cases because the type signature Optional[int] documents the possibility at the call site. The Rust edition has Option<u8> for the same reason.

Exercise 4 - Predicate over a single value

def red_mask(suits):
    return suits < 2                   # 0=♠, 1=♥, 2=♦, 3=♣ - wait, suits 0 and 1 are spades and hearts here

# the chapter assumes suit indexing where 0,1 are red. Use the project's convention.
# If suits 1 (♥) and 2 (♦) are red:
def red_mask(suits):
    return (suits == 1) | (suits == 2)

# singleton case naturally:
suit = 1
is_red = red_mask(np.array([suit]))[0]

The array version covers the singleton; the singleton wraps the array version’s input in a one-element array. There is no separate code path. (The exact suit indexing - which numbers are red - is a convention to pick once and write down; the book’s elsewhere-conventions can drift between editions.)

Exercise 5 - Count overhead

import timeit, numpy as np

# at N = 52
t_arr = timeit.timeit(lambda: int(face_cards(ranks).sum()), number=10_000) / 10_000
t_loop = timeit.timeit(lambda: sum((int(ranks[i]) >= 10) for i in range(52)), number=1_000) / 1_000
print(f"N=52:        array={t_arr*1e6:.2f} µs   loop={t_loop*1e6:.1f} µs   ratio={t_loop/t_arr:.0f}×")

N=52:        array=  2.29 µs   loop=    5.8 µs   ratio= 3×
N=100,000:   array= 25.00 µs   loop= 1199.0 µs   ratio=48×
N=1,000,000: array=228.70 µs   loop=12525.0 µs   ratio=55×

At N=52 the array version is only 3× faster - numpy’s per-call overhead matters at small N. At N=100K the ratio settles at ~50× and stays there as N grows. The interpreter-vs-bandwidth gap from §1 is this ratio.

The lesson: even at N=52, where the array version’s overhead is dominant, it is still faster. Where there’s one, there’s many; the array version is never slower beyond a couple dozen elements, and is wildly faster past a few hundred.

Exercise 6 - The dataclass twin, revisited

from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int

def face_count_aos(cards):
    return sum(1 for c in cards if c.rank >= 10)

def face_count_soa(ranks):
    return int((ranks >= 10).sum())

n = 1_000_000
ranks_col = np.tile(np.arange(13, dtype=np.uint8), n // 13 + 1)[:n]
cards = [Card(0, int(ranks_col[i])) for i in range(n)]

t_aos = timeit.timeit(lambda: face_count_aos(cards), number=5) / 5
t_soa = timeit.timeit(lambda: face_count_soa(ranks_col), number=100) / 100
print(f"AoS face count: {t_aos*1e3:.1f} ms   SoA face count: {t_soa*1e3:.2f} ms   ratio: {t_aos/t_soa:.0f}×")

AoS face count: 12.5 ms   SoA face count: 0.23 ms   ratio: 55×

Same 55× ratio as §7’s count_held. The cost gap is not query-specific; it is a property of any per-element work done in pure Python over getattr-accessed fields. Every loop you write in CPython that walks for entity in entities: ... entity.field ... lives in this cost regime. SoA + numpy primitives moves the loop into C and out of the regime.

Exercise 7 - From a tutorial (stretch)

Pick almost any “Object-oriented programming in Python” tutorial that builds a card game (Real Python, Programiz, GeeksforGeeks, the Python docs themselves all have versions). The canonical shape is:

class Card:
    SUITS = ['♠', '♥', '♦', '♣']
    RANKS = ['A', '2', ..., 'K']
    def __init__(self, suit, rank):
        self.suit = suit
        self.rank = rank
    def __repr__(self):
        return f"{self.RANKS[self.rank]}{self.SUITS[self.suit]}"
    def is_face(self):
        return self.rank >= 10

class Deck:
    def __init__(self):
        self.cards = [Card(s, r) for s in range(4) for r in range(13)]
    def shuffle(self):
        random.shuffle(self.cards)
    def deal(self, n):
        return [self.cards.pop() for _ in range(n)]

The numpy rewrite is approximately:

import numpy as np

class Deck:
    def __init__(self):
        self.suits = np.repeat(np.arange(4, dtype=np.uint8), 13)
        self.ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
        self.locations = np.zeros(52, dtype=np.uint8)
        self.dealt_at = np.full(52, 255, dtype=np.uint8)
    def shuffle(self, rng):
        order = rng.permutation(52)
        self.suits[:] = self.suits[order]
        self.ranks[:] = self.ranks[order]
        self.locations[:] = self.locations[order]
        self.dealt_at[:] = self.dealt_at[order]

Line counts: the OOP version is typically 30-50 lines for Card + Deck. The numpy version is ~15 lines. And “all face cards across the table” is one numpy call (np.where(self.ranks >= 10)[0]) instead of a loop over per-card method invocations.

Beyond line count: the numpy version is the precondition for everything in Phase 3+. Persistence is np.savez(self.suits, self.ranks, self.locations, self.dealt_at) - three or four arrays out, the same arrays in. Replay is “store the seed, replay the operations.” Parallel partitioning is “split the index range.” None of these work cleanly when the data lives behind self.cards = list[Card]. The savings show up not in this chapter but in the rest of the book.

9 - Sort breaks indices

Concept node: see the DAG and glossary entry 9.

In §5 - Identity is an integer, exercise 10 left you with a bug. Player 1 was holding the index list [3, 17, 21, 28, 41]. The dealer sorted the deck columns by suit. Player 1’s hand was now wrong - the same indices, the same slots, but different cards.

That bug is the structural fact this section names. Sorting did not damage anything; the player’s reference was never robust to begin with. An index points at a slot, not at a thing. When the slot’s contents change, the index quietly changes meaning.

It is not only sorting. Any rearrangement does it: swap_remove (a O(1) deletion that moves the last row into the freed slot, coming in §21), reshuffling for locality (§28), compacting after a batch of deletions. The same index, the same array, the same line of code, now means a different card.

“But Python objects are stable references - can’t I just go back to that?”

This is the moment many readers feel the urge to retreat. The Python reflex from §6 - class Card with attributes - gave you object identity for free. A Card instance you held a reference to last week is still the same Card object today, regardless of what happened to the list it was in. id(card) does not change. The pointer through the Python interpreter to the heap-allocated Card is stable for the lifetime of the object.

So the temptation is real: keep the index-aligned numpy columns and a parallel list[Card] of object references, and use the objects when you need stability. Or just go back to list[Card] entirely - at least the references work.

This trade does not survive contact with the §3 footprint table or the §7 access-cost table. The numpy-SoA layout is 5× smaller and 75× faster at single-column queries than list[Card]; carrying a parallel object list to “rescue” reference stability gives back most of the footprint win and adds the synchronisation problem of keeping the column data in step with the object data. You have not solved the problem; you have hidden it inside an additional invariant.

The structural fix is the one §10 builds: an id column that travels with the row across rearrangements, plus (for variable-quantity tables) a generation counter on top. The card itself is a slot; the card’s name is an integer that we choose to be stable. The cost is one extra np.uint32 column. The benefit is that every rearrangement we will need from now on - sort, swap_remove, locality-driven reordering, compaction - works without breaking outside references.

This section’s only job is to make the slot vs name distinction concrete enough that §10’s solution feels inevitable rather than ceremonial.

Note - Why feel the pain first? Because the fix in §10 is small - one extra column - and small fixes only stick if the student knows what they fix. Reading “always store an id” without first feeling the bug produces students who add ids cargo-culted, then drop them when the codebase looks too cluttered. Reading it after watching player 1 lose their hand produces students who never drop them.

Exercises

You should still have your deck.py from §5. These exercises extend it.

1. Reproduce the bug. With player 1 holding [3, 17, 21, 28, 41], sort the deck columns themselves (suits, ranks, and locations in lockstep) by suit. The pattern is order = np.argsort(suits, kind="stable"); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]. Print player 1’s hand using card_to_string. Confirm the cards have changed.
2. A second rearrangement. Instead of sorting, swap two cards’ positions:

   suits[[3, 17]] = suits[[17, 3]]
   ranks[[3, 17]] = ranks[[17, 3]]
   locations[[3, 17]] = locations[[17, 3]]
   

   Print player 1’s hand again. Same bug shape, different cause.
3. A third rearrangement. Remove the card at slot 7 with the swap_remove pattern (move the last row into slot 7, then drop the last row): suits[7] = suits[-1]; suits = suits[:-1] and likewise for the other columns. Print player 1’s hand. Note that the cards at slots [17, 21, 28, 41] are unchanged but slot 3 may now hold what was previously the last card; meanwhile slot 51 has silently been deleted.
4. Quantify the breakage. Write a function that takes the original [3, 17, 21, 28, 41] plus a freshly built deck, applies a Fisher-Yates shuffle to the deck columns themselves (order = rng.permutation(52) and reorder all three columns), and counts how many of the five references still point at the same (suit, rank) value. Run it 100 times. Roughly what fraction of references survive a random shuffle of the deck? (Spoiler: very small. With probability 1/52 per slot, the expected number that survive by accident is 5/52 ≈ 0.1.)
5. A reference that can survive. Without writing any new code - on paper - describe what kind of reference would survive a shuffle. (Hint: you already know. The card’s (suit, rank) is unique to that card. The reference that survives is the one that does not depend on the slot.)
6. The “object reference” non-fix. Build a parallel list[Card] (use a @dataclass if you wish) alongside the numpy columns. Fill them so that cards[i] mirrors (suits[i], ranks[i], locations[i]). Now sort the numpy columns by suit without updating the object list. What does player 1 see if they read from the object list? What if they read from the numpy columns? Note that you have introduced a new bug - desynchronised state - without fixing the old one.
7. (stretch) The cost of never rearranging. Suppose you decide to never sort, swap, or remove from the deck columns, to avoid this bug forever. How would shuffling work? How would discarding a card work? Why does this not scale to ten thousand creatures?

Reference notes for these exercises in 09_sort_breaks_indices_solutions.md.

What’s next

Exercise 5 points at the answer; exercise 7 makes the never-rearrange option look bad. The real fix is to store identity separately from position - an id column that travels with the row across rearrangements, with a generation counter on top for variable-quantity tables. §10 - Stable IDs and generations builds it.

Solutions: 9 - Sort breaks indices

These exercises produce the bug, vary it, and quantify it. The structural fix is in §10.

Exercise 1 - Reproduce the bug

import numpy as np
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)

# Shuffle once so positions are non-trivial
rng = np.random.default_rng(42)
order = rng.permutation(52)
suits[:] = suits[order]; ranks[:] = ranks[order]

# Player 1 records indices [3, 17, 21, 28, 41]
held = [3, 17, 21, 28, 41]
locations[held] = 1
print("before sort:",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# ['K♦', '3♣', 'A♦', '4♥', 'A♠']

# Sort the columns themselves by suit
order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

print("after sort:",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Slots [3, 17, 21, 28, 41] now hold completely different cards. Player 1’s reference list has not changed; the slot contents have. The same line of code (ranks[3]) returns a different value before and after the sort. The bug is not in the sort; the bug is that the index was never a name for the card.

Exercise 2 - A second rearrangement

# fresh deck (rebuild from ex 1)
suits[[3, 17]] = suits[[17, 3]]
ranks[[3, 17]] = ranks[[17, 3]]
locations[[3, 17]] = locations[[17, 3]]

print([f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in [3, 17, 21, 28, 41]])

Two cards swap. Player 1’s references at indices 3 and 17 now point at each other’s old contents. References at 21, 28, 41 are unchanged. Same shape of bug - index is a slot, not a name - different cause.

Exercise 3 - A third rearrangement

# swap_remove slot 7
suits[7] = suits[-1]
ranks[7] = ranks[-1]
locations[7] = locations[-1]
suits     = suits[:-1]
ranks     = ranks[:-1]
locations = locations[:-1]

Slot 7 now holds what was the last card (slot 51). Slot 51 no longer exists - the array is length 51. Player 1’s references at indices 17, 21, 28, 41 still see the original cards (those slots untouched). Reference at 3 is unchanged because slot 3 was untouched too - but the card formerly at slot 51 has been silently removed from the universe of “cards.”

This is the §21 swap_remove pattern: O(1) deletion at the cost of moving one row’s worth of data, plus changing the index of one other row. Cheap, fast, and devastating to external references.

Exercise 4 - Quantify the breakage

def survival(rng):
    suits = np.repeat(np.arange(4, dtype=np.uint8), 13)
    ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
    # shuffle once so the deck is non-trivial
    o = np.random.default_rng(42).permutation(52)
    suits[:] = suits[o]; ranks[:] = ranks[o]

    held = [3, 17, 21, 28, 41]
    pairs = [(suits[i], ranks[i]) for i in held]

    # rearrange and count survivors
    o = rng.permutation(52)
    suits[:] = suits[o]; ranks[:] = ranks[o]
    return sum(1 for (s, r), i in zip(pairs, held) if (suits[i], ranks[i]) == (s, r))

rng = np.random.default_rng(0)
print(f"survived: {sum(survival(rng) for _ in range(100))} / 500")

survived: 13 / 500

Expected value: each reference has probability 1/52 ≈ 1.9% of pointing at its original card after a uniform shuffle. Five references × 100 trials × 1/52 ≈ 9.6 expected survivors. Empirically: 13 - within Poisson noise of the prediction.

98% of references are wrong after one shuffle. Not “occasionally broken in edge cases” - catastrophically broken in the common case.

Exercise 5 - A reference that can survive

The reference that survives a shuffle is the one that does not depend on the slot. The natural-key reference (suit, rank) survives any rearrangement because (suits[i], ranks[i]) is a property of the card, not of the slot. The dealer rearranges slots; the cards themselves are not changed.

But natural keys break in two cases:

• Duplicates. Variable-quantity tables (creatures, items, projectiles) routinely have rows with identical field values; “the creature with energy=10 at position (5,5)” can have many matches. A natural key needs to be a guaranteed-unique property of the row.
• Re-issues. A row removed and a new row added with the same values is indistinguishable by natural key. For variable-quantity tables this is a bug waiting to happen.

The structural fix in §10 is to invent a name and write it down: an id column whose values are guaranteed unique within the table, plus a generation counter to handle re-issues.

Exercise 6 - The “object reference” non-fix

from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

# parallel lists
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)
cards = [Card(int(suits[i]), int(ranks[i]), int(locations[i])) for i in range(52)]

# sort the numpy columns, NOT the object list
order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

# now: numpy columns sorted, cards list still in original order
print(f"numpy slot 3:   {RANK[ranks[3]]}{SUIT[suits[3]]}")
print(f"object cards[3]: rank={cards[3].rank} suit={cards[3].suit}")
# they disagree

The object list and the numpy columns now describe two different decks. Player 1 reading from cards[3] sees one card; reading (suits[3], ranks[3]) sees another. You have not fixed the index-into-slot problem - you have added a synchronisation problem on top of it.

This is why §9 explicitly rejects the parallel-object-list approach: it preserves stable references at the cost of doubling memory and inventing an alignment invariant the original problem didn’t have. The cure is worse than the disease.

Exercise 7 - The cost of never rearranging (stretch)

If the deck columns are never sorted, swapped, or compacted:

• Shuffling: must produce an order array each time and read through it indirectly. for i in range(52): print(card_at(order[i])). Every read pays one extra indirection. Workable for 52 cards.
• Discarding a card: cannot remove it from the columns; must mark it dead via a status column (e.g., locations[i] = 255). The columns grow forever. For 52 cards over a single game session, fine.
• Adding a card: np.concatenate to grow each column. O(N) per addition.

Why this doesn’t scale to 10,000 creatures (let alone the simulator’s 100M):

1. Forever-growing tables. A simulator that runs for an hour and births 10K creatures per second has 36M dead rows by the end. Reading through them costs proportionally; bandwidth is the budget; you’ve spent it on tombstones.
2. No compaction means no locality. Live and dead rows are interleaved. Cache lines hold half-tombstones. The §26/§28 compaction-for-locality pass is impossible.
3. Parallel partition is impossible. §31-§32 split the table by index range; if the live data is sparse and randomly distributed across a forever-growing array, you can’t carve clean ranges.

The never-rearrange policy works for constant-quantity tables (52 cards, fixed grid sizes). It fails for everything that breathes - births, deaths, additions, removals. The book’s simulator is variable-quantity, so the next chapter builds the fix.

10 - Stable IDs and generations

Concept node: see the DAG and glossary entry 10.

In §9 you watched a player’s reference go stale because they were holding slots, not names. The fix is to give each row a name - a stable identifier - that travels with the row when it moves.

A stable id is one extra column. For the deck:

ids = np.arange(52, dtype=np.uint32)

Now every card has both a slot (its current index in the columns) and an id (its name). When you sort the columns, you reorder ids in lockstep with everything else:

order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]
ids[:]       = ids[order]

The card with id == 17 is still the same card - its suit, rank, and location are unchanged. It is just at a different slot.

To find a card by id, scan the ids column:

def slot_of(ids: np.ndarray, target: int) -> int | None:
    matches = np.where(ids == target)[0]
    return int(matches[0]) if matches.size else None

That is O(N), which is fine for a 52-card deck and slow for a million creatures. The fix - an id_to_slot map maintained on every rearrangement - is §23 - Index maps. For now the linear scan is honest pedagogy.

Generations: when slots are reused

The deck is constant-quantity. Always 52 cards, never more, never less. The simple ids column is enough.

For variable-quantity tables - creatures that are born and die, packets that arrive and are processed, sessions that come and go - slots get reused. A new creature is born in the slot that just held a dead one. The ids column for such a table behaves like an auto-incrementing primary key in a database: every new row gets a fresh, never-reused integer; old rows keep their original ids forever. The simulator differs from a database in one structural way - it recycles slots to keep memory bounded, while a database table just grows. That recycling is what generations exist for. Imagine code that held a reference to the dead creature: their reference points at a slot that may now hold a different creature with possibly the same id (if id reuse happens) or - worse - a valid-looking row that is no longer the row they cared about.

One more column fixes it: a gens (generation) counter that increments every time a slot is recycled. A reference is now a pair (id, gen). To dereference it, you check that the row’s stored gen still matches the reference’s gen. If it does, the reference is live. If it does not, the slot has been recycled since the reference was taken, and the dereference returns None.

from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

def get_slot(creatures, ref: CreatureRef) -> int | None:
    slot = creatures.id_to_slot.get(ref.id)
    if slot is None:
        return None
    if int(creatures.gens[slot]) != ref.gen:
        return None
    return slot

(This is one of the few places in the book where a NamedTuple earns its weight: a CreatureRef is a value passed through external code, and giving it field names makes the API readable. Per §6, the cost is real - a NamedTuple allocation per reference - but references are rare, not per-tick. Where the same lesson runs through hot data, the answer is still numpy columns.)

This is the pattern called a generational arena. It is the single mechanism behind every “handle” type in every ECS engine: Bevy’s Entity, Rust’s slotmap::SlotMap, C++’s entt::registry, and the indirect-handle pattern in databases. They differ in details - width of the id, packing into a u64, generation overflow handling - but the structural idea is the same: one column for identity, one for generation, a checked dereference.

Where new slots come from

The arena hands out a slot on every birth and takes one back on every death. A death pushes the freed slot onto the free list; a birth pops from free if a recycled slot is waiting. Only when free is empty does a birth need genuinely new ground, and that is the moment §3 warned about. A numpy column is a fixed buffer with no in-place append: growing it means allocating a larger buffer and copying every existing value across. Do that once per birth and a run of N births costs O(N²) in copying alone.

The fix is the one CPython’s list already performs on your behalf, made explicit here because numpy will not do it for you. The arena carries two integers beside its columns: a capacity, the real length of the buffers, and a count, the high-water mark of slots it has ever handed out. A birth with an empty free list takes slot count and increments it; only when count reaches capacity does the arena reallocate, doubling the buffers and copying once. Amortised over many births that reallocation disappears, exactly as it does for list.append - the difference is that here the machinery is visible and the over-allocation policy is yours to choose.

This is why “how many creatures?” is a question the simulator answers once, as a starting capacity, instead of dodging by appending forever. A list lets you dodge it; a numpy column makes you name it. Sizing the buffer is not ceremony - it is the price of the contiguous bytes that make every system after this one fast.

That is enough machinery for the rest of the book to lean on. Sorting now works because the id column travels with the row. Deletion now works because the generation counter rejects stale references. Append-only and recycling tables (§24) are two policies on the same machinery.

Note - The strong form of §5 still applies. If your row has a natural key - (suit, rank), (date, ticker), (species, position) - you do not need a surrogate id. The card-game deck can be played without ids; the reference that survives is the (suit, rank) pair, because the data is unique by construction. Surrogate ids and generations earn their keep when the data has no natural unique tuple - which is most of the time once you start producing rows at runtime.

Exercises

These extend the §5 deck once more, then take a step toward the simulator’s variable-quantity case.

1. Add the id column. Add ids = np.arange(52, dtype=np.uint32) to your deck. Modify your sort so it reorders ids along with the other columns. Verify the original ids are still there, just in a new order.
2. Find a card by id. Implement slot_of(ids, target) as in the prose. Use it to look up the card with id == 17 after a sort.
3. Resolve the §9 bug. With player 1 holding ids [3, 17, 21, 28, 41] (not slots), sort the deck. Use slot_of to translate ids to slots and print the hand. Confirm the cards are unchanged.
4. Permutation-friendly hand query. Rewrite cards_held_by(locations, ids, player) -> np.ndarray to return ids, not slots. The player now holds names. Test by sorting the deck after a deal and confirming cards_held_by still returns the same five cards.
5. A first generation counter. Add gens = np.zeros(52, dtype=np.uint32). The 52-card deck does not actually recycle, but extend a small swap_remove-like operation: pop the last card from the deck (location 0), insert a “fresh” card at the freed slot, and bump that slot’s gens by one. Take a CreatureRef-style (id, gen) reference before the operation. After the operation, look up the slot by id; check gens[slot] against the reference’s gen. Confirm the dereference correctly reports stale.
6. (stretch) A tiny generational arena, sized honestly. Outside the deck, build a Creatures class with pos: np.ndarray (float32), gens: np.ndarray (uint32), a free: list[int] of recycled slots, and two integers: capacity (the length of the columns) and count (the high-water mark of slots ever handed out). insert(pos) -> CreatureRef pops a slot from free if one is waiting; otherwise it takes slot count and increments it; and if count would reach capacity first, it grows the columns - allocate new arrays of 2 * capacity, copy the old values across, update capacity. remove(ref) returns the slot to free and bumps that slot’s gens. get(ref) -> float | None does the checked dereference. Confirm two things by example: a stale reference cannot read a fresh creature’s data, and inserting 10 * the initial capacity triggers only a handful of reallocations rather than one per insert. Then answer in a comment: why would pos = np.append(pos, value) on every insert have been quadratic?
7. (stretch) The shape of id_to_slot. Right now slot_of is O(N). Sketch (do not implement) the id_to_slot array - np.full(N_ids, MAX, dtype=np.uint32) - that lets you do the lookup in O(1). Note what has to happen on every reorder: when slot i is the new home of id k, id_to_slot[k] = i. This is a foreshadow of §23 - Index maps. The lookup speedup costs you another column to keep aligned.
8. (stretch) Compare with a real ECS handle. Read the Entity documentation for bevy_ecs (Rust) or look at the EntityHandle docs of any Python ECS library. Identify which of your fields and operations correspond. What does the production library add that you didn’t need for the simulator? Decide consciously whether to adopt it. (This is the from-scratch-then-price-the-crate move from §41 - Deferred abstraction and §42 - You can only fix what you wrote.)

Reference solutions for the deck exercises (1-5) in 10_stable_ids_and_generations_solutions.md. The arena and library exercises follow the same shape and are worth working without reference.

What’s next

You now have stable references. The next thing the simulator will need is to look up a row by id in O(1) rather than O(N) - an id_to_slot map maintained on every reordering. That is §23 - Index maps. It is one extra np.ndarray, updated whenever the columns move.

Part 2 is closed. Identity is an integer; rows align in lockstep; SoA is the default; the singleton drops out; sort breaks indices and ids fix it. The next phase is Time & passes, starting with §11 - The tick. The ecosystem simulator from code/sim/SPEC.md is about to start running.

Solutions: 10 - Stable IDs and generations

The fix from §9: one extra column.

Exercise 1 - Add the id column

import numpy as np

def new_deck():
    suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
    ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
    locations = np.zeros(52, dtype=np.uint8)
    ids       = np.arange(52, dtype=np.uint32)        # the new column
    return suits, ranks, locations, ids

def reorder(suits, ranks, locations, ids, order):
    suits[:]     = suits[order]
    ranks[:]     = ranks[order]
    locations[:] = locations[order]
    ids[:]       = ids[order]                          # one extra line

# verify ids permute, not regenerate
suits, ranks, locations, ids = new_deck()
order = np.argsort(suits, kind="stable")
reorder(suits, ranks, locations, ids, order)
assert sorted(ids.tolist()) == list(range(52))         # same set, just permuted

The id column is just a numpy column. The reorder function gains one line.

Exercise 2 - Find a card by id

def slot_of(ids: np.ndarray, target: int) -> int | None:
    matches = np.where(ids == target)[0]
    return int(matches[0]) if matches.size else None

# after a sort, find the card with id = 17
slot = slot_of(ids, 17)
print(f"id 17 is now at slot {slot}: "
      f"{RANK[ranks[slot]]}{SUIT[suits[slot]]}")

O(N) on each lookup. Fine for 52 cards. For million-row tables, §23 caches the inverse map; that’s an optimisation, not a correction.

Exercise 3 - Resolve the §9 bug

# fresh deck, pre-shuffled so positions are non-trivial
suits, ranks, locations, ids = new_deck()
rng = np.random.default_rng(42)
reorder(suits, ranks, locations, ids, rng.permutation(52))

# Player 1 records IDs [3, 17, 21, 28, 41] - names, not slots
held_ids = [3, 17, 21, 28, 41]
slots = [slot_of(ids, k) for k in held_ids]
locations[slots] = 1
print("before sort:",
      [f"{RANK[ranks[s]]}{SUIT[suits[s]]}" for s in slots])
# ['4♠', '5♥', '9♥', '3♦', '3♣']

# Sort the columns by suit (in lockstep with ids)
reorder(suits, ranks, locations, ids, np.argsort(suits, kind="stable"))

# Look up the same ids - get the new slots - read the cards
slots2 = [slot_of(ids, k) for k in held_ids]
print("after sort: ",
      [f"{RANK[ranks[s]]}{SUIT[suits[s]]}" for s in slots2])
# ['4♠', '5♥', '9♥', '3♦', '3♣']  - same cards!

The slots changed; the cards did not. Player 1’s reference list is in the id domain - names, not addresses - and survives any rearrangement of the columns.

Exercise 4 - Permutation-friendly hand query

def cards_held_by(locations: np.ndarray, ids: np.ndarray, player: int) -> np.ndarray:
    return ids[locations == player]                    # return ids, not slots

# deal, then sort, then re-query - should return the same set
suits, ranks, locations, ids = new_deck()
locations[[0, 1, 2, 3, 4]] = 1
held_before = set(cards_held_by(locations, ids, 1).tolist())

reorder(suits, ranks, locations, ids, np.argsort(suits, kind="stable"))
held_after = set(cards_held_by(locations, ids, 1).tolist())

assert held_before == held_after                       # same five ids, regardless of sort

locations == player is a boolean mask of slots in the player’s hand. Indexing the ids column with that mask returns the names of those cards. The set of names is invariant under reordering of the columns; the set of slots is not.

Exercise 5 - A first generation counter

from typing import NamedTuple

class CardRef(NamedTuple):
    id:  int
    gen: int

suits, ranks, locations, ids = new_deck()
gens = np.zeros(52, dtype=np.uint32)

# Take a reference to the card with id=17 BEFORE we recycle anything
slot = slot_of(ids, 17)
ref  = CardRef(id=17, gen=int(gens[slot]))             # gen=0

# A swap_remove-like operation: pop the card from slot 51, fill slot 17 with a "fresh" card
# (not realistic for a 52-card deck, but mimics the simulator pattern)
suits[17]     = suits[51]                              # recycle: move the last card here
ranks[17]     = ranks[51]
locations[17] = locations[51]
ids[17]       = 52                                     # fresh id (would be next sequence number)
gens[17]     += 1                                      # bump the generation: slot was reused

def deref(ids, gens, ref: CardRef) -> int | None:
    slot = slot_of(ids, ref.id)
    if slot is None:                                   # id no longer in the table
        return None
    if int(gens[slot]) != ref.gen:                     # slot recycled since ref taken
        return None
    return slot

print(deref(ids, gens, ref))                           # None - correctly stale

The (id, gen) pair is the read receipt. After the recycle, slot_of(ids, 17) returns None (id 17 was overwritten with id 52). Even if id 17 had been re-issued - e.g., into slot 17 - the generation bump (0 → 1) would have caught it: the reference’s gen=0 would not match the slot’s gens[slot]=1, and deref would correctly report stale.

This is the generational arena pattern in 30 lines. The same shape carries the simulator’s variable-quantity tables in the rest of the book.

Exercise 6 - A tiny generational arena (stretch)

import numpy as np
from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

class Creatures:
    def __init__(self, capacity: int = 1024):
        self.cap   = capacity
        self.pos   = np.zeros((capacity, 2), dtype=np.float32)
        self.ids   = np.full(capacity, np.iinfo(np.uint32).max, dtype=np.uint32)  # MAX = empty
        self.gens  = np.zeros(capacity, dtype=np.uint32)
        self.free: list[int] = list(range(capacity - 1, -1, -1))                  # stack of free slots
        self.next_id = 0

    def insert(self, x: float, y: float) -> CreatureRef:
        if not self.free:
            raise MemoryError("Creatures table full")
        slot = self.free.pop()
        self.pos[slot, 0] = x
        self.pos[slot, 1] = y
        new_id = self.next_id
        self.next_id += 1
        self.ids[slot] = new_id
        return CreatureRef(id=new_id, gen=int(self.gens[slot]))

    def _slot_of(self, target_id: int) -> int | None:
        m = np.where(self.ids == target_id)[0]
        return int(m[0]) if m.size else None

    def remove(self, ref: CreatureRef) -> bool:
        slot = self._slot_of(ref.id)
        if slot is None or int(self.gens[slot]) != ref.gen:
            return False
        self.ids[slot] = np.iinfo(np.uint32).max        # mark empty
        self.gens[slot] += 1                            # bump generation
        self.free.append(slot)
        return True

    def get(self, ref: CreatureRef) -> tuple[float, float] | None:
        slot = self._slot_of(ref.id)
        if slot is None or int(self.gens[slot]) != ref.gen:
            return None
        return float(self.pos[slot, 0]), float(self.pos[slot, 1])

# Stale-reference test
c = Creatures(capacity=4)
ref_a = c.insert(1.0, 2.0)             # id=0, gen=0
c.remove(ref_a)
ref_b = c.insert(99.0, 99.0)           # id=1, possibly in the same slot, gen=1 there

assert c.get(ref_a) is None             # stale ref correctly rejected
assert c.get(ref_b) == (99.0, 99.0)

A 70-line generational arena. The contract: a CreatureRef is the only valid handle into the table; the table guarantees that a get or remove against a stale ref returns None/False rather than reading or writing the wrong row.

Exercise 7 - The shape of id_to_slot (stretch)

# capacity-bounded inverse map: id → slot, kept in step with the columns
MAX_IDS = 1_000_000
id_to_slot = np.full(MAX_IDS, np.iinfo(np.uint32).max, dtype=np.uint32)

def slot_of_o1(id_to_slot: np.ndarray, target_id: int) -> int | None:
    s = int(id_to_slot[target_id])
    return None if s == np.iinfo(np.uint32).max else s

What the inverse map costs:

• Memory: MAX_IDS × 4 bytes - 4 MB at 1M ids. Constant per id, not per row in the table.
• Update on every reorder: when order is applied to the columns, also rebuild id_to_slot so id_to_slot[ids[i]] = i for every new slot. That’s another loop of length N (one numpy primitive: id_to_slot[ids] = np.arange(N)).

What it buys: O(1) lookups at every dereference. For a simulator that does 100K+ dereferences per tick, this is the difference between a feasible inner loop and a broken one. §23 builds it properly with the lifecycle (ids issued, freed, recycled) handled.

Exercise 8 - Compare with a real ECS handle (stretch)

bevy_ecs::entity::Entity is conceptually two values packed into a u64:

• index: 32-bit slot in the entity table (≈ this chapter’s id field)
• generation: 32-bit reuse counter (≈ this chapter’s gen field)

Mapping:

What bevy adds that you don’t strictly need: packed handle (one u64 vs two integers), explicit Entity::PLACEHOLDER constant, deserialisation tagging, integration with bevy’s reflection/inspector. None of these are required for a working ECS - they’re ergonomics for a public API used by hundreds of downstream crates.

This is the §41 / §42 move. Build the small version yourself first; you now know what Entity does. When you later read bevy’s source you can see what it adds and price each addition against your needs. Most simulators don’t need a packed u64 handle; some do. The cost-benefit is yours, with the from-scratch version in hand.

11 - The tick

Concept node: see the DAG and glossary entry 11.

A program’s life has a shape:

• Start-up - initialisation. Tables are allocated, inputs are opened, the RNG is seeded, the world reaches a known state.
• Steps - ticks of the clock in a simulation, turns in a card game, requests in a server. The repeating unit of forward motion.
• Save and load - the in-memory state is preserved to disk so a future run can resume from where this one left off. Optional, but if you want it, it lives here.
• Exit - resources are returned to the kernel. Memory, file handles, sockets, lockfiles. Failure to do this cleanly is called a memory leak (or a stale lock, or a broken socket).

This section is about the step. The step is where the time budget binds, where the systems - the functions that read and write the world’s tables - run in the order set by their DAG, where determinism either holds or breaks. The other phases are real - the book returns to save and load when persistence is named at §36, and exit is mostly the operating system’s job - but the inner step is what makes or breaks every other property the book builds on.

Each step is a tick. State at the start of a tick is read; state at the end is written; nothing is half-updated mid-tick. Even an interactive program - a card game waiting for the next move, a text editor waiting for a keystroke - is a tick loop, just with an external trigger driving it. A program that does a single pass over a file and exits is a degenerate tick loop with N=1.

Two shapes of tick

A time-driven tick fires at a fixed rate. The simulator from code/sim/SPEC.md runs at 30 Hz: one tick every 33 ms. The loop wakes up, advances every system by one step, sleeps until the next tick. Most simulations, games, control loops, audio engines, and animation systems are time-driven. The rate is a contract with the rest of the world: at this rate, output appears.

A turn-based tick fires when an event arrives. A card game ticks when a player makes a move. A chess engine ticks when its opponent moves. A discrete-event simulator ticks at the timestamp of the next pending event, however far in the future that is. The clock advances with the events, not under them. Turn-based ticks have no fixed rate; their pace is set by the input stream.

Both are ticks. The difference is what triggers the next pass:

# time-driven
import time

TICK_S = 1.0 / 30.0  # 33.3 ms

while running:
    start = time.perf_counter()
    run_all_systems(world)
    elapsed = time.perf_counter() - start
    if elapsed < TICK_S:
        time.sleep(TICK_S - elapsed)

# turn-based
while running:
    event = wait_for_next_event()
    apply_event(world, event)

The §0 simulator runs time-driven. The card game from §5 ran turn-based - every card you dealt was one tick. Both are valid; both fit the same framework.

Not asyncio. Not threads.

Two reflexes the modern Python reader will reach for, and neither is the right tool here.

The asyncio reflex says “control loops are async.” asyncio is a scheduler for I/O-bound work - code that spends most of its time waiting for sockets, files, or sleeps. A simulation tick is CPU-bound: every tick, you have computation to do, and the goal is to do it as fast as possible and then sleep precisely until the next deadline. The asyncio event loop adds dispatch overhead (awaitable wrapping, task stepping, the event loop’s own bookkeeping) without giving you anything in return - you are not waiting on external I/O. A synchronous while True: loop with time.sleep is the correct shape, and it is shorter.

The threading reflex says “use a Timer thread to fire ticks.” This is worse. CPython’s GIL means the timer thread and the main thread cannot run Python code simultaneously; the timer thread firing the tick at 33 ms intervals contends for the same lock the simulation needs. You add scheduler nondeterminism (the OS picks who gets the GIL after each tick interval), you add the GIL-acquisition cost on every wakeup, and you gain nothing - you could have called time.sleep from the main thread directly.

A simulation tick wants three things: precision (sleep until exactly the next deadline), determinism (the same input produces the same output), and simplicity (one place to read to understand the loop). A synchronous loop with time.perf_counter and time.sleep provides all three. The two reflexes above provide none of them. Reach for the simplest tool that gives you the property you actually need.

What fits in a tick

The budget binds the design. From code/measurement/tick_budget.py, one motion system (pos += vel * dt) measured on this machine:

Read the rows. At 100,000 entities, both layouts fit comfortably at 30 Hz, but the dataclass loop already uses 125× more of the budget than the numpy version. At 1,000,000 entities, the dataclass version eats 84% of the 30 Hz budget on one system - the rest of the simulator has 5 ms left for everything else. It does not fit at 60 Hz at all. The numpy version still has 98% of the budget free. At 10,000,000 entities, even the numpy version is at 87% of the 30 Hz budget; the simulation has hit a scale limit on this hardware, and the next move is either reducing the work per element, partitioning the work across processes (§31), or accepting a slower tick rate.

The dataclass version at 10,000,000 was skipped because it would extrapolate to ~280 ms per tick - eight ticks of 30 Hz budget - for one system, before any other work. The right reading of that gap is not “numpy is fast” but “an interpreter-bound inner loop puts a hard ceiling on the population your tick can sustain, and the ceiling is much lower than most readers expect.”

When a tick runs long - the dataclass loop above at a million entities, a load spike, more work than you planned - the budget has a visible failure mode: the frame is dropped. The loop wakes late, the rate sags below its contract, and downstream something stutters. That visibility is worth instrumenting from the first day, because it is the cheapest operations tool you will ever have. Time each tick, and when it overruns the budget, raise in development so a regression stops the run, and warn and count in production so a degraded run is recorded rather than silent. The count of late ticks is the first number you reach for when asked whether the loop is keeping up - a habit worth building long before anyone is watching a dashboard at 2 AM.

The budget is also where mixing turn-based and time-driven thinking in the same loop produces drift: the turn-based subsystem’s pace bleeds into the time-driven subsystem’s budget. The fix is to keep the two cleanly separated - typically one outer loop and the other as an event source feeding it.

A tick is the unit of forward motion in any program that has forward motion. The next sections name what fits in one tick, in what order, and what does not.

Exercises

You will need a fresh project for these. mkdir tick_lab && cd tick_lab && uv init is enough.

1. A 30 Hz time-driven loop. Write a main() that loops at 30 Hz. Each iteration, print the elapsed time since program start. Sleep between ticks to maintain the rate. Run it for 10 seconds. Did you actually get 300 iterations? Use time.perf_counter() - time.time() can go backwards on clock corrections.
2. The naive sleep mistake. Replace your sleep logic with time.sleep(1/30) (no measurement of work time). Run for 30 seconds. Does the program drift over time? Why? (Hint: each iteration’s work + sleep is now 33 ms + work_ms, not 33 ms total.)
3. Dropped frames. Inside the loop, sleep for 50 ms - longer than the budget. The loop is now running at 20 Hz; it has missed frames. Print a warning when this happens, and keep a running count of late ticks - that count is the first number to reach for when asked whether the loop is keeping up. The right way to detect: if elapsed > TICK_S: late += 1; print(f"missed deadline by {elapsed - TICK_S:.3f} s").
4. A turn-based loop. Write a tiny REPL: print > , read a line with input(), print you said: <line>. Each line is one tick. Run it. Note that the loop has no fixed rate - its pace is your typing.
5. Run the tick-budget exhibit. uv run code/measurement/tick_budget.py. Note the row where the dataclass version stops fitting at 60 Hz. Note the row where it stops fitting at 30 Hz. Note that the numpy version is still fine at both N values. The book is asking you to keep the numpy line running for the next thirty chapters.
6. The asyncio comparison. Rewrite exercise 1 using asyncio.run and await asyncio.sleep. Measure: does it tick at the same rate? Does the program use more memory? More wall time per tick? Compare your two implementations side by side. Most readers will find the asyncio version harder to read and not measurably faster - exactly the calibration the prose above predicts.
7. (stretch) A discrete-event tick loop. Maintain a list of (timestamp, message) events sorted by timestamp. Pop the smallest-timestamp event, advance a “simulation clock” to that timestamp, print the message, repeat until the queue is empty. This is the structure of a discrete-event simulator and a preview of §12. Use heapq for the priority queue.

Reference notes in 11_the_tick_solutions.md.

What’s next

Exercise 7 hints at the next section. The clock can live on the events themselves, independent of how often the loop fires. §12 - Event time vs tick time names that separation.

Solutions: 11 - The tick

Exercise 1 - A 30 Hz time-driven loop

import time

TICK_S = 1.0 / 30.0
start = time.perf_counter()
end   = start + 10.0
ticks = 0
while time.perf_counter() < end:
    t0 = time.perf_counter()
    ticks += 1
    print(f"t={t0 - start:6.3f}s  tick={ticks}")
    elapsed = time.perf_counter() - t0
    if elapsed < TICK_S:
        time.sleep(TICK_S - elapsed)

print(f"{ticks} ticks in {time.perf_counter()-start:.2f}s")

Expected: 300 ticks ± 1 in 10 seconds. The loop sleeps for TICK_S - work_done, so each iteration ends exactly TICK_S after it began (modulo OS scheduling). time.perf_counter() is monotonic; time.time() can step backwards on NTP corrections and is the wrong tool here.

Exercise 2 - The naive sleep mistake

while True:
    do_some_work()
    time.sleep(1/30)            # always 33 ms, regardless of work time

Each iteration takes work_ms + 33 ms, not 33 ms total. If the work consistently takes 5 ms, the loop ticks at 1 / (0.005 + 0.033) ≈ 26.3 Hz, not 30. Over a minute that is 1,580 ticks instead of 1,800 - a 12% deficit, and the program reports “running at 30 Hz” because that’s what it asked for.

The drift is silent: nothing in the program complains. Only an external observer (the wall clock, an event log, an animation that runs slow) notices. The fix is to measure work time and subtract, as in exercise 1.

Exercise 3 - Dropped frames

while running:
    t0 = time.perf_counter()
    do_some_work()                    # may take longer than TICK_S
    elapsed = time.perf_counter() - t0
    if elapsed > TICK_S:
        print(f"missed deadline by {(elapsed - TICK_S)*1000:.1f} ms")
    else:
        time.sleep(TICK_S - elapsed)

If do_some_work() sleeps 50 ms (longer than the 33 ms budget), the loop runs at 20 Hz and prints missed deadline by 16.7 ms every iteration. Detecting missed deadlines is half the battle; responding to them is the rest. The simplest response is “log it and continue”; smarter responses (skip a frame’s interpolation, drop secondary work, lower the visible LOD) live at the application layer.

A simulator that has missed its tick budget is a simulator running on the wrong hardware or with the wrong N. Naming the deadline-miss is how you know.

Exercise 4 - A turn-based loop

while running:
    line = input("> ")
    if line.strip() in {"quit", "exit"}: break
    print(f"you said: {line}")

Each input() blocks until a line arrives. The loop has no fixed rate - its pace is whatever the typist provides. The same shape carries a chess engine (one tick per move), a card game (one tick per play), a discrete-event simulator (one tick per event timestamp). The trigger is “a thing happened”, not “33 ms passed”.

Exercise 5 - Run the tick-budget exhibit

uv run code/measurement/tick_budget.py

Source: code/measurement/tick_budget.py.

          N  layout                  tick (ms)   30 Hz            60 Hz
--------------------------------------------------------------------------------
     10,000  numpy SoA                   0.005   fit  (  0.0%)    fit  (  0.0%)
     10,000  Python dataclass list       0.272   fit  (  0.8%)    fit  (  1.6%)
    100,000  numpy SoA                   0.019   fit  (  0.1%)    fit  (  0.1%)
    100,000  Python dataclass list       2.750   fit  (  8.3%)    fit  ( 16.5%)
  1,000,000  numpy SoA                   0.278   fit  (  0.8%)    fit  (  1.7%)
  1,000,000  Python dataclass list      27.525   fit  ( 82.6%)    OVER (  165%)
 10,000,000  numpy SoA                  16.609   fit  ( 49.8%)    fit  ( 99.7%)
 10,000,000  Python dataclass list   (skipped)   extrapolates over   extrapolates over

The 60 Hz line on 1M dataclass: 165% over budget. The 30 Hz line on 1M dataclass: 82.6% used by one motion system, leaving 5.7 ms for everything else the simulator needs to do per tick. The book is asking you to keep the numpy line because that is the population at which Python becomes feasible. Below 100K entities the layout choice doesn’t matter much; above 100K it determines whether the simulator runs at all.

Exercise 6 - The asyncio comparison

import asyncio, time

TICK_S = 1.0 / 30.0

async def loop():
    start = time.perf_counter()
    end   = start + 10.0
    while time.perf_counter() < end:
        t0 = time.perf_counter()
        # do_work()
        elapsed = time.perf_counter() - t0
        if elapsed < TICK_S:
            await asyncio.sleep(TICK_S - elapsed)

asyncio.run(loop())

Tick rate: same as the synchronous version (~30 Hz). Memory: ~1-2 MB more, for the event loop, the task object, and the awaitable infrastructure. Wall time per tick: 5-20 µs higher because each iteration goes through the event loop’s task-stepping machinery to schedule the next wakeup.

What you got for the cost: nothing. The work is CPU-bound; there are no other awaitables to interleave; the event loop has no useful work to do during the sleep. await asyncio.sleep becomes a slightly more expensive time.sleep. The asyncio scheduler is the right shape for I/O-bound programs (web servers, network clients) where a single thread juggles many waiting connections; it is the wrong shape for a CPU-bound tick loop.

The lesson is the chapter’s: reach for the simplest tool that gives you the property you actually need. Asyncio is correct for many programs. This is not one of them.

Exercise 7 - A discrete-event tick loop (stretch)

import heapq

events: list[tuple[float, str]] = []
heapq.heappush(events, (1.0, "creature_birth"))
heapq.heappush(events, (0.5, "food_spawn"))
heapq.heappush(events, (2.5, "starvation_check"))
heapq.heappush(events, (1.5, "creature_birth"))

clock = 0.0
while events:
    t, msg = heapq.heappop(events)
    clock = t                                   # advance to the event's timestamp
    print(f"t={clock:.2f}  event: {msg}")

t=0.50  event: food_spawn
t=1.00  event: creature_birth
t=1.50  event: creature_birth
t=2.50  event: starvation_check

Two properties to notice:

• The clock advances with the events, not in fixed steps. There is no “tick” between t=0.5 and t=1.0; the simulation simply jumps. Long quiet periods cost nothing.
• No external time reference is needed. Everything is internal - events have timestamps, the clock follows them. This is the discrete-event-simulation (DES) shape that production tools (SimPy, NS-3, OMNeT++, MATLAB Simulink) build on.

§12 - Event time vs tick time names this distinction: the clock the simulation uses doesn’t have to be the clock the loop uses. A 30 Hz time-driven loop with a discrete-event subsystem inside it is a common shape - the outer loop advances the world by 33 ms; the inner DES processes all events with timestamps in the next 33 ms.

12 - Event time is separate from tick time

Concept node: see the DAG and glossary entry 12.

Most beginners assume the loop’s frequency sets the model’s time resolution. If the loop runs at 30 Hz, surely the model can only resolve events at 1/30 s = 33 ms? This is wrong, and the confusion costs many simulations their precision.

The tick rate is how often the loop runs. It says nothing about what the loop does inside one tick. Inside one tick, the loop can process events at arbitrary timestamps - microsecond, picosecond, whatever the data carries. The clock lives on the events, not on the loop.

Concretely: a 30 Hz loop receiving 1,000 events per tick, each with microsecond-precision timestamps, processes them in timestamp order - applying each event’s effect with the precision the timestamp implies. Output to the rest of the world (rendering, logging, network) happens at 30 Hz, but the physics inside runs at microsecond resolution. The tick is a sampling rate; the events are the actual phenomena.

This is the model used by:

• Discrete-event simulators (queueing networks, traffic, supply chains): events fired at exact times.
• Game replay systems (rollback netcode, multiplayer): events arrive late but with their original timestamps.
• Trade execution engines: orders carry nanosecond timestamps; the loop processes them in order.
• Logic simulators in chip design: gate transitions at picosecond resolution; the simulator advances one transition at a time.

In each case, the tick rate of the host loop is irrelevant to the simulation’s resolution. The data carries the time.

How time wants to be stored

The Python reflex when a chapter mentions “timestamps” is to reach for datetime. It is the obvious choice - the standard library provides it, every tutorial uses it, comparisons work with < and >, subtractions return a readable timedelta. It is also one of the most expensive ways to store time at scale.

From code/measurement/event_time_storage.py, one million events covering an hour at microsecond resolution, on this machine:

The headline numbers, both ways:

• 7× smaller footprint moving from datetime list to either typed numpy column. Each datetime instance is ~56 bytes (header, refcount, eight integer fields, pointer); each numpy element is 8 bytes (an int64 micro-since-epoch under datetime64[us], or a float64 second-from-base for the f8 representation).
• ~26× faster count of “how many events happened before time T?” - the per-tick query that decides what gets processed this tick. The numpy versions evaluate the comparison as one bandwidth-bound bulk op; the datetime version pays per-element interpreter dispatch and a < method call.
• Sort time is mixed and dtype-sensitive - measure your specific case. On this run numpy’s float64 sort was slower than its datetime64 sort, which was slightly faster than Python’s Timsort on the already-sorted datetime list. Sort cost matters for ingestion; count cost matters per tick. The tick is the binding budget.

The simlog reference implementation (vendored at .archive/simlog/logger.py) stores time as f8 - float64 seconds. That is the disciplined choice for an event log: small, sortable, amenable to bulk numpy ops, and the same width as everything else in the column store. datetime64[us] is a reasonable alternative when you need to read the timestamps as wall-clock dates without conversion. Use datetime objects only at the boundary - formatting a string for a log line, comparing against a user-supplied timestamp from a request - never as your in-memory storage at simulation scale.

The decoupling, in code

The pitfall is hard-coding the tick interval as the simulation’s clock granularity. Code that says

# anti-pattern: bad!
creature.energy -= 1.0 / 30.0  # "one tick worth of fuel"

is conflating the two clocks. The right shape is

energy[mask] -= elapsed_event_seconds * burn_rate[mask]

using the actual elapsed event-time, not the tick interval. The numpy form is also column-shaped - mask is a boolean filter selecting the affected creatures, burn_rate is per-creature. The same computation works for one event affecting one creature and a thousand events affecting a thousand creatures, because event time and tick time are decoupled. The same model can be sampled at any tick rate the application needs - visualisation at 30 Hz, recording at 60 Hz, fast-forward replay at 1 kHz - without changing what the model means.

This separation is what makes the simulator’s pending_event table possible. Each tick, the loop builds a list of events that should fire - collisions, eats, reproductions - each tagged with its predicted timestamp as an f8. The events fire in timestamp order regardless of which tick they were predicted in. A creature that “would have eaten 2 µs into the tick” has its eat applied at that exact moment, not at the start or end of the tick.

Exercises

These extend the discrete-event loop from §11 exercise 7.

1. A tiny event queue. Use numpy arrays: times = np.array([...], dtype=np.float64) of timestamps and messages = np.array([...], dtype=object) of strings. Push 10 events with random timestamps in [0, 10] seconds. Pop them in time order using order = np.argsort(times). Print each as [t=<sec>] <message>. Verify the output is timestamp-sorted.
2. The wrong way: tick-rate clock. Run a 30 Hz loop. In each tick, advance a counter by 1.0 / 30.0. Use this counter as your “simulation time”. Try to fire an event at t = 0.005 s (5 ms). What happens? When does the event fire? (Hint: 5 ms < 33 ms; the event waits for the next tick boundary, losing 28 ms of resolution.)
3. The right way: timestamp on events. Run the same 30 Hz loop, but each tick pop all events with timestamp ≤ current real time, applied in timestamp order. Fire an event at t = 0.005 s. Show that the event applies at exactly that time, not at the next tick boundary.
4. Sampling at different rates. Run the same model under a 30 Hz loop, then a 60 Hz loop, then a 1 Hz loop. The events should fire at the same simulation times in all three runs (down to whatever precision the loop allows).
5. Float and time. What is the smallest time step np.float32 can represent for events at t ≈ 1 hour? At t ≈ 1 day? At t ≈ 1 year? When do you need np.float64? (See §2. Hint: np.spacing(np.float32(3600)) is a fast way to find the answer for one hour.)
6. Run the storage exhibit. uv run code/measurement/event_time_storage.py. Note the count-time row - that is the per-tick query cost in three layouts. Note where the datetime list lands and where the numpy columns land.
7. (stretch) A budget-aware loop. Modify your 30 Hz loop: at the start of each tick, pop events until either (a) the queue is empty or (b) you have used 25 ms of the 33 ms budget. Defer remaining events to the next tick. This is the soft-real-time pattern used in interactive simulators.

Reference notes in 12_event_time_vs_tick_time_solutions.md.

What’s next

§13 - A system is a function over tables introduces the building block of every tick: the system. Read-set in, write-set out, no hidden state, no surprises.

Solutions: 12 - Event time is separate from tick time

Exercise 1 - A tiny event queue

import numpy as np

rng = np.random.default_rng(0)
times    = rng.uniform(0.0, 10.0, size=10).astype(np.float64)
messages = np.array([f"event_{i}" for i in range(10)], dtype=object)

order = np.argsort(times)
for t, m in zip(times[order], messages[order]):
    print(f"[t={t:.3f}] {m}")

The order array is the only thing that’s “sorted.” times and messages are unchanged. The decoupling pattern from §5: data lives in columns; iteration goes through an index array.

Exercise 2 - The wrong way: tick-rate clock

import time
TICK_S = 1.0 / 30.0

# anti-pattern: bad! "simulation time" advances in tick-sized steps
sim_time = 0.0
target   = 0.005       # 5 ms event
fired    = False

while sim_time < 1.0 and not fired:
    if sim_time >= target:
        print(f"event fired at sim_time={sim_time:.4f}")
        fired = True
    sim_time += TICK_S                  # 33 ms granularity
    time.sleep(TICK_S)

Output: event fired at sim_time=0.0333 - 28 ms late. The event’s “true time” was 5 ms; the sim_time clock cannot resolve below 33 ms because that is the step size. Every event between two tick boundaries gets snapped to the next boundary, losing precision proportional to the tick rate.

This is the conflation the chapter warns against. The 30 Hz tick rate is how often the loop wakes up; it is not the resolution of the model. Hard-coding 1.0/30.0 as the simulation’s time delta makes them the same thing - and pins the simulation’s accuracy to the loop’s wake-up rate.

Exercise 3 - The right way: timestamp on events

import time, heapq, numpy as np

events: list[tuple[float, str]] = []
heapq.heappush(events, (0.005, "early_event"))
heapq.heappush(events, (0.040, "second_event"))
heapq.heappush(events, (0.080, "third_event"))

start = time.perf_counter()
TICK_S = 1.0 / 30.0

while events:
    now = time.perf_counter() - start
    while events and events[0][0] <= now:
        t, msg = heapq.heappop(events)
        print(f"[real={now:.4f}, sim={t:.4f}] {msg}")
    time.sleep(TICK_S)

Each tick processes all events whose timestamp has passed. The 5 ms event fires inside the first tick (the loop has been running for >5 ms by the time the tick finishes). The event’s t is preserved - print(...) shows the original 0.005, not the snapped tick boundary.

The simulator processes the event with its own time, not the loop’s. Same model, sub-tick precision, no overhead beyond a heap pop per event.

Exercise 4 - Sampling at different rates

def run(tick_hz, events_in):
    import time, heapq
    events = list(events_in)
    heapq.heapify(events)
    start = time.perf_counter()
    tick_s = 1.0 / tick_hz
    fired_times = []
    while events:
        now = time.perf_counter() - start
        while events and events[0][0] <= now:
            t, msg = heapq.heappop(events)
            fired_times.append(t)
        time.sleep(tick_s)
    return fired_times

events_in = [(0.005, "a"), (0.040, "b"), (0.080, "c"), (0.150, "d")]
for hz in (30, 60, 1):
    print(f"{hz:>3} Hz fires at: {run(hz, events_in)}")

The list of fired event times is the same in all three runs (modulo floating-point comparison): [0.005, 0.040, 0.080, 0.150]. The 30 Hz, 60 Hz, and 1 Hz runs differ only in how often the loop checked - they all see and apply the same set of events at the same simulation timestamps. The model is sample-rate-independent.

Exercise 5 - Float and time

import numpy as np
print(np.spacing(np.float32(3600)))          # ~2.4e-04 = 244 µs
print(np.spacing(np.float32(86400)))         # ~7.8e-03 = 7.8 ms
print(np.spacing(np.float32(31_536_000)))    # 2.0 s
print(np.spacing(np.float64(31_536_000)))    # ~3.7e-09 = 3.7 ns

float32 runs out of precision fast once the absolute time grows. A simulation that runs for more than a day at sub-millisecond resolution needs float64. This is the §2 catastrophic-cancellation lesson re-applied: precision is a function of the magnitude of the values you’re representing, not just the size of the differences you care about.

Exercise 6 - Run the storage exhibit

uv run code/measurement/event_time_storage.py

Source: code/measurement/event_time_storage.py.

vs 'list of datetime objects', 3-run medians. The **memory** column is stable; **sort** and **count** are timing-noisy, so round figures:

  numpy datetime64[us]                7.0× smaller    ~1× sort       ~26× faster count
  numpy float64 (seconds-from-base)   7.0× smaller    ~0.2× sort     ~26× faster count

The per-tick query is count <T: ~26× faster on float64 vs the datetime list. That is the column the simulator hits every tick to decide what events fire. Sort cost is one-off (ingestion); count cost compounds across millions of ticks. The tick is the binding budget, so the count column is the one to optimise.

Exercise 7 - A budget-aware loop (stretch)

import time, heapq

TICK_S       = 1.0 / 30.0
SOFT_BUDGET  = 0.025                              # 25 ms of the 33 ms tick

events: list[tuple[float, str]] = [...]           # populated from outside
heapq.heapify(events)

while True:
    tick_start = time.perf_counter()
    deadline   = tick_start + SOFT_BUDGET

    processed = 0
    while events and events[0][0] <= time.perf_counter() - tick_start:
        if time.perf_counter() > deadline:
            break                                 # over budget - defer the rest
        t, msg = heapq.heappop(events)
        apply_event(msg)
        processed += 1

    elapsed = time.perf_counter() - tick_start
    if events and elapsed > SOFT_BUDGET:
        print(f"deferred {len(events)} events; tick used {elapsed*1000:.1f}ms")

    sleep_for = TICK_S - elapsed
    if sleep_for > 0:
        time.sleep(sleep_for)

This is the soft real-time pattern: the loop prefers to process every due event each tick, but guarantees it will return within budget. Surplus events spill into the next tick.

This shape is what runs game engines, animation systems, and interactive simulators. It is also what the simulator’s §35 - boundary is the queue builds on - events at the edge of the tick belong to the next tick’s queue, not this one’s stretch goal.

The pattern fails gracefully when overloaded: latency degrades but the loop continues. The alternative - process every event whatever it costs - fails catastrophically when overloaded: the loop blows its tick budget, drops the next deadline, and either loses real-time properties or piles up an ever-growing deficit.

13 - A system is a function over tables

Concept node: see the DAG and glossary entry 13.

A system is a function that reads from one or more tables and writes to one or more tables. It declares its inputs (the read-set) and its outputs (the write-set). It has no hidden state, no global side effects, no interaction with the outside world during a tick. The signature is the contract.

def motion(pos_x: np.ndarray, pos_y: np.ndarray,
           vel_x: np.ndarray, vel_y: np.ndarray,
           dt: float) -> None:
    pos_x += vel_x * dt
    pos_y += vel_y * dt

Read-set: vel_x, vel_y, dt. Write-set: pos_x, pos_y. That is the entire contract. This system can run any time those four columns and dt are available and nothing else is writing pos_x or pos_y. It runs once per tick over the whole population - there is no per-creature loop in the body. The for-loop disappeared into numpy.

Three shapes

Every system takes one of three shapes.

An operation is 1→1: every input row produces exactly one output row. motion is an operation - each creature’s position is updated to its new position. Most update functions are operations.

A filter is 1→{0, 1}: every input row produces zero or one output rows. apply_starve (from code/sim/SPEC.md) is a filter - each creature with energy ≤ 0 produces an entry in to_remove; creatures with energy > 0 produce nothing. The numpy form is one line:

def starving(energy: np.ndarray) -> np.ndarray:
    return np.where(energy <= 0)[0]  # returns the indices to remove

An emission is 1→N: every input row produces zero or more output rows. apply_reproduce is an emission - a parent above the energy threshold produces two offspring (a 1→2 emission).

These three shapes are the same shapes a database query takes. SELECT * FROM t WHERE p is a filter, SELECT a + b FROM t is an operation, SELECT explode(arr) FROM t is an emission. A system is a database operation written in Python against numpy columns instead of SQL against tables. If you have ever written SQL, you already know the vocabulary; the work is recognising your simulation in those terms.

Return type is half the contract

The three shapes also fix the return type. An operation mutates its write-set in place and returns None - the work has already happened by the time the call returns. A filter returns a new array of indices. An emission returns one or more new arrays. The pattern is: mutators return None; producers return the thing they produced.

The reason for the asymmetry is that the alternative - having a mutator return its own write-set - is a silent aliasing bug. world = step(world) reads like it produces a new world, but if step mutates and returns the same object, both names point at the same state and the caller cannot tell from the call site which is true. Python’s standard library encodes the rule exactly: list.sort() returns None so that xs = xs.sort() fails loudly; sorted() returns a new list. The system convention is the same rule applied to columns.

There is one named exception: a function that builds a world from nothing - from a seed, a file, or a log - returns the new world. Its signature gives it away: it does not take an existing world to mutate. build_world(seed), load(path), replay(initial, log) are constructors, not systems. The return value is the only place the new state can go.

The OOP method is the anti-shape

This is the moment to name what most Python tutorials teach instead. The method-on-object shape - class Creature: def tick(self, dt): self.pos += self.vel * dt - is the same lesson rotated through self, and the rotation costs you everything important. The signature def tick(self, dt) does not tell you what the method reads or writes. The body does, but only after you read it. The contract is no longer expressible at the call site; it is implicit in the body of the method, which means you cannot reason about composition without inlining every method.

It also costs you the loop. The natural caller for Creature.tick is for c in creatures: c.tick(dt) - a Python-level loop, one method dispatch per element, interpreter-bound at the floor of ~5 ns per element from §1, plus another ~50-100 ns of getattr and method-call overhead per attribute. From code/measurement/tick_budget.py the cost is 27.9 ms per tick at 1,000,000 creatures for one motion system, against 0.6 ms for the function-over-columns form. The system shape is not just clearer - it is the only one that fits inside a 30 Hz budget at scale.

The wider rule: a function that takes self does not have a declared read-set or write-set. A function that takes columns does. This is one of the two or three places where “OOP versus data-oriented” is not a stylistic choice - it is whether your system has a contract you can read.

Logging is a separate system

The other reflex Python encourages is to write to stdout from inside the loop. print(f"creature {i} starved"), logger.info(...), traceback.print_exc() - all of these are side effects that violate the system’s no-hidden-output contract. The fix is the same shape as everything else in this book: there is a log_events table, a logging system writes to it, and a separate flush system writes the table to disk or stdout.

The book builds this discipline at §37 - The log is the world. For now, the rule is: if a system needs to communicate with the outside, it does so through a column declared in its write-set. There are no surprise prints.

Observability and tests are systems too

A debug inspector is a system whose read-set is “all relevant columns” and whose write-set is “nothing observable” - it gathers data for inspection and produces no side effects on the world. In production it is absent, not gated by a flag - the program simply does not contain it.

A test is also a system. assert pos.shape == vel.shape and not np.any(np.isnan(pos)) is a system whose read-set is pos and vel, write-set is nothing, and whose effect is to fail loudly if the contract of the previous system was violated. Tests-as-systems is the §43 topic, but you have been writing them since §5 exercise 1.

A system declares its inputs, declares its outputs, and does no more. That is the shape that lets every other discipline in the book work.

A few patterns to watch for

A function that reads a column, writes to it, and reads it again in the same call is not a system - it has implicit ordering inside the body. Either split it into two systems with explicit ordering, or buffer the writes until the function exits. A function that takes a world object and mutates whatever it likes is not a system - it has no declared write-set, and you cannot reason about it from its signature.

The contract that the system has no hidden state is what makes systems compose. Two systems with disjoint write-sets can run in parallel without coordination (§31). Two systems whose read-set and write-set form a chain must run in order (§14). The contract is the basis for all of this.

Exercises

Use the deck from §5, your tick_lab from §11, or the §0 simulator skeleton; any of them provides enough tables.

1. Identify the shape. Classify each as operation, filter, or emission:
   • Squaring every entry in a np.ndarray of float32.
   • Filtering even integers from a np.ndarray of int32.
   • Splitting each string in a list[str] into words, returning all words.
   • Computing the sum of a np.ndarray of int32.
2. Write motion as a system. With pos_x, pos_y, vel_x, vel_y as numpy float32 columns of length 100, write motion(pos_x, pos_y, vel_x, vel_y, dt) as defined in the prose. Apply it to 100 creatures with random initial positions and velocities. Print the position of one creature across 10 ticks. The body is two lines.
3. Declare the contract. Add a docstring to motion listing its read-set and write-set explicitly. The signature plus the docstring is the system’s contract.
4. Write a filter. With energy: np.ndarray, write starving(energy) returning a numpy array of indices where energy[i] <= 0. This is the read-only first half of apply_starve.
5. Write an emission. With parent_energy: np.ndarray, threshold threshold: float, write reproduce(parent_energy, threshold) returning two parallel arrays - parent_indices and offspring_energies - for each parent above threshold, with two entries each. This is a 1→2 emission. (Hint: mask = parent_energy > threshold; idx = np.where(mask)[0]; np.repeat(idx, 2).)
6. Observe non-systems. Find a function in your previous work (or any Python tutorial) that takes self and mutates whatever it likes, or writes to a global, or calls print from inside the body. Note what makes it not a system. Try to express its read-set and write-set from the signature alone - confirm you cannot.
7. The OOP cost in your fingers. Run uv run code/measurement/tick_budget.py. Read the table. Note that you have just seen, at 1,000,000 creatures, what happens when the loop is in the body of a method instead of in numpy. The 30 Hz row is over for the Python dataclass version. The system-shaped version uses 1.8% of the budget.
8. (stretch) A test as a system. Write def no_creature_moved_too_far(prev_pos_x, prev_pos_y, cur_pos_x, cur_pos_y, max_step) returning indices where any creature moved further than max_step between two ticks. The “test” is just an inspection system reading the world. Hint: dx = cur_pos_x - prev_pos_x; dy = cur_pos_y - prev_pos_y; np.where(dx*dx + dy*dy > max_step*max_step)[0].

Reference notes in 13_system_as_function_solutions.md.

What’s next

§14 - Systems compose into a DAG takes the next step: when many systems run together, how do they fit?

Solutions: 13 - A system is a function over tables

Exercise 1 - Identify the shape

Reductions deserve a footnote: they collapse a column into a scalar. They are systems too - read-set is the column, write-set is one scalar. The book mostly uses reductions inline (sum, max, count_nonzero) rather than as named systems, but the contract still applies.

Exercise 2 - Write motion as a system

import numpy as np

def motion(pos_x, pos_y, vel_x, vel_y, dt):
    pos_x += vel_x * dt
    pos_y += vel_y * dt

rng = np.random.default_rng(0)
n = 100
pos_x = rng.uniform(0, 10, n).astype(np.float32)
pos_y = rng.uniform(0, 10, n).astype(np.float32)
vel_x = rng.uniform(-1, 1, n).astype(np.float32)
vel_y = rng.uniform(-1, 1, n).astype(np.float32)

dt = 1.0 / 30.0
for t in range(10):
    print(f"t={t}: creature 17 at ({pos_x[17]:.3f}, {pos_y[17]:.3f})")
    motion(pos_x, pos_y, vel_x, vel_y, dt)

Two lines in the body. No per-creature loop, no method dispatch, no self. The += operator on a numpy column is a single C-level pass.

Exercise 3 - Declare the contract

def motion(pos_x, pos_y, vel_x, vel_y, dt):
    """Advance every creature's position by one tick of motion.

    Read-set:  vel_x, vel_y, dt
    Write-set: pos_x, pos_y     (in-place)
    Contract:  pos_*.shape == vel_*.shape; arrays are float32 columns.
    """
    pos_x += vel_x * dt
    pos_y += vel_y * dt

The signature plus the docstring is the entire contract. A reader of motion does not need to inline the body to know it does not touch energy or birth_t; the docstring says so. The §14 DAG construction reads exactly this declaration to schedule the system.

A test that the contract is honest (a §43 test-as-system) compares the declared write-set to the columns the function actually mutated. If motion ever silently writes to energy, the test catches it.

Exercise 4 - Write a filter

def starving(energy: np.ndarray) -> np.ndarray:
    """Return indices of creatures with energy <= 0.

    Read-set: energy
    Write-set: nothing  (returns indices for a separate apply step)
    """
    return np.where(energy <= 0)[0]

energy = np.array([3.0, -1.0, 5.0, 0.0, 7.0], dtype=np.float32)
print(starving(energy))     # [1 3]

The filter is read-only. It returns the indices that satisfy the predicate; a separate “apply” system writes them into to_remove. This separation is the §22 mutations buffer discipline applied at the smallest scale: filter and apply are separate systems with different read-sets and write-sets.

Exercise 5 - Write an emission

def reproduce(parent_energy: np.ndarray, threshold: float):
    """For each parent above threshold, produce two offspring (1→2 emission).

    Read-set: parent_energy, threshold
    Write-set: nothing  (returns parallel arrays for the apply step)
    Returns:
        parent_indices: which parent each offspring came from (length 2*K)
        offspring_energies: starting energy for each offspring (length 2*K)
    """
    mask = parent_energy > threshold
    idx  = np.where(mask)[0]
    parent_indices    = np.repeat(idx, 2)                    # parent appears twice
    offspring_energies = np.repeat(parent_energy[idx] / 2, 2) # half-energy each
    return parent_indices, offspring_energies

energies = np.array([3.0, 7.0, 1.0, 9.0, 5.0, 11.0], dtype=np.float32)
p, o = reproduce(energies, 5.0)
print(f"parents:    {p}")              # [1 1 3 3 5 5]
print(f"offspring:  {o}")              # [3.5 3.5 4.5 4.5 5.5 5.5]

np.repeat(arr, 2) is the emission primitive: each input row produces two output rows in column form. For a 1→N emission with variable N per row, np.repeat(arr, counts) takes a per-row count array. The shape is “filter, then expand”; the apply system later inserts the rows into the table.

Exercise 6 - Observe non-systems

A canonical non-system from the wild:

class GameObject:
    def update(self):
        self.pos += self.vel * GLOBAL_DT
        if self.energy <= 0:
            print(f"{self.name} died")          # side effect - not in any signature
            self.dead = True
            World.remove(self)                  # mutates global state
        for nearby in World.find_nearby(self):  # reads global state
            self.energy += nearby.value

What the signature def update(self) declares: nothing. What the body actually does:

• Reads self.pos, self.vel, self.energy, self.name, World.objects (implicit, through find_nearby)
• Reads global GLOBAL_DT
• Writes self.pos, self.dead, self.energy, World.objects (implicit, through World.remove)
• Writes stdout

You cannot tell any of this from the signature. To compose update with another system, you’d have to inline the body and trace every method call. Two update calls cannot run in parallel because both write World.objects. Tests cannot mock the read-set without mocking the world. The function has no contract anyone can read; it has behaviour, which is not the same thing.

Exercise 7 - The OOP cost in your fingers

uv run code/measurement/tick_budget.py

The 1M-creatures row:

The dataclass form has one motion system eating 82.6% of the 30 Hz budget; the simulator has 5.7 ms left for everything else (collision, energy, reproduction, rendering). At 60 Hz the loop has already missed its deadline. The system-as-function-over-numpy form runs the same logic in 0.278 ms and leaves 32.7 ms (98%) of the 30 Hz budget for the rest of the simulator.

The 100× cost gap is the cost of putting the per-creature loop inside the interpreter instead of inside numpy. There is no syntactic refactor of the OOP version that closes this gap - the cost is structural.

Exercise 8 - A test as a system (stretch)

def no_creature_moved_too_far(prev_pos_x, prev_pos_y,
                              cur_pos_x, cur_pos_y, max_step) -> np.ndarray:
    """Return indices of creatures that moved further than max_step between ticks.

    Read-set:  prev_pos_*, cur_pos_*, max_step
    Write-set: nothing  (the caller decides whether to assert, log, or correct)
    """
    dx = cur_pos_x - prev_pos_x
    dy = cur_pos_y - prev_pos_y
    return np.where(dx * dx + dy * dy > max_step * max_step)[0]


# Used as an assertion in a tick:
violators = no_creature_moved_too_far(prev_x, prev_y, x, y, 1.0)
assert violators.size == 0, f"creatures {violators} teleported"

This is a system by the chapter’s definition: declared read-set, no write-set, no hidden state. Its presence in the program is what an invariant looks like - the rest of the program is required to keep no_creature_moved_too_far returning an empty array. Failing this test is the simulator telling you the motion system has a bug.

The Rust edition would write this as a fn taking slices; the only difference is that Python tests run inside the same loop while Rust tests usually run as #[cfg(test)] builds. The discipline is identical - the test is a system over the same tables, with the same contract shape, that happens to report rather than transform. §43 - tests are systems generalises this.

14 - Systems compose into a DAG

Concept node: see the DAG and glossary entry 14.

A program with one system is uninteresting; a program with many systems must say what runs in what order. The order is given by data dependencies: a system that reads a table must run after every system that writes that table within the same tick. No ordering is fixed by intuition; everything is given by the read-sets and write-sets §13 just made you declare.

Draw the dependency graph. Each system is a node. For every system that reads table T and every system that writes T, draw an edge writer → reader. The result is a directed acyclic graph - the DAG. A topological sort gives a valid execution order: any sort that respects the edges is correct. The program executes one such sort.

The simulator’s tick from code/sim/SPEC.md:

flowchart TB
    food_spawn --> motion
    motion --> next_event
    next_event --> apply_eat
    next_event --> apply_reproduce
    next_event --> apply_starve
    apply_eat --> cleanup
    apply_reproduce --> cleanup
    apply_starve --> cleanup
    cleanup --> inspect

food_spawn runs first because its output is food, which motion and next_event read. next_event produces pending_event, which the three appliers consume in parallel (their write-sets are disjoint). cleanup runs after all of them because its read-set includes their writes. inspect runs last because it reads everything and writes nothing.

This is the same shape as a query plan in a database. The query optimiser takes a SQL statement, builds a graph of relational operations (each one a system!), and topo-sorts them into an execution plan. A simulator is a query plan running every tick. Students who follow this thread end up writing their own minimal query engine without realising it.

Not callbacks. Not signals. Not pub/sub.

This is where the Pythonic “loose coupling” idioms come asking, and the right answer is to refuse them. Three patterns to name and exclude:

Observers / event buses. A system subscribes to an event (“a creature was born”) and runs some handler. The order in which handlers fire is whoever subscribed first, or whatever the framework picks, or - most commonly - unspecified by design. This is the opposite of what this chapter is asking for. The DAG fixes order; an event bus deliberately does not.

Django/Flask-style signals. Frameworks teach signal.connect(handler) so that any module can wire itself into any lifecycle point. The result is a tick whose execution order depends on which modules were imported, in what order, and which connect calls ran. The DAG depends on declared data dependencies; signals depend on import order.

Callbacks. A system “calls back” to user code at some point in its body. Now the user code is part of the tick, but it has no declared read-set, no declared write-set, and runs at a moment determined by the implementation of the calling system. The contract from §13 is gone.

In all three cases the problem is the same: order is not declared; it is emergent from runtime accidents. A reader that runs before its writer reads stale data - yesterday’s snapshot of a table that was supposed to have been updated. A reader that runs after its consumer reads garbage - a half-written table mid-update. The DAG is the contract that prevents both. Each of the three patterns above replaces the contract with a hope.

A simulator’s tick is a topologically-sorted call list:

def tick(world: World, dt: float) -> None:
    food_spawn(world.food, dt)
    motion(world.pos_x, world.pos_y, world.vel_x, world.vel_y, dt)
    next_event(world.pending_event, world.pos_x, world.pos_y, world.food, ...)
    apply_eat(world.energy, world.food, world.pending_event)
    apply_reproduce(world.to_insert, world.energy, world.pending_event)
    apply_starve(world.to_remove, world.energy, world.pending_event)
    cleanup(world.to_remove, world.to_insert, ...)
    inspect(world)  # read-only, write-set empty

Eight function calls, in topological order. Adding a system means adding a line and re-deriving the order from the new system’s read-set and write-set. There is no register(), no subscribe(), no signal.connect(). The sequence is the program; the program is the sequence.

Why acyclic

A cycle is a contradiction. Suppose system A writes table T, system B reads T and writes U, system A reads U. Now A both produces T (which B reads) and consumes U (which B writes). A and B cannot both run before each other in the same tick.

flowchart LR
    A -->|"T (A writes, B reads)"| B
    B -->|"U (B writes, A reads)"| A

A cycle in the system graph is a design bug; it must be broken - usually by buffering one system’s write so it is consumed next tick instead of this tick. That buffering is exactly what §15 - State changes between ticks names. Cycles do not disappear when you write a simulation; they get a name and a discipline.

Parallelism for free

Once the DAG is explicit, parallelism becomes trivial. Any two systems on the same DAG level - neither one a transitive dependency of the other - can run on different processes. In the simulator above, apply_eat, apply_reproduce, and apply_starve all consume pending_event and produce disjoint output tables (energy / food, to_insert, to_remove); they can run in parallel without coordination. The schedule is implied by the graph. §31 picks this up under the GIL.

The observer-pattern alternative cannot offer this. Without an explicit DAG, the framework cannot tell which handlers are independent and which are not - so it either runs everything serially or relies on the user to add manual synchronisation. The DAG-first design gets parallelism for free the moment the read-sets and write-sets are accurate; the observer-first design has to invent it back.

Exercises

1. Draw the DAG. Take the eight simulator systems (motion, food_spawn, next_event, apply_eat, apply_reproduce, apply_starve, cleanup, inspect) and draw the dependency graph yourself, deriving the edges from each system’s read-set and write-set in code/sim/SPEC.md. Compare with the diagram above.

2. Spot the cycle. Suppose apply_starve writes to food (returning fuel to the world when a creature dies). Now apply_starve writes food, which food_spawn reads. food_spawn writes food, which next_event reads. next_event writes pending_event, which apply_starve reads. Where’s the cycle? How would you break it? (Hint: §15.)

3. Topological sort by hand. Given:

   • A writes X
   • B reads X, writes Y
   • C reads X, writes Z
   • D reads Y and Z, writes W

   Which systems can run in parallel? What’s a valid execution order? Are there multiple valid orders?

4. Topological sort in Python. Implement def topo_sort(systems: list[tuple[str, set[str], set[str]]]) -> list[str] taking (name, read_set, write_set) triples and returning a valid execution order. Use Kahn’s algorithm. Apply it to your answer to exercise 1 - it should produce the same ordering (or one of the valid alternatives).

5. Compose two systems. Write motion (operation, writes pos_x, pos_y) and next_event (operation, writes pending_event). Wire them into a tick(world, dt) function that calls them in order. Inspect pending_event after the tick.

6. Add cleanup. Add a cleanup system that processes to_remove and to_insert (both initially empty arrays). Wire it after next_event. Confirm the call list reads top-to-bottom in dependency order.

7. The wrong way: an observer. Implement the same three-system tick using an event-bus pattern: bus.subscribe("tick", motion); bus.subscribe("tick", next_event); bus.subscribe("tick", cleanup); bus.fire("tick", world). Run it. Note that the order is now implicit in registration order, and any new subscriber inserted at runtime can change the order silently. Compare reading the resulting code to reading the function-call form. Which one tells you what runs when?

8. (stretch) A query planner. Take five hand-written SQL queries (each one a system shape) and draw the relational-algebra plan for each. Compare with how motion → next_event → apply_* decomposes the simulator. The shape is the same.

Reference notes in 14_systems_compose_into_a_dag_solutions.md.

What’s next

§15 - State changes between ticks is the rule that makes the DAG actually work: mutations buffer; the world transitions atomically.

Solutions: 14 - Systems compose into a DAG

Exercise 1 - Draw the DAG

Reading each system’s read-set and write-set:

food_spawn       reads {}                                         writes {food}
motion           reads {vel_x, vel_y, food}                       writes {pos_x, pos_y}
next_event       reads {pos_x, pos_y, food}                       writes {pending_event}
apply_eat        reads {pending_event}                            writes {energy_delta}
apply_reproduce  reads {pending_event}                            writes {to_insert}
apply_starve     reads {pending_event}                            writes {to_remove}
cleanup          reads {to_remove, to_insert, energy_delta}       writes {next_state}
inspect          reads {pos_x, pos_y, energy, food, ids}          writes {}

Edges (writer → reader):

food_spawn → motion             (food)
food_spawn → next_event         (food)
food_spawn → inspect            (food)
motion     → next_event         (pos_x, pos_y)
motion     → inspect            (pos_x, pos_y)
next_event → apply_eat          (pending_event)
next_event → apply_reproduce    (pending_event)
next_event → apply_starve       (pending_event)
apply_eat       → cleanup       (energy_delta)
apply_reproduce → cleanup       (to_insert)
apply_starve    → cleanup       (to_remove)

This matches the chapter’s diagram. The three appliers form a “fan-out”; cleanup is the “fan-in” that consumes their outputs.

Exercise 2 - Spot the cycle

If apply_starve writes food (returning fuel when a creature dies), the chain becomes:

food_spawn → next_event → apply_starve → food_spawn? (already ran this tick!)

food_spawn writes food; apply_starve reads pending_event (from next_event, which reads food from food_spawn); apply_starve writes food - but food_spawn already wrote food earlier this tick. The cycle is:

food_spawn → next_event → apply_starve → food_spawn   (back-edge: both write `food`)

A cycle of writers to the same column is the same-tick contradiction the chapter warns against. Break it by buffering: apply_starve writes to a food_returns buffer; food_spawn next tick reads food_returns and incorporates it into the new food table. The cycle becomes a tick boundary - the §15 mutations buffer discipline.

Exercise 3 - Topological sort by hand

A writes X
B reads X, writes Y
C reads X, writes Z
D reads Y and Z, writes W

Dependencies:

• B depends on A (X)
• C depends on A (X)
• D depends on B (Y) and C (Z)

Parallelism: B and C have the same predecessor (A) and disjoint write-sets (Y vs Z). They can run in parallel.

Valid execution orders:

• A, B, C, D
• A, C, B, D
• A, {B || C}, D (B and C concurrent)

All three are correct; the schedule chooses one. Multiple valid sorts is the norm - any sort respecting the edges is correct, and the DAG itself does not pick.

Exercise 4 - Topological sort in Python (Kahn’s algorithm)

def topo_sort(systems: list[tuple[str, set[str], set[str]]]) -> list[str]:
    """Kahn's algorithm. systems = [(name, read_set, write_set), ...]"""
    writers: dict[str, set[str]] = {}
    for name, _, ws in systems:
        for t in ws:
            writers.setdefault(t, set()).add(name)

    edges:  dict[str, set[str]] = {name: set() for name, _, _ in systems}
    in_deg: dict[str, int]      = {name: 0    for name, _, _ in systems}

    for name, rs, _ in systems:
        for t in rs:
            for w in writers.get(t, ()):
                if w != name and name not in edges[w]:
                    edges[w].add(name)
                    in_deg[name] += 1

    queue = sorted(n for n, d in in_deg.items() if d == 0)
    order: list[str] = []
    while queue:
        queue.sort()                                  # deterministic across runs
        n = queue.pop(0)
        order.append(n)
        for m in sorted(edges[n]):
            in_deg[m] -= 1
            if in_deg[m] == 0:
                queue.append(m)

    if len(order) != len(systems):
        raise ValueError("cycle in DAG")
    return order


# Apply to the sim DAG (with cleanup writing to a buffer to break the cycle from §2)
sim = [
    ("food_spawn",     set(),                                     {"food"}),
    ("motion",         {"vel_x","vel_y","food"},                  {"pos_x","pos_y"}),
    ("next_event",     {"pos_x","pos_y","food"},                  {"pending_event"}),
    ("apply_eat",      {"pending_event"},                         {"energy_delta"}),
    ("apply_reproduce",{"pending_event"},                         {"to_insert"}),
    ("apply_starve",   {"pending_event"},                         {"to_remove"}),
    ("cleanup",        {"to_remove","to_insert","energy_delta"},  {"next_state"}),
    ("inspect",        {"pos_x","pos_y","energy","ids","food"},   set()),
]

print(topo_sort(sim))
# ['food_spawn', 'motion', 'inspect', 'next_event', 'apply_eat',
#  'apply_reproduce', 'apply_starve', 'cleanup']

A valid order. inspect lands earlier than the chapter diagram suggests because it has no consumers - Kahn’s algorithm pulls it as soon as its read-set is satisfied. Both placements (right after motion or right at the end) are correct topological sorts.

For exercise 3:

sys2 = [("A", set(), {"X"}),
        ("B", {"X"}, {"Y"}),
        ("C", {"X"}, {"Z"}),
        ("D", {"Y","Z"}, {"W"})]
print(topo_sort(sys2))     # ['A', 'B', 'C', 'D']

Exercise 5 - Compose two systems

import numpy as np

class World:
    def __init__(self, n):
        rng = np.random.default_rng(0)
        self.pos_x = rng.uniform(0, 10, n).astype(np.float32)
        self.pos_y = rng.uniform(0, 10, n).astype(np.float32)
        self.vel_x = rng.uniform(-1, 1, n).astype(np.float32)
        self.vel_y = rng.uniform(-1, 1, n).astype(np.float32)
        self.pending_event = []          # list of (timestamp, kind, idx)


def motion(w: World, dt: float) -> None:
    w.pos_x += w.vel_x * dt
    w.pos_y += w.vel_y * dt


def next_event(w: World) -> None:
    w.pending_event.clear()
    # toy: an event for whichever creature is closest to (0, 0)
    d2 = w.pos_x ** 2 + w.pos_y ** 2
    i  = int(np.argmin(d2))
    w.pending_event.append((float(d2[i]), "closest", i))


def tick(w: World, dt: float) -> None:
    motion(w, dt)
    next_event(w)


w = World(100)
tick(w, 1.0 / 30.0)
print(w.pending_event)        # one event per tick

The tick is two function calls in topological order. The DAG is two nodes, one edge (motion → next_event via pos_x/pos_y).

Exercise 6 - Add cleanup

def cleanup(w: World) -> None:
    # toy: drop the closest-to-origin creature (an "eaten" event)
    if w.pending_event:
        _, _, i = w.pending_event[0]
        keep = np.ones(len(w.pos_x), dtype=bool)
        keep[i] = False
        w.pos_x = w.pos_x[keep]
        w.pos_y = w.pos_y[keep]
        w.vel_x = w.vel_x[keep]
        w.vel_y = w.vel_y[keep]


def tick(w: World, dt: float) -> None:
    motion(w, dt)
    next_event(w)
    cleanup(w)


w = World(100)
for _ in range(10):
    tick(w, 1.0 / 30.0)
print(f"after 10 ticks: {len(w.pos_x)} creatures left")    # 90

Three function calls, top to bottom in dependency order. Adding a fourth system means writing one line and re-running topo_sort if the order is non-trivial. There is no register(), no subscribe(). The sequence is the program; the program is the sequence.

Exercise 7 - The wrong way: an observer

class EventBus:
    def __init__(self):
        self.subs: dict[str, list] = {}
    def subscribe(self, event, handler):
        self.subs.setdefault(event, []).append(handler)
    def fire(self, event, *args, **kwargs):
        for h in self.subs.get(event, []):
            h(*args, **kwargs)

bus = EventBus()
bus.subscribe("tick", motion)
bus.subscribe("tick", next_event)
bus.subscribe("tick", cleanup)

w = World(100)
bus.fire("tick", w, 1.0 / 30.0)        # works - but only because we registered in order

Three subtle problems with this version:

1. Order is implicit in registration order. Swap the two subscribe lines for next_event and motion - the program runs without error, with stale data. There is no signal that the order is wrong.
2. A new subscriber inserted at runtime can change the order silently. Some plugin loads at startup, calls bus.subscribe("tick", validate_invariants), and inserts itself in the middle. The loop now runs in a different order; whether that’s correct depends entirely on the plugin’s read/write set, which the bus doesn’t know.
3. Reading the program is harder. To know what bus.fire("tick", ...) does, you must find every bus.subscribe("tick", ...) call across the entire codebase, in import order. Compare to def tick(w, dt): motion(w, dt); next_event(w); cleanup(w) - three lines, locally readable, ordering visible.

The function-call form tells you what runs when. The bus form tells you what can run when. The DAG-explicit version is the one that can be reasoned about, parallelised, tested, and trusted.

Exercise 8 - A query planner (stretch)

Take five SQL queries and decompose into relational-algebra operators:

-- Query 1: "active users by country, top 10"
SELECT country, COUNT(*) AS n FROM users WHERE active = TRUE
GROUP BY country ORDER BY n DESC LIMIT 10;

Plan:

LIMIT(10,
  SORT(n DESC,
    AGGREGATE(GROUP BY country, COUNT(*),
      FILTER(active = TRUE, SCAN(users)))))

Each level is a system in the chapter’s sense:

• SCAN reads the underlying table, writes a stream of rows
• FILTER reads the stream + predicate, writes a filtered stream
• AGGREGATE reads the stream, writes grouped rows
• SORT reads grouped rows, writes ordered rows
• LIMIT reads ordered rows, writes prefix

Each operator declares its read-set (the input stream) and write-set (the output stream); the plan is a topo-sorted DAG. Database optimisers explore alternative plans (a different join order, an index scan instead of a full scan), pick the cheapest, and execute.

A simulator does the same thing every tick, but with the plan fixed at design time rather than chosen by an optimiser. Students who write five small plans by hand notice that a tick-loop and a query plan are the same shape: a DAG of small operators consuming and producing tables.

15 - State changes between ticks

Concept node: see the DAG and glossary entry 15.

Inside a tick, the world is frozen. Systems read consistent snapshots of their inputs; mutations are queued, not applied; only at the tick boundary does the world step forward in one atomic transition.

This is the rule that makes the DAG from §14 actually work. If motion could mutate pos while next_event is reading pos, the data is inconsistent: half the creatures have moved, half have not. Even if the schedule is “correct” by topological order, what each system reads is no longer well-defined. By forbidding mutations to apply in-tick, the world becomes a clean function world_{t+1} = step(world_t, inputs_t). Every system reads world_t; every system writes into a buffer that becomes world_{t+1} only at the tick boundary.

Concretely: apply_starve does not call np.delete(creatures, slot) or pop from a Python list. It writes the doomed slot into to_remove. The creatures columns are unchanged for the rest of the tick. After every system has run, cleanup consumes to_remove and to_insert together, applying every queued change in one sweep. Now the next tick begins with a consistent new world state.

This pattern is called double buffering: there is the world the systems read (world_t), and the buffer of changes that becomes the world the next tick reads (world_{t+1}). The pattern shows up everywhere - graphics frame buffers, database transactions, event-sourced systems. The rule is always the same: writes accumulate, then commit.

The Python footguns this rule prevents

Python has two famous in-place-mutation footguns the discipline above eliminates.

The list-during-iteration bug. Removing from a list while iterating it silently skips elements. The iterator advances by index; list.remove shifts everything down by one; the next element is now at the index the iterator already passed:

# anti-pattern: bad!
creatures = [c1, c2, c3, c4, c5]   # all five starving
for c in creatures:
    if c.energy <= 0:
        creatures.remove(c)        # skips c2 and c4 - they survive
# Surviving creatures: 2 out of 5. The starvation system is broken
# and the simulation will run forever.

The dict-during-iteration bug. Removing from a dict while iterating raises:

# anti-pattern: bad!
for cid, c in creatures.items():
    if c.energy <= 0:
        del creatures[cid]
# RuntimeError: dictionary changed size during iteration

The list version is the dangerous one - it fails silently and hands you a wrong-but-finite simulation. The dict version is dangerous in a different way: the RuntimeError trains the reader to fix it locally (for cid in list(creatures.keys()):) without ever recognising the structural problem. Both are the same lesson: mutating a container while another piece of code is reading it is the bug, regardless of whether the language catches it.

The disciplined Python equivalent in numpy is one boolean mask per buffer:

def apply_starve(energy: np.ndarray, ids: np.ndarray, to_remove: list[int]) -> None:
    starvers = np.where(energy <= 0)[0]          # read-only scan -> slots
    to_remove.extend(ids[starvers].tolist())      # buffer the *ids*, not the slots

def cleanup(world: World, to_remove: list[int], to_insert: list[CreatureRow]) -> None:
    # apply removals first (swap_remove pattern, §21), then inserts
    ...

to_remove holds ids, not slots. Slots move when rows are deleted or sorted; ids do not, so the buffer stays valid for cleanup to resolve through id_to_slot (§22, §23). The starvation system only writes to to_remove. It never touches creatures. The creatures columns are unchanged when apply_starve returns - they are unchanged when apply_eat and apply_reproduce return. They are mutated exactly once per tick, by cleanup, after every other system is done. There is no window in which a system could see an inconsistent world.

What this looks like in code

The removal case buffered which slots to drop. The far more common case, a value that changes every tick, buffers how much it changed - and when several systems change the same value, each writes its own buffer and cleanup combines them in one commit:

energy                  = np.array([1, 2, 3, 4, 5])     # world_t: every system reads this
energy_used             = np.array([-1, -1, 0, 0, -1])  # motion + apply_starve write here
energy_gained_from_food = np.array([1, 2, 0, 0, 3])     # apply_eat writes here

# ... the systems run during the tick. Each reads `energy` and writes only
#     its own buffer - never `energy`, never another system's buffer ...

# tick boundary - one atomic commit:
energy += energy_used + energy_gained_from_food         # energy is now world_{t+1}

No system reads a half-updated energy, because no system writes to it. motion and apply_starve write only energy_used; apply_eat writes only energy_gained_from_food. The buffers are separate, so the systems never collide and can run in parallel (§14); cleanup sums them into energy in one step at the boundary. That is the whole rule: each system owns a buffer, structural edits go in to_remove/to_insert, value edits go in a delta column, and the writes combine exactly once.

When the commit must read the old values while computing the new ones - a diffusion step, an averaging filter, anything where element i depends on its neighbours - the in-place += is unsafe, and you write into a second array and swap the two names at the boundary instead. That swap is the literal double buffer the production logger (exercise 8) is built on: read one array, write the other, exchange them between ticks. The delta-add above is its cheapest special case, for when each element’s change is independent of the rest.

Costs and trade

Two costs to absorb. First, every mutation is one extra entry pushed to a to_remove or to_insert buffer. Second, the cleanup pass is now its own system in the DAG. The benefit dwarfs the costs: every other system in the book composes cleanly, and parallelism becomes easy. With in-tick mutation, every parallel scheduling decision becomes a race condition. With buffered mutation, races are structurally impossible - disjoint write-sets are disjoint by construction.

A subtle case is insertions. A creature born during a tick (via apply_reproduce) does not appear in any system’s read-set during that tick - it is in to_insert, not in creatures. The newborn lives its first life on the next tick. This is the right behaviour for almost every simulation: it gives every creature an equal first tick of life. The alternative - applying inserts mid-tick - is a closed-loop bug factory.

Within one system, the writes can be in-tick: a system that updates pos_x[:] = pos_x + vel_x * dt for every creature in one numpy call applies all writes “at once” inside that system, because the rest of the system is the only reader and the only writer. The buffering rule is between systems, not between iterations within one system. Inside a system, the writes are sequential (or vectorised); between systems, the writes are batched.

The shape that emerges is: read everything into local arrays at system entry; do work; write outputs to buffers at system exit; commit at tick boundary. It is the same shape as the audio engine’s frame buffer, the database’s transaction commit, and the version-controlled file system’s commit-and-merge. They all solve the same problem: how do you read consistent state while the world is changing?

Exercises

These build on the simulator skeleton. Your to_remove: list[int] and to_insert: list[CreatureRow] should already exist.

1. The list bug. Build a list of 100 creatures where 30 have energy <= 0. Iterate the list, calling creatures.remove(c) whenever c.energy <= 0. Count how many starvers survive. Why did the bug only affect some of them? (Hint: every removal shifts the iterator past one extra element.)
2. The dict bug. Build a dict[int, Creature] of 100 with the same 30 starvers. Iterate creatures.items(), calling del creatures[cid] whenever c.energy <= 0. Note the RuntimeError. Now “fix” it locally with for cid in list(creatures.keys()): - does the simulation now produce the right answer? Yes, but only because the local fix accidentally makes a complete copy first; you have papered over the structural problem at the cost of an O(N) allocation per tick.
3. The buffered fix. Rewrite the function to compute starvers = np.where(energy <= 0)[0] (read-only scan) and append the starvers’ ids (ids[starvers]) to to_remove - the buffer holds ids, not slots, so it survives the rows moving during cleanup (§22/§23). After the loop completes, apply all removals in one pass using the swap_remove pattern (preview of §21). Verify all 30 starvers die.
4. The cleanup pass. Write def cleanup(world, to_remove, to_insert). Apply removals first (using swap_remove on each affected column), then insertions. Why this order, and not the other? (Hint: insertions may reuse slots freed by removals - see §24.)
5. Show two ticks. Run the loop for two ticks. After tick 1, log the population. After tick 2, log it again. Confirm that creatures killed in tick 1’s apply_starve do not appear in tick 2’s input - they were removed at the tick boundary, between the two ticks.
6. Insertions are tick-delayed. A creature reproduces in tick 5: parent in creatures, two offspring in to_insert. After cleanup, the offspring are in creatures. In tick 6 the offspring receive their first system pass. Confirm by adding an age_in_ticks column and watching offspring start at 0 in tick 6, not in tick 5.
7. (stretch) A bad design that almost works. Try to apply mutations in-tick carefully - collect dead creatures first, then process them in reverse-index order to avoid the iterator-skip bug. Show one specific case where this still corrupts state. (Hint: a reproduction produces an offspring whose new index conflicts with an in-progress death.)
8. (stretch) Read the simlog. Open .archive/simlog/logger.py. Find the two Container instances. Find the line where they swap. Find the function the background thread runs. Note that the logger never holds both containers locked simultaneously - the swap is atomic, the dump is on the inactive container. This is the production version of what exercise 3 teaches.

Reference notes in 15_state_changes_between_ticks_solutions.md.

What’s next

§16 - Determinism by order is the property the buffering rule guarantees: same inputs, same system order, same outputs. Reproducibility is structural.

Solutions: 15 - State changes between ticks

Exercise 1 - The list bug

class Creature:
    def __init__(self, energy): self.energy = energy

import random
random.seed(0)
cs = [Creature(-1)] * 30 + [Creature(10)] * 70
random.shuffle(cs)

for c in cs:
    if c.energy <= 0:
        cs.remove(c)

remaining_starvers = sum(1 for c in cs if c.energy <= 0)
print(f"30 starvers initially → {remaining_starvers} survived after remove-during-iter")

30 starvers initially → 8 survived

The iterator advances by index. When cs.remove(c) shifts every later element down one slot, the index the iterator advances to skips the next element. So roughly every other starver is missed. The exact count depends on the shuffle (8 here, anywhere from 5-15 typical).

The bug is silent. The simulation runs, the program terminates, the answer is wrong. Nothing complains. The fact that you have to count the survivors to detect it is precisely why the discipline of buffered mutation exists.

Exercise 2 - The dict bug

cs = {i: Creature(-1 if i < 30 else 10) for i in range(100)}

# Naive remove-during-iter
try:
    for cid, c in cs.items():
        if c.energy <= 0:
            del cs[cid]
except RuntimeError as e:
    print(f"got expected: {e}")
# RuntimeError: dictionary changed size during iteration

# Local "fix": iterate a snapshot of the keys
cs = {i: Creature(-1 if i < 30 else 10) for i in range(100)}
for cid in list(cs.keys()):                    # snapshot - O(N) allocation
    if cs[cid].energy <= 0:
        del cs[cid]
print(f"survivors: {len(cs)}")                  # 70 - correct!

The dict version crashes loudly, which is better than silently wrong, but its lesson trains the reader to apply a local fix (list(cs.keys())) without recognising the structural problem. The local fix:

• Costs an O(N) allocation per tick (the snapshot).
• Hides the mutation pattern; the next reviewer assumes the iteration is safe.
• Doesn’t fix the underlying issue: a system reading cs while another piece of code (you, in this case) writes to it.

Both bugs are the same lesson: mutating a container while another piece of code is reading it is the bug, regardless of whether the language catches it.

Exercise 3 - The buffered fix

import numpy as np

energy = np.array([-1.0]*30 + [10.0]*70, dtype=np.float32)
np.random.default_rng(0).shuffle(energy)
ids    = np.arange(100, dtype=np.uint32)

to_remove: list[int] = []

def apply_starve(energy: np.ndarray, to_remove: list[int]) -> None:
    starvers = np.where(energy <= 0)[0]
    to_remove.extend(starvers.tolist())

def cleanup(energy, ids, to_remove):
    """Apply queued removals via swap_remove (preview of §21)."""
    if not to_remove: return energy, ids
    keep = np.ones(len(energy), dtype=bool)
    keep[to_remove] = False
    return energy[keep], ids[keep]

apply_starve(energy, to_remove)
energy, ids = cleanup(energy, ids, to_remove)
print(f"30 starvers → {(energy <= 0).sum()} remain after one tick")    # 0

The starvation system is read-only: it scans energy and writes only to to_remove. The energy array does not change during apply_starve - it changes exactly once per tick, in cleanup. There is no window in which two systems could see different states.

Exercise 4 - The cleanup pass

def cleanup(world, to_remove: list[int], to_insert: list[dict]) -> None:
    """Apply removals first, then insertions.

    Removals first because:
    - swap_remove (§21) frees specific slots by moving the last row in.
    - Inserts can target those freed slots (§24 recycling).
    - Doing inserts first would force them to allocate fresh slots even
      when freed slots are about to become available.
    """
    # Removals via swap_remove
    for slot in sorted(to_remove, reverse=True):     # high-to-low avoids index shifting
        last = len(world.energy) - 1
        if slot != last:
            for col in (world.pos_x, world.pos_y, world.energy, world.ids, world.gens):
                col[slot] = col[last]
        # truncate (in real code: world.live_count -= 1)
        ...
    to_remove.clear()

    # Insertions into freed (or newly-allocated) slots
    for row in to_insert:
        slot = world.allocate_slot()                 # reuses recycled slots first
        for col_name, value in row.items():
            getattr(world, col_name)[slot] = value
    to_insert.clear()

Removals first means freed slots immediately host inserts on the same tick; the table doesn’t grow when births and deaths balance. Inserts first would force every newborn to push the table to a fresh slot before the dying creatures return their slots - a one-tick high-water mark proportional to the death rate. §24 makes this explicit.

Exercise 5 - Show two ticks

energy = np.array([-1.0, -1.0, 5.0, 8.0, -1.0], dtype=np.float32)
ids    = np.array([10, 11, 12, 13, 14], dtype=np.uint32)
to_remove: list[int] = []

# Tick 1
apply_starve(energy, to_remove)
energy, ids = cleanup(energy, ids, to_remove)
to_remove.clear()
print(f"after tick 1: ids={ids.tolist()}, energy={energy.tolist()}")
# ids=[12, 13], energy=[5.0, 8.0]

# Tick 2 - only the survivors run
apply_starve(energy, to_remove)         # nothing dies
print(f"tick 2 input: ids={ids.tolist()} (the dead ids 10,11,14 are not in this list)")

The dead creatures from tick 1 are gone between ticks. Tick 2’s apply_starve sees only the survivors. The systems don’t have to know about the death - the cleanup pass handled the bookkeeping at the tick boundary.

Exercise 6 - Insertions are tick-delayed

ages = np.zeros(N, dtype=np.uint16)

def age_creatures(ages):
    ages += 1                                   # one increment per tick

# Tick 5: a parent reproduces; offspring go into to_insert with age=0
to_insert.append({"pos_x": 1.0, "pos_y": 2.0, "energy": 5.0, "ages": 0})

cleanup(world, to_remove, to_insert)            # offspring now in `creatures`

# Tick 6: age_creatures runs over all live creatures, including the new ones
age_creatures(world.ages)                       # offspring goes 0 → 1

The newborn does not appear in any system’s read-set during tick 5 - it is in to_insert, not in creatures. Its first tick of life is tick 6, where it is incremented from 0 to 1 by age_creatures. Every creature gets a full tick of life on its first tick, regardless of when in the previous tick it was born.

The alternative (in-tick insertion) would mean a creature born at the start of tick 5 ages from 0 → 1 in the same tick, while one born at the end of tick 5 ages 0 → 0. That arbitrariness is what the rule prevents.

Exercise 7 - A bad design that almost works (stretch)

The “fix” of processing dead creatures in reverse-index order:

# anti-pattern: bad!
def apply_starve_inplace(creatures):
    dead = [i for i, c in enumerate(creatures) if c.energy <= 0]
    for i in sorted(dead, reverse=True):
        del creatures[i]                        # high-to-low avoids index-shift

The case where it still corrupts state: insertions during the same tick. Suppose apply_reproduce runs after apply_starve and pushes new creatures onto the same creatures list:

# tick body
apply_starve(creatures)                          # deletes some, frees indices
apply_reproduce(creatures)                       # appends new ones
inspect(creatures)                               # sees a mixed-state world

What inspect sees: a mix of creatures who were alive at the start of the tick (still here, at possibly-different indices), creatures born this tick (already alive in creatures), and the gaps if the death pattern wasn’t a clean suffix. Other systems that captured indices at the start of the tick (e.g., a pending_event from next_event) now point at wrong rows.

The buffered approach prevents all of this by definition: nothing changes in creatures until cleanup, and cleanup applies removals + insertions in one consistent sweep.

Exercise 8 - Read the simlog (stretch)

The vendored copy at .archive/simlog/logger.py is the production-grade version of this chapter’s pattern. Things to find:

• Two Container instances. The logger maintains two pre-allocated numpy column buffers (active and inactive). The simulation writes only to active.
• The atomic swap. When active fills, the logger atomically swaps the two references (active, inactive = inactive, active). The simulation’s next write goes to the now-empty buffer; the previously-active buffer is now inactive and ready for flushing.
• The background flush thread. A worker thread sleeps until inactive is non-empty, then writes its contents to disk (.npz chunks) and clears it.

The simulator never holds both containers at once. The flush thread never sees a write in progress. The whole apparatus is the chapter’s “writes accumulate, then commit” rule - at production scale, with a background flush, ~1 µs per logged event, and zero coordination cost on the hot path. Worth reading once you have written the toy version yourself.

16 - Determinism by order

Concept node: see the DAG and glossary entry 16.

A program is deterministic if the same inputs and the same execution produce the same outputs, every time. Sounds obvious. It is not - most modern Python programs are not deterministic by default. Threads run in OS-scheduled order. Sets iterate in randomised order across processes. The system clock differs by run. random.random() reads from a global instance whose state depends on import order and prior calls.

In an ECS architecture, determinism is structural. Same world state at tick start + same system order + same inputs (events, RNG seed) = same world state at tick end. Bit-identical. Every time.

This is not a quality goal; it is a precondition for almost everything the book builds on:

• Replay. The world is the log decoded (§37). Replay reconstructs world state by re-running the inputs through the same system sequence. Without determinism, replay is impossible.
• Testing. A property test fixes an RNG seed and asserts the simulator behaves identically across runs. Without determinism, every test is flaky.
• Distributed simulation. Multiple machines run identical copies of the world. Without determinism, they drift apart by tick 1.
• Debugging. A bug at tick 4783 should appear at tick 4783 every run. Without determinism, debugging real-time bugs becomes guesswork.

The recipe, Python edition

The recipe for determinism is to forbid every source of non-determinism in the inner systems. In Python the sources have specific names.

No raw set iteration. From code/measurement/set_iteration_order.py, three fresh subprocesses iterating the same six-element set produced three different orders:

run 1: delta,foxtrot,echo,bravo,charlie,alpha
run 2: bravo,foxtrot,delta,echo,alpha,charlie
run 3: echo,delta,foxtrot,charlie,bravo,alpha

CPython hashes strings using a per-process random seed (PYTHONHASHSEED), and set iteration order is a function of the hash table’s bucket layout. Across processes, the layout differs; the iteration order differs. This is by design - it protects servers from hash-flooding attacks - but it is also a source of non-determinism that the simulator forbids. Never iterate a set inside a system. If you need an iteration order, use a sorted list, a numpy array, or a dict (which is insertion-ordered since CPython 3.7 and survives the same test):

run 1: alpha,bravo,charlie,delta,echo,foxtrot
run 2: alpha,bravo,charlie,delta,echo,foxtrot
run 3: alpha,bravo,charlie,delta,echo,foxtrot

No system clock inside a system. Get time from input events, not from time.time() or time.perf_counter(). Time is a value passed into the system, not read from the OS. The tick loop’s outer scaffolding may read the wall clock; the systems inside the tick may not.

One RNG, seeded. A single np.random.default_rng(seed) per simulator instance, used in a defined order. Each system that needs randomness reads from it in DAG order. Never random.random() (reads global state), never np.random.random() without the rng object (uses a global). Pass the rng as a parameter - it has a declared read-set just like any other input.

No threads inside a system. A system runs single-threaded internally. The GIL does not save you from non-determinism here; it serialises Python bytecode but not the order in which threads acquire it. Parallelism happens between systems with disjoint write-sets (§31) using multiprocessing, not inside one system using threading.

Buffered mutations. §15’s rule: mutations apply at tick boundaries, not mid-tick.

One Python-specific footnote: hash() itself. Hash randomisation has been on by default since CPython 3.3 for str and bytes (and the containers that derive from them, including frozenset). If a system computes hash(some_string) and uses that value as part of its output, the output is non-deterministic across processes. Use hashlib.blake2b(s.encode()).digest() - or any deterministic hash - when you need a stable hash inside a system.

These rules are restrictive. They are also the price of every benefit listed above. Most modern Python programs decline to pay this price and accept the costs - flaky tests, unreproducible bugs, divergent distributed simulation. The book pays the price.

The cost is at the boundary, not in the body

The cost of determinism is not absolute. Within a system, the implementation is free to use whatever it likes - vectorised numpy, low-level optimisations, even occasional non-deterministic libraries - as long as the inputs and outputs are bit-identical to what the abstract specification demands. The discipline is at the system boundary: between systems, everything must be reproducible.

Inside motion, you can use pos_x += vel_x * dt (numpy bulk op, deterministic) or np.einsum or write your own Cython kernel. As long as the output pos_x for given inputs is bit-identical across runs, the system is deterministic regardless of how its internals work. The contract is at the function boundary; the freedom is inside.

Testing for determinism

A test for determinism is concrete. Run the simulator twice with the same seed, the same input event log, the same system order. After 1,000 ticks, hash the entire world state - feed every numpy column through hashlib.blake2b(arr.tobytes()).hexdigest() and combine. If the hashes match, you are deterministic. If they do not, find the system whose output first differs, and trace the source of variability. Often: a set iterated, a time.time() call, a random.random() reading global state.

A simulator that is deterministic is also a simulator that can be tested. Once that property holds, every other quality goal - performance, parallelism, distribution - becomes safe to optimise toward. Without determinism, every optimisation is a coin flip.

The full payoff of determinism arrives at the save and load phase named in §11. The simulator can be paused, its tables serialised to disk, reloaded later, and resumed - and the result must be indistinguishable from a run that never paused. The mechanics arrive in §36 - Persistence is table serialization: a snapshot is the world’s columns written as .npz files - the same bytes they have in memory. Combined with the input event log, replay is structural - read the snapshot, replay events through the same DAG with the same seed, you reconstruct the world at any later tick exactly. Determinism (this section), serialization (§36), and log-as-world (§37) are the three legs of replay.

Exercises

1. Run the iteration-order exhibit. uv run code/measurement/set_iteration_order.py. Observe the set rows differ; the dict rows do not. Note that the dict survival is not a guarantee against frozenset keys, dict.values() derived from a set, or any operation that goes through hash bucket order - only the surface-level “I added these in order” pattern survives.
2. Hash the world. Write def hash_world(world) -> str that produces a hex digest by feeding every column through hashlib.blake2b(arr.tobytes()).update(...). Use this to compare world states across runs.
3. Two identical runs. Run the simulator twice with the same RNG seed (np.random.default_rng(42)) and the same input events. Hash the world at tick 100. Confirm they are equal.
4. Introduce non-determinism deliberately. Replace your seeded default_rng(42) with np.random.default_rng() (no seed - uses entropy). Run twice. Show the hashes differ.
5. Find the culprit. Suppose your hashes differ. Hash the world after each system in the DAG. Identify which system’s output first differs, and what source of non-determinism it pulls from. Common offenders: for k in some_set:, time.time(), random.random(), hash(some_string).
6. Time as input. Find a system that uses time.perf_counter() and refactor it to instead take current_time: float as a parameter. The system is now deterministic; the source of current_time is the only place non-determinism can enter.
7. The set trap up close. Build a set of 1,000 random integers (use a default_rng(42) so the set contents are deterministic). Iterate it three times in the same process. Are the orders the same? Now run the program twice in two fresh shells. Are the orders the same across runs? (Hint: the answers are yes and no, in that order. The trap is that a single test run will not catch the bug; two test runs in two CI workers will.)
8. (stretch) A property test. Hand-roll a simple property test: generate 100 random seeds. For each, run the simulator for 100 ticks. Hash the resulting world. Verify that the same seed always produces the same hash, and that different seeds usually produce different hashes.

Reference notes in 16_determinism_by_order_solutions.md.

What’s next

You have closed Time & passes. Determinism is structural; replay is architectural; the next phase is Existence-based processing, starting with §17 - Presence replaces flags. The simulator’s hunger and starvation systems are about to lose their booleans.

Solutions: 16 - Determinism by order

Exercise 1 - Run the iteration-order exhibit

uv run code/measurement/set_iteration_order.py

Source: code/measurement/set_iteration_order.py.

Set iteration order across runs:
  run 1: delta,bravo,foxtrot,echo,alpha,charlie
  run 2: alpha,foxtrot,delta,charlie,echo,bravo
  run 3: bravo,echo,alpha,foxtrot,charlie,delta
  → 3 distinct orders - sets are non-deterministic.

Dict iteration order across runs:
  run 1: alpha,bravo,charlie,delta,echo,foxtrot
  run 2: alpha,bravo,charlie,delta,echo,foxtrot
  run 3: alpha,bravo,charlie,delta,echo,foxtrot
  → orders match - dicts are insertion-ordered since CPython 3.7.

The dict survival is the insertion order of the keys, which is a property of how you populated the dict. It’s not a guarantee against:

• dict(some_set) - values come from set iteration; first survival breaks.
• frozenset keys - same hash-bucket randomness applies.
• dict.fromkeys(some_set) - same.

Survive only what you can prove. When in doubt: sorted(...).

Exercise 2 - Hash the world

import hashlib
import numpy as np

def hash_world(world) -> str:
    h = hashlib.blake2b(digest_size=16)
    for col in (world.pos_x, world.pos_y, world.vel_x, world.vel_y,
                world.energy, world.ids, world.gens):
        h.update(col.tobytes())
    h.update(np.array([len(world.pos_x)], dtype=np.int64).tobytes())  # length too
    return h.hexdigest()

arr.tobytes() returns the contiguous in-memory bytes. blake2b is fast and 16 bytes is plenty for run-to-run comparison. Including the length prevents two worlds with different sizes but identical-prefix data from hashing the same.

Exercise 3 - Two identical runs

def run(seed: int, ticks: int) -> str:
    rng = np.random.default_rng(seed)
    world = build_world(rng, n=100)
    for _ in range(ticks):
        tick(world, rng, dt=1.0/30.0)
    return hash_world(world)

assert run(42, 100) == run(42, 100)             # same seed → same hash
print(run(42, 100))                             # any deterministic 32-char hex

Bit-identical. If this fails, your simulator has a non-determinism source somewhere - exercise 5 is the diagnostic.

Exercise 4 - Introduce non-determinism deliberately

def run_unseeded(ticks: int) -> str:
    rng = np.random.default_rng()               # no seed - entropy from OS
    world = build_world(rng, n=100)
    for _ in range(ticks):
        tick(world, rng, dt=1.0/30.0)
    return hash_world(world)

print(run_unseeded(100))   # something
print(run_unseeded(100))   # something else

Without a seed, default_rng() reads os.urandom. Two consecutive runs draw from different entropy and produce different results. The hashes differ. The simulator is now untestable.

Exercise 5 - Find the culprit

def hash_world_per_system(world) -> dict:
    """Run each system and hash the world after each."""
    snapshots = {}
    snapshots["start"] = hash_world(world)
    food_spawn(world)
    snapshots["after food_spawn"] = hash_world(world)
    motion(world)
    snapshots["after motion"] = hash_world(world)
    next_event(world)
    snapshots["after next_event"] = hash_world(world)
    apply_eat(world)
    snapshots["after apply_eat"] = hash_world(world)
    # ... and so on
    return snapshots

Run twice with the same seed; compare the two snapshots dicts. The first key whose hash differs identifies the offending system. Look inside its body for:

• for x in some_set:
• time.time(), time.perf_counter(), datetime.now()
• random.random(), np.random.random() without an rng instance
• hash(some_string) used in any output computation
• os.environ, os.getpid()
• Iteration over a frozenset, dict_keys, or dict.fromkeys(set)

Most simulator non-determinism in the wild is one of these patterns hiding inside a “harmless” helper.

Exercise 6 - Time as input

# Before - system reads the OS clock:
def schedule_event_bad(events):
    now = time.perf_counter()                   # non-deterministic
    events.append((now + 0.5, "fire"))

# After - system takes time as a parameter:
def schedule_event(events, current_time: float):
    events.append((current_time + 0.5, "fire"))

# The tick loop scaffolding reads the wall clock - once, at the boundary:
def tick(world, current_time: float, dt: float):
    schedule_event(world.events, current_time)
    motion(world, dt)
    ...

current_time = 0.0
for _ in range(100):
    tick(world, current_time, dt)
    current_time += dt

The systems are pure functions of their inputs. The tick loop chooses what current_time is. For a real-time simulator, current_time = time.perf_counter() - start. For a deterministic replay, current_time comes from the event log. Same systems, two execution modes, the difference at the boundary, not in the body.

Exercise 7 - The set trap up close

import numpy as np
rng = np.random.default_rng(42)
s   = set(rng.integers(0, 1_000_000, size=1000).tolist())   # contents deterministic

# Three iterations IN THE SAME PROCESS
o1 = list(s); o2 = list(s); o3 = list(s)
print(o1 == o2 == o3)                # True - same process, same hash table layout

Within one process, set iteration order is stable (the hash table layout doesn’t change between iterations). Run the same program in two fresh shells and you get two different orders, because PYTHONHASHSEED is randomised per process.

The trap: a single test run does not catch the bug. The CI worker that runs the same test in five parallel processes catches it. The user who reports “works on my machine, fails in CI” has hit it.

The fix: never iterate a set in a system. Always:

for x in sorted(s):                  # deterministic across runs
    ...

Or store the data in something that’s already ordered (a list, a numpy array). Or use a dict with insertion order if order matters and uniqueness matters.

Exercise 8 - A property test (stretch)

import numpy as np

def property_test_determinism(n_seeds: int = 100, ticks: int = 100):
    seeds = list(range(n_seeds))
    for seed in seeds:
        h1 = run(seed, ticks)
        h2 = run(seed, ticks)
        assert h1 == h2, f"non-deterministic at seed {seed}: {h1} vs {h2}"
    distinct = len({run(s, ticks) for s in seeds})
    assert distinct > n_seeds * 0.95, "different seeds collapse to same world - bug"
    print(f"OK: {n_seeds} seeds, all reproducible, {distinct} distinct worlds")

property_test_determinism()

Two assertions:

1. Same seed → same world. Catches non-determinism (a set iteration, a time.time() call).
2. Different seeds → different worlds (mostly). Catches accidental seed-loss (a global random.seed() overriding the per-run rng) - without it, every run produces the same hash regardless of the seed parameter.

This is the entire core of property-based simulation testing. Hypothesis (the Python library) builds elaborate generators and shrinkers around it; the underlying assertion is the same.

17 - Presence replaces flags

Concept node: see the DAG and glossary entry 17.

A creature can be hungry. Three ways to model it.

The instinct most Python programmers arrive with is a boolean field on the object: is_hungry: bool on every Creature, set to True when energy drops below a threshold, set to False when energy is restored. Every system that cares about hunger checks the flag: for c in creatures: if c.is_hungry: .... This is everywhere; it is the natural choice; it is what most Python tutorials reach for. It is also the worst of the three options - it is both AoS (per-creature object) and flag-shaped (one bit per creature regardless of state), and it forces every consumer to scan all N creatures to find the K hungry ones.

The middle option - better than per-instance booleans, still not the disciplined choice - is a boolean column. is_hungry = np.zeros(N, dtype=bool), indexed in lockstep with the rest of the creature table. This is what most readers will reach for after Part 2. It pays one byte per creature (numpy’s bool is one byte, not one bit), but the bytes are contiguous; numpy vectorises the scan; SIMD reads forty creatures per cache line. Compared to the per-object form, it is one-to-two orders of magnitude faster. Compared to the disciplined form below, it still costs N bytes regardless of how few creatures are hungry.

The data-oriented alternative is membership. There is a hungry table - a np.ndarray of the slots of hungry creatures (the rows they occupy in the columns), of length K (the number of currently-hungry creatures), no longer than it has to be. A creature is hungry if and only if its slot is in hungry. The flag does not exist as a field; it exists as a fact about which table the creature appears in. (Why the slot and not the stable id from §10? Because a system that acts on the hungry reaches straight into the columns - energy[hungry] - and the slot is that reach; an id would need a lookup first. Until a death relocates anyone a creature’s slot equals its id - ids = np.arange(N) - so the distinction is invisible here; §23 makes it precise.)

# Three representations of "is this creature hungry?" (i is the creature's slot)
is_hungry_attr     = creatures[i].is_hungry          # AoS bool field
is_hungry_mask     = bool(is_hungry[i])              # SoA bool column, O(N) bytes
is_hungry_presence = np.isin(i, hungry)              # presence table, O(K) bytes

The substitution looks small: a bool field becomes a row in another table. The implications are not.

Four shifts that follow

Dispatch changes shape. The flag version is a per-creature filter inside every consuming system - walk all creatures, check the flag, do work if true. The membership version skips the filter - walk hungry, do work for every entry. At 1,000,000 creatures with 100,000 hungry, the flag version processes 1,000,000 rows; the membership version processes 100,000 - a 10× difference in work, and a 10× difference in memory bandwidth. §19 names this.

Storage changes shape. A np.bool_ column stores one byte per creature whether the flag is set or not. A creature with eight possible states needs eight bool columns = 8 bytes per creature; a million creatures store 8 MB of flags, most of which are False. Eight presence tables store only the entries that are set - if 10% of creatures are hungry, the hungry table is 10% the size of the flag column.

Persistence changes shape. Serialising a flag column writes the flag for every creature, including the ones where it is False. Serialising a presence table writes only the entries that exist. The latter is also closer to the natural shape of an event log (§37): a hungry_added event per entry, and that is the whole story.

Concurrency changes shape. Two bool columns on the same creature table sit adjacent in memory; concurrent writers to either column fight over the same cache lines (§33 - false sharing). Two presence tables are physically separate numpy arrays; concurrent writers to disjoint tables never collide (§31).

The reversal

The clean way to phrase the move: instead of asking each entity about its state, ask the state-table which entities have that state. The query is reversed; the lookup is reversed; the work shrinks. Most programs spend their lives doing the wrong direction; the data-oriented mindset is to reverse it.

A production example: in a real ECS daemon, an admission decision is is_admitted = peer_id in established_contacts. There is no is_admitted: bool on a peer; there is only the question “is this peer’s id in the table?” - O(1), no I/O, no enum. (Here the table is keyed by the id, not a slot: peer identity outlives any row, so it is the boundary case §23 keeps ids for.)

When flags are right

Presence is not the only valid representation. A bool column is sometimes right - when nearly every entity has the state set (a near-universal flag wastes nothing as a column and saves on the membership scan); when the predicate is so cheap to compute on the fly that materialising it is silly (is_positive_x = pos_x > 0); when the data is short-lived and persistence does not matter; when the lookup pattern is “give me this creature’s hunger state” (per-creature query, where a column lookup is O(1) but a presence-table membership scan is O(K) without an index).

In this book, presence is the default; flags are a tradeoff to earn.

Exercises

These extend the §0 simulator skeleton.

1. Add a hungry table. Add hungry = np.empty(0, dtype=np.uint32) to your world. It is empty at start.
2. Populate it. Write a system def classify_hunger(energy) -> np.ndarray that returns the slots of all creatures with energy[slot] < HUNGER_THRESHOLD. The body is one numpy line: np.flatnonzero(energy < HUNGER_THRESHOLD). Replace the world’s hungry with the result each tick.
3. Build the flag version. Add a parallel is_hungry = np.zeros(N, dtype=bool) indexed by creature slot. Write the equivalent classification system that sets the bool column.
4. Build the AoS version. Build a list[Creature] where Creature is a @dataclass(slots=True) with an is_hungry: bool field. Write the equivalent classification - a Python for loop. (Foreshadow: this is the version most tutorials teach.)
5. Time all three at 1M creatures, 10% hungry. Time classify_hunger (presence), the bool-column version (flag), and the AoS version. Note the ordering and the magnitudes. Presence and flag should be within ~2-5× of each other (both numpy); the AoS version should be one to two orders of magnitude slower than either (interpreter-bound, per §1).
6. The membership query. Write def is_hungry_p(hungry, slot) -> bool (presence - bool(np.any(hungry == slot))) and def is_hungry_f(is_hungry_col, slot) -> bool (flag - bool(is_hungry_col[slot])). Time both at 1M creatures. Note: presence is O(K) without an index map; the flag is O(1). §23 - Index maps is the fix that makes presence O(1) too.
7. “How many are hungry?” Write it three ways. Presence: len(hungry). Flag column: int(is_hungry.sum()). AoS: sum(1 for c in creatures if c.is_hungry). Compare wall times at 1M creatures with 10% hungry. The presence version is constant-time; the flag-column version walks all 1M as a single numpy reduction; the AoS version walks all 1M with interpreter dispatch on every step.
8. (stretch) Persist both. Serialise the flag-column version with np.save("is_hungry.npy", is_hungry) and the presence version with np.save("hungry.npy", hungry). Note the disk size for 1M creatures with 10% hungry. The presence file is ~400 KB; the flag-column file is ~1 MB even though 90% of the bits are 0. (Compression closes some of this gap, but not all of it - np.savez_compressed will help on the flag column more than on the presence array, because the flag column has a long run of zeros to compress and the presence array is already small.)

Reference notes in 17_presence_replaces_flags_solutions.md.

What’s next

§18 - Add/remove = insert/delete names what changes between the two representations: in the presence world, state transitions are structural moves between tables, not flag flips.

Solutions: 17 - Presence replaces flags

Exercise 1 - Add a hungry table

import numpy as np

class World:
    def __init__(self, n):
        self.energy = ...                                 # existing
        self.ids    = np.arange(n, dtype=np.uint32)
        self.hungry = np.empty(0, dtype=np.uint32)        # the new presence table

Empty array. No bool column, no flag, no boolean attribute. The world starts with zero hungry creatures.

Exercise 2 - Populate it

HUNGER_THRESHOLD = 10.0

def classify_hunger(energy: np.ndarray) -> np.ndarray:
    return np.flatnonzero(energy < HUNGER_THRESHOLD)   # the slots that pass

One numpy line. The system’s read-set is energy; the write-set is whatever the caller assigns the result to. np.flatnonzero returns the positions where the mask is true - the slots - which is exactly what the hungry table holds. Each tick:

world.hungry = classify_hunger(world.energy)

Exercise 3 - Build the flag version

is_hungry = np.zeros(N, dtype=bool)              # one byte per creature

def classify_flag(energy, is_hungry):
    is_hungry[:] = energy < HUNGER_THRESHOLD     # in-place, broadcasts

Same data, parallel column shape. Length N regardless of how many are actually hungry; one byte per creature wasted on the false ones.

Exercise 4 - Build the AoS version

from dataclasses import dataclass

@dataclass(slots=True)
class Creature:
    energy: float
    is_hungry: bool = False

creatures = [Creature(float(e), False) for e in energy]

def classify_aos(creatures):
    for c in creatures:
        c.is_hungry = c.energy < HUNGER_THRESHOLD

The Python tutorial canonical version. Every consumer of “is this creature hungry” reads c.is_hungry and pays for getattr on every access.

Exercise 5 - Time all three at 1M creatures

classify presence:  0.35 ms  (100K hungry of 1M)
classify flag:      0.06 ms
classify AoS:      13.2  ms

Two surprises:

• Flag is faster than presence at the classification step. Building the boolean mask alone is cheap (0.06 ms); building the list of slots that pass the mask needs an extra pass (np.flatnonzero) to materialise the index array. For one-shot classification the flag column still wins - but by a ~6× gap, not the orders of magnitude the downstream story might lead you to expect.
• AoS is ~38× slower than the slower numpy version. That’s the cost of the per-element interpreter loop, exactly as §13 promised.

The presence advantage shows up downstream - at the consumer step, not the classifier. Next exercise.

Exercise 6 - The membership query

def is_hungry_p(hungry: np.ndarray, slot: int) -> bool:
    return bool(np.any(hungry == slot))              # O(K)

def is_hungry_f(is_hungry: np.ndarray, slot: int) -> bool:
    return bool(is_hungry[slot])                     # O(1)

flag:     ~50 ns
presence: O(K) ms - proportional to len(hungry)

The flag wins for single-creature lookup - direct array indexing is faster than scanning. Presence wins for whole-table operations (count, iterate the hungry set) because there is no scanning of the false rows. The right answer depends on the query pattern; the wrong reflex is to assume one is always faster than the other.

§23 - index maps is the fix that makes presence O(1) for membership too: a sparse set - a second array mapping a slot to its position in hungry - lets you check membership in one read, with no per-creature boolean. With it, presence beats flag on every operation that matters in the simulator.

Exercise 7 - “How many are hungry?”

count presence:  30 ns         (len(hungry))
count flag:     204 µs         (int(is_hungry.sum()))
count AoS:        10 ms         (sum(1 for c in creatures if c.is_hungry))

Presence is 6800× faster than flag here. Why? len(hungry) is a single Python attribute read on a numpy array - it does not iterate. The flag version has to iterate (sum a million booleans). The AoS version pays for it 50,000× over.

This is where presence pays back. The classification cost is paid once per tick; the count is read by every system that needs to know “how many are hungry?” If even one consumer asks for the count per tick, the presence form pays back its classification overhead instantly. Most simulators have several such consumers (UI display, log entry, decision in the food-spawn policy, etc.).

Exercise 8 - Persist both (stretch)

np.save("is_hungry.npy", is_hungry)               # 1 MB (1 byte × 1M)
np.save("hungry.npy",    hungry)                  # ~400 KB (4 bytes × 100K)

np.savez_compressed("is_hungry.npz", is_hungry)   # ~120 KB (compresses runs of zeros)
np.savez_compressed("hungry.npz",    hungry)      # ~395 KB (already dense)

Uncompressed: presence is 2.5× smaller. Compressed: flag becomes smaller because 90% of its bytes are zeros that compress almost to nothing; presence is essentially incompressible random integers.

This reverses the conclusion at storage time but not at I/O time: writing the flag column requires reading 1 MB of bytes from RAM to compress, while writing the presence array reads 400 KB. In RAM, presence wins; on disk after compression, flag wins (sometimes); at write time, presence wins. Pick the layout that matches your dominant access pattern; persistence is one consideration among several.

For the simulator’s case - frequent in-memory operations, infrequent persistence - presence is the right default. For an archive that’s mostly written once and read rarely, the trade is closer.

18 - Add/remove = insert/delete

Concept node: see the DAG and glossary entry 18.

In the flag world, a state transition is a write. To make a creature hungry, set is_hungry = True. To stop it being hungry, set is_hungry = False. The flag was always there; only its value changed.

In the presence world, a state transition is a move between tables. To make a creature hungry, insert a row into hungry. To stop it being hungry, remove the row. The state has no field to flip; it has only the question of which table the creature is currently a row of.

# flag (canonical Python tutorial)
def become_hungry_flag(is_hungry: np.ndarray, slot: int) -> None:
    is_hungry[slot] = True

# presence
def become_hungry_presence(hungry: list[int], slot: int) -> None:
    hungry.append(slot)

def stop_being_hungry_presence(hungry: np.ndarray, slot: int) -> np.ndarray:
    pos = np.where(hungry == slot)[0]            # O(N) scan for now; §23's sparse set makes it O(1)
    if pos.size:
        # swap_remove: move last entry into the freed position, drop last
        hungry[pos[0]] = hungry[-1]
        return hungry[:-1]
    return hungry

“But I just set the bool, what’s the problem?”

The Python idiom that this chapter is asking you to abandon is older and more universal than is_hungry. It is creature.alive = False - the soft delete. Every Python tutorial that introduces classes teaches it: when a thing should stop being processed, set a bool, and check that bool before processing it. Tens of thousands of production codebases run on exactly this pattern.

The cost is real. From code/measurement/alive_fraction.py, one motion update over 1,000,000 creatures at varying alive-fraction:

Read the rows. At 1% alive - the typical case for a transient state like “hungry,” “dying,” or “just-spawned” - presence is 10× faster than the bool-mask version, and 150× faster than the AoS version. As alive-fraction climbs, the gap closes; around 80-90% alive the bool mask starts winning, and at 100% alive it is faster (numpy spots the all-True mask and uses a contiguous slice path instead of fancy indexing).

The AoS column is flat at 25-35 ms regardless of alive-fraction. The interpreter is iterating all one million creatures and paying the getattr(c, "alive") cost on every one, even when 99% of them are skipped a moment later. The “soft delete” pattern saves the actual work but never escapes the per-element dispatch tax.

The honest reading of the table: presence is the right default for transient state (low alive-fraction, the common case for hungry/dying/sleeping-and-soon-to-wake); bool masks are the right default for near-universal state (alive ≥ 90%); AoS is wrong at every alive-fraction. There are no scale ranges where the interpreter loop wins.

Note - “Alive” generalises further than this chapter uses it. In an MMORPG, the relevant set of creatures is the ones inside the player’s render radius - and the radius itself can shrink dynamically when CPU is tight, trading visible-creature count against the tick-budget headroom from §4. The presence table is a query, not a metaphysical state; its entries change when the system asks a different question. “Alive,” “hungry,” “in-scope,” “subscribed,” “active-this-frame” - same shape, different question. The crossover numbers above apply to whichever question your simulation is asking, with whichever fraction the answer happens to have.

Two consequences worth naming

The transition is structural. When a creature crosses the hunger threshold, a row in hungry actually appears or disappears. There is no in-place mutation; the table grows by one or shrinks by one. This is why §22 (mutations buffer; cleanup is batched) exists - adds and removes during a tick must be queued, then applied at the boundary, so that the iteration in progress does not see half the change. The deferred-cleanup pattern is born in this section.

The vocabulary disappears. There is no set_hungry(True), no set_hungry(False), no is_hungry() accessor pair. There is become_hungry (insert) and stop_being_hungry (remove), and even those are usually inlined into the system that detects the transition. The data-oriented program does not have getters and setters; it has systems that move rows between tables. No @property. No __setattr__ hooks. No “validation lives on the model” decorators. The system that detects the threshold is the validation, is the transition, is the audit trail.

A useful test: can you describe the transition without naming a bool? “This creature became hungry” - well, did anything change? Yes: the hungry table grew by one entry. “This creature stopped being hungry” - the table shrank by one entry. Every state change in the system has a structural counterpart, and the structural counterpart is the canonical description.

Multi-table transitions

The same pattern handles richer transitions. Imagine a creature that can be hungry, sleepy, or dead. Three tables: hungry, sleepy, dead. A creature transitions by moving between them. Becoming sleepy while hungry adds a row to sleepy (it can be in both). Dying removes the creature from hungry and sleepy (cleanup affects all relevant presence tables) and adds to dead. The transition is a multi-table operation, but each table is still just a numpy array of slots.

This shape - state changes as inserts and removes - is the precondition for everything else EBP gives you. The dispatch in §19 iterates over the table directly, so the table’s contents being the canonical state of the world is structurally necessary. There is no flag to consult; there is only what is in the table right now.

Exercises

1. Hunger transitions. Use your hungry table from §17. Each tick: read energy; for any creature that crossed below the threshold, append to a hungry_to_add buffer; for any that crossed back above, append to a hungry_to_remove buffer; apply both at the tick boundary. Run for 100 ticks with energy varying randomly; verify hungry always contains exactly the creatures whose current energy is below threshold.
2. Run the alive-fraction exhibit. uv run code/measurement/alive_fraction.py. Note the crossover row - the alive-fraction at which the bool mask starts beating presence. Note that the AoS column does not have a crossover; it loses at every fraction.
3. No bool, no setter. Search your code for any boolean field on a creature. Replace it with a presence table. The setter and getter both disappear. Search for any @property decorator that wraps a state field; same fate.
4. A second presence state. Add a sleepy table. A creature is sleepy if its energy is high enough that it does not need to eat right now. A creature can be in both sleepy and hungry? No - by definition the conditions are mutually exclusive. (Or: design them so they are.) Verify the invariant by checking after each tick that np.intersect1d(hungry, sleepy).size == 0.
5. Death. Add a dead table. When a creature’s energy drops below zero, append to dead and remove from hungry (and from sleepy if present). The cleanup logic is now multi-table; introduce a small transition_to_dead(slots, hungry, sleepy, dead) helper that handles all the affected presence tables.
6. The transition log. Add events: list[tuple[int, int, str]] (tick number, creature id, event name). Every insert/remove emits a row. Note the entry is the entity id, not the slot: the membership tables move by slot, but the log is a boundary artifact replayed later (§37) - when slot positions no longer hold - so it records identity (ids[slot]). After 100 ticks, the events log is the canonical history - every state change recorded. This is a preview of §37 - The log is the world.
7. (stretch) Reconstruct from the log. Given only the events log and the initial creature ids, reconstruct the final hungry, sleepy, and dead tables. The reconstruction is a one-shot replay; if it produces the same tables as the live simulation, your transitions are correctly captured.
8. (stretch) The crossover, on your machine. Re-run the exhibit varying alive-fraction more finely between 70% and 95% - say at 70, 75, 80, 85, 90, 95%. Find the alive-fraction at which mask and presence cross over on your hardware. The exact crossover depends on cache size, branch predictor, and the specific numpy build.

Reference notes in 18_add_remove_insert_delete_solutions.md.

What’s next

§19 - EBP dispatch names the dispatch shape that the table-membership representation makes free.

Solutions: 18 - Add/remove = insert/delete

Exercise 1 - Hunger transitions

import numpy as np

THRESHOLD = 10.0

def classify_transitions(prev_hungry, energy):
    """Return (to_add, to_remove) for the hungry presence table, as slots."""
    is_hungry_now  = energy < THRESHOLD
    was_hungry     = np.zeros(len(energy), dtype=bool)
    if prev_hungry.size:
        was_hungry[prev_hungry] = True       # prev_hungry holds slots
    just_became    = np.flatnonzero( is_hungry_now & ~was_hungry)
    just_recovered = np.flatnonzero(~is_hungry_now &  was_hungry)
    return just_became, just_recovered

def apply_hunger_changes(hungry: np.ndarray,
                          to_add: np.ndarray,
                          to_remove: np.ndarray) -> np.ndarray:
    if to_remove.size:
        hungry = hungry[~np.isin(hungry, to_remove)]
    if to_add.size:
        hungry = np.concatenate([hungry, to_add])
    return hungry

# Per-tick
to_add, to_remove = classify_transitions(world.hungry, world.energy)
# (events are batched; apply at tick boundary, §22)
world.hungry = apply_hunger_changes(world.hungry, to_add, to_remove)

# Invariant after the tick:
assert set(world.hungry.tolist()) == set(np.flatnonzero(world.energy < THRESHOLD).tolist())

The invariant check verifies the table’s contents match the predicate at the end of every tick. A simulator that respects this invariant has correctly implemented the structural transition.

Exercise 2 - Run the alive-fraction exhibit

uv run code/measurement/alive_fraction.py

 alive %    AoS (ms)   mask (ms)   presence (ms)    mask/presence
-----------------------------------------------------------------
    1.0%        8.85       0.696           0.070             9.9×
   10.0%       17.48       3.908           0.607             6.4×
   50.0%       23.13       9.718           2.438             4.0×
   90.0%       31.19       3.512           4.559             0.8×
  100.0%       32.80       1.518           4.928             0.3×

The crossover is somewhere between 50% and 90% alive - at 50% presence is 4× faster, at 90% mask is 1.3× faster. The exact crossover depends on hardware (next exercise).

The AoS column has no crossover. At every alive-fraction it loses to both numpy versions by 5-50×. The interpreter loop is paying the per-creature dispatch tax regardless of how few creatures actually need work.

Exercise 3 - No bool, no setter

A typical search shows fields like:

class Creature:
    is_hungry: bool = False
    is_alive:  bool = True
    is_visible: bool = True
    is_in_combat: bool = False

After the refactor:

class World:
    hungry:    np.ndarray = np.empty(0, dtype=np.uint32)
    visible:   np.ndarray = np.empty(0, dtype=np.uint32)
    in_combat: np.ndarray = np.empty(0, dtype=np.uint32)
    # `alive` becomes `live_count` + the implicit "all rows up to live_count are alive"

@property decorators that wrap state fields disappear too. The “validation” they encoded becomes part of the system that detects the transition - the system that causes a row to enter in_combat is also the only place where the validity of “this entity entered combat” gets checked. There’s no separate setter to wrap.

The vocabulary shrinks. creature.set_hungry(True) is replaced by whatever system produced the threshold crossing appending to hungry_to_add. There is no setter; there is a transition.

Exercise 4 - A second presence state

SLEEP_THRESHOLD = 80.0   # high energy → sleepy (won't need to eat)
HUNGER_THRESHOLD = 10.0

def classify_states(energy):
    hungry = np.flatnonzero(energy < HUNGER_THRESHOLD)
    sleepy = np.flatnonzero(energy >= SLEEP_THRESHOLD)
    return hungry, sleepy

# Invariant: a creature cannot be in both
hungry, sleepy = classify_states(energy)
assert np.intersect1d(hungry, sleepy).size == 0

The mutual exclusion is structural (the predicate ranges don’t overlap) - energy < 10 and energy >= 80 cannot both hold. If the predicates could overlap (e.g., is_hungry and is_running), one option is to enforce mutual exclusion in the apply step; another is to allow a creature to appear in both tables and let the dispatch code in §19 handle the overlap explicitly.

Exercise 5 - Death

def transition_to_dead(dying_slots: np.ndarray,
                      hungry: np.ndarray,
                      sleepy: np.ndarray,
                      dead:   np.ndarray):
    """A multi-table transition. Removes from all 'live state' tables, adds to dead."""
    new_hungry = hungry[~np.isin(hungry, dying_slots)]
    new_sleepy = sleepy[~np.isin(sleepy, dying_slots)]
    new_dead   = np.concatenate([dead, dying_slots])
    return new_hungry, new_sleepy, new_dead

dying = np.flatnonzero(world.energy < 0)
world.hungry, world.sleepy, world.dead = transition_to_dead(
    dying, world.hungry, world.sleepy, world.dead
)

A multi-table transition is one helper, not three independent updates. The helper is the audit trail: any change to the affected tables goes through it. If you later add a frozen table, you add it to the helper signature in one place. No place outside the helper writes to these tables - the §25 ownership-of-tables discipline at the multi-table scale.

Exercise 6 - The transition log

events: list[tuple[int, int, str]] = []          # (tick, creature_id, event_name)

def log_transitions(events, tick, ids, became_slots, recovered_slots):
    # The membership buffers hold SLOTS; the log records IDENTITY. ids[slot] == slot
    # today, but §21's swap_remove relocates slots and breaks that - which is exactly
    # why the log stores the id, the one handle that survives a move.
    for slot in became_slots.tolist():
        events.append((tick, int(ids[slot]), "became_hungry"))
    for slot in recovered_slots.tolist():
        events.append((tick, int(ids[slot]), "stopped_being_hungry"))

# After 100 ticks, the events list is the canonical history of state transitions.
print(f"events captured: {len(events)}")
print(f"first 5: {events[:5]}")

Every state transition is now a row in the events log. The current state of hungry is equivalent to the sequence of became_hungry and stopped_being_hungry events applied in order. This equivalence is the §37 log-is-world claim - once you have it, replay, audit, and rollback all become projections of the log.

For a real simulator the events would be stored in numpy columns (timestamp, creature_id_int, event_kind_int) - see the simlog reference. For the exercise, a Python list is fine; converting at end-of-run is cheap.

Exercise 7 - Reconstruct from the log (stretch)

def replay(events: list[tuple[int, int, str]]) -> dict[str, set[int]]:
    """Reconstruct the live tables from an event log."""
    tables = {"hungry": set(), "sleepy": set(), "dead": set()}
    for _, cid, name in events:
        if name == "became_hungry":          tables["hungry"].add(cid)
        elif name == "stopped_being_hungry":  tables["hungry"].discard(cid)
        elif name == "became_sleepy":         tables["sleepy"].add(cid)
        elif name == "stopped_being_sleepy":  tables["sleepy"].discard(cid)
        elif name == "died":
            tables["hungry"].discard(cid)
            tables["sleepy"].discard(cid)
            tables["dead"].add(cid)
    return tables

# Compare with the live simulator. replay() yields sets of entity IDS; the live
# tables hold SLOTS, so map slots -> ids to compare. (ids[slot] == slot until §21
# relocates anyone, but the mapping is the honest comparison: the log speaks
# identity, the tables speak position.)
live_tables = {
    "hungry": set(world.ids[world.hungry].tolist()),
    "sleepy": set(world.ids[world.sleepy].tolist()),
    "dead":   set(world.ids[world.dead].tolist()),
}
assert replay(events) == live_tables

If the assertion holds, the events captured every transition. The event log is now the canonical state; the in-memory tables are the projection. Snapshots of the tables become a performance optimisation (reading the snapshot is faster than replaying from t=0); the truth is the log.

This is exactly the architecture of every event-sourced system, every database WAL, every blockchain.

Exercise 8 - The crossover, on your machine (stretch)

Finer sweep between 70% and 95%:

# add to the SHARES list in alive_fraction.py:
SHARES = [0.01, 0.10, 0.50, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00]

Expected pattern: presence and mask cross somewhere between 75% and 90% on most modern machines. The exact crossover varies by:

• Cache size - bigger L2/L3 means the bool mask stays warm at higher fractions.
• Memory bandwidth - more bandwidth helps the mask version (which reads more bytes).
• Branch predictor quality - modern predictors handle the regular branches in the bool sum well; older CPUs were worse at it.

The point of the exercise is not to memorise a number. The point is that the right layout depends on your alive-fraction and your hardware. Measure, then choose. Defaulting to presence (the chapter’s stance) is right for transient state; defaulting to bool masks is right for near-universal state. Both happen to be correct on a wide range of hardware, just at different fractions.

19 - EBP dispatch

Concept node: see the DAG and glossary entry 19.

A system that needs to act on hungry creatures has two ways to find them.

Filtered iteration. Walk all creatures; for each, ask “is it hungry?”; do work if yes:

for slot in range(len(creatures)):
    if is_hungry[slot]:
        drive_hunger_behaviour(slot)

Existence-based dispatch. Walk the hungry table directly; do work for every entry:

for i in hungry:                 # i is a slot
    drive_hunger_behaviour(i)

In numpy, both shapes lift to one bulk operation:

# filtered (mask-based)
energy[is_hungry] -= HUNGER_BURN_RATE * dt

# EBP (presence-based)
energy[hungry] -= HUNGER_BURN_RATE * dt

The two produce the same result. The two have very different costs.

The filtered version evaluates is_hungry for every creature - a 1,000,000-byte scan to find the 100,000 hungry ones. The EBP version reads the 100,000 entries of hungry and indexes directly. From code/measurement/alive_fraction.py (the §18 exhibit), at both 1% and 10% sparsity the presence version was ~6.5× faster than the bool mask version. Note what does not happen: the advantage does not grow as the state gets sparser. In a scalar language it would (work proportional to the live fraction), but numpy’s mask scan is itself vectorised and bandwidth-cheap, so presence wins by a steady ~6.5× across the low-sparsity range rather than 10×, 100×, 1000×. Most simulator states are sparse - a small fraction of creatures are eating, reproducing, or dying at any given tick - so that steady advantage shows up on every consumer of the state.

A useful intuition: it is the difference between a wandering shopper trying to remember what they need and a shopper with a list. The list version is shorter, faster, and correct by construction. You do not consult the list to ask “is this aisle on my list?” - you walk down the list and visit each aisle once.

Three Python anti-shapes that collapse to “filtered iteration”

Python tutorials teach several patterns that all amount to filtered iteration. Each looks different on the page; they all consult a per-entity predicate instead of walking a presence table.

1. isinstance chains. When entities are modelled as a class hierarchy - Hungry(Creature), Sleepy(Creature), Dead(Creature) - dispatch usually walks one big list:

# anti-pattern: bad!
for entity in entities:
    if isinstance(entity, Hungry):
        drive_hunger(entity)
    elif isinstance(entity, Sleepy):
        drive_sleep(entity)
    elif isinstance(entity, Dead):
        # nothing to do
        pass

The list contains every entity; the body asks the type-tag predicate per entity. The presence-table version splits this into three independent systems, each iterating its own table.

2. Polymorphic method dispatch. The “more Pythonic” version uses dynamic dispatch:

# anti-pattern: bad!
for entity in entities:
    entity.update(dt)

Where Creature.update is overridden in Hungry, Sleepy, Dead. The if/elif is gone from the source code; it has been hidden inside Python’s method resolution order. Every iteration still pays an attribute lookup, an MRO walk, and a function-call setup. The predicate is now invisible but it is still being consulted per entity, and the cache penalty for jumping into a different method body for each subclass type is real. EBP replaces this with three explicit functions, each over its own table.

3. List-comprehension filters. The Pythonic functional-flavoured version:

# anti-pattern: bad!
hungry_creatures = [c for c in creatures if c.is_hungry]
for c in hungry_creatures:
    drive_hunger(c)

This looks like EBP - there is a list of just the hungry ones - but the list was built by scanning all N creatures and allocating a fresh Python list with K pointers. The filter pass is the same cost as the filtered-iteration version, plus a list allocation. EBP avoids the scan because the presence table was kept up to date as state transitions happened (§18); reads do not have to recompute it.

All three anti-shapes consult the predicate at iteration time. EBP arranges the world so the predicate has already been answered before the system runs - the table itself is the answer.

What EBP looks like as a system

A system that uses EBP looks like:

def drive_hunger(hungry: np.ndarray,      # slots (column positions), not entity ids
                 energy_used: np.ndarray,
                 dt: float) -> None:
    """Read-set: hungry (the slots of the hungry creatures).
       Write-set: energy_used, at those slots."""
    energy_used[hungry] -= HUNGER_BURN_RATE * dt

Read-set declared. Write-set declared. No per-row branch; the table is the dispatcher. The signature is the contract - exactly the system shape from §13. EBP is not a separate idea; it is the natural shape that a system takes when its inputs are presence tables.

Because hungry holds slots, energy_used[hungry] indexes the columns directly - one gather, with no id-to-slot lookup inside the loop. An entity-id list would not work here: it would need the §10 id_to_slot hop, and worse, it would read the wrong rows after any sort or swap_remove (§9). That directness is the whole point of keying the table by slot; §26 measures what it is worth (and why the table holds slots, not ids), once the lifecycle in §24 makes slots stable enough to store.

EBP also composes cleanly with parallelism. A million creatures with 100,000 hungry can be split across multiple processes - each takes a slice of hungry and does its work. The processes never need to consult creatures that are not hungry; their reads do not interfere. §31 develops this under multiprocessing + shared_memory.

The takeaway: EBP is the dispatch that falls out of §17’s presence-replaces-flags substitution. You do not need to choose to use EBP - once your state is in presence tables, every system naturally iterates them. The filtered-iteration version does not even arise.

Exercises

1. Re-read your alive-fraction numbers. From §18 exercise 2 you have measurements for AoS, bool mask, and presence at five alive-fractions. The same numbers tell the EBP story: the presence column is the EBP dispatch path. Confirm by mapping the §18 row labels to the §19 vocabulary - “presence” = “EBP,” “bool mask” = “filtered iteration.”
2. Implement both, on creatures. Implement drive_hunger_filtered(creatures, is_hungry, dt) (walks creatures, checks the bool column, applies the burn) and drive_hunger_ebp(hungry, energy, dt) (indexes the columns by the slots in hungry). Run both on a 1M-creature world with 10% hungry. Time both with timeit. Note the ratio.
3. The isinstance trap. Build a list[Creature] where some are Hungry(Creature), some are Sleepy(Creature), some are plain Creature. Implement dispatch via if isinstance(c, Hungry) chains. Time it at 1M creatures with 10% Hungry. Now implement the EBP version: three numpy presence tables, three system functions. Time it. The ratio is the cost of consulting the predicate per entity.
4. The polymorphic-method trap. Convert exercise 3 to class Hungry(Creature): def update(self): ... and a single for c in creatures: c.update(). Time it. Note that the source-code complexity fell (the if/elif is gone), but the runtime cost did not - the predicate moved into Python’s method resolution order, where it is still consulted on every iteration.
5. The list-comprehension filter. Implement hungry = [c for c in creatures if c.is_hungry] followed by for c in hungry: drive(c). Time it. Compare against EBP. Note that the filter pass is the cost of the filtered-iteration version plus a list allocation; the EBP version pays neither, because the hungry table was maintained at state-transition time, not at read time.
6. A multi-state system. A creature can be in any combination of hungry, sleepy, dead. Write three EBP systems: drive_hunger, drive_sleep, drive_death. Each iterates only its own presence table. Compare with a single filtered loop that handles all three with if/elif. Note that the EBP version has no shared state between the three systems and could trivially run them in parallel (§31).
7. (stretch) A naive EBP bug. A system that iterates hungry while also calling hungry.append on the table corrupts iteration. (You knew this from §9 and §15.) Construct a small case that demonstrates the bug - a creature that “becomes hungry” mid-iteration. Then fix it via deferred cleanup: write to to_become_hungry, apply at tick boundary.

Reference notes in 19_ebp_dispatch_solutions.md.

What’s next

§20 - Empty tables are free names the consequence at scale: cost is proportional to active rows, not to population.

Solutions: 19 - EBP dispatch

Exercise 1 - Re-read your alive-fraction numbers

The §18 alive-fraction exhibit is the EBP-vs-filtered comparison:

At 1% sparsity (typical for transient state): EBP is ~6.5× faster than the filtered numpy version, ~84× faster than the AoS version. The numpy advantage is steady (~6.5×) across the 1-10% range rather than growing with sparsity - numpy’s mask scan is vectorised, so it does not pay per-element for the dead rows the way a scalar predicate would. As the live fraction rises past ~50% the EBP advantage shrinks; at 100% live the bool mask wins because the “filter” is a no-op.

The takeaway: EBP is the right default for sparse states; bool masks are the right default for near-universal states. Both happen to be correct on a wide range of hardware; AoS is wrong at every fraction.

Exercise 2 - Implement both, on creatures

import numpy as np, timeit

n = 1_000_000
rng = np.random.default_rng(0)
energy = rng.uniform(0, 100, n).astype(np.float32)
hungry = np.flatnonzero(energy < 10)                # 10% sparsity, as slots
is_hungry = energy < 10
HUNGER = 0.5
dt = 1/30

def filtered(energy, is_hungry, dt):
    energy[is_hungry] -= HUNGER * dt

def ebp(energy, hungry, dt):
    energy[hungry] -= HUNGER * dt

t_f = timeit.timeit(lambda: filtered(energy.copy(), is_hungry, dt), number=200) / 200
t_e = timeit.timeit(lambda: ebp(energy.copy(), hungry, dt),         number=200) / 200
print(f"filtered: {t_f*1e6:.0f} µs   EBP: {t_e*1e6:.0f} µs   ratio: {t_f/t_e:.1f}×")

filtered: 2756 µs   EBP: 421 µs   ratio: 6.5×

At 10% sparsity, EBP is 6.5× faster on this machine. The filtered version reads is_hungry in full (1M bytes scanned) plus energy at the masked positions. The EBP version reads only hungry (the K = 100K hungry indices, 400 KB) plus energy at those positions. EBP’s working set is 90% smaller.

Exercise 3 - The isinstance trap

from dataclasses import dataclass

@dataclass(slots=True)
class Creature: energy: float
class Hungry(Creature): pass
class Sleepy(Creature): pass

# build a 1M list with three types mixed
ents = []
for i in range(n):
    e = float(energy[i])
    if e < 10:    ents.append(Hungry(e))
    elif e > 80:  ents.append(Sleepy(e))
    else:         ents.append(Creature(e))

def isinstance_dispatch(ents, dt):
    for e in ents:
        if isinstance(e, Hungry):
            e.energy -= HUNGER * dt

t_i = timeit.timeit(lambda: isinstance_dispatch(ents, dt), number=3) / 3
print(f"isinstance chain: {t_i*1e3:.1f} ms")

isinstance chain: 32.4 ms

At 1M entities with 10% Hungry, the isinstance chain costs 77× more than EBP (32.4 ms vs 0.42 ms). The cost is not the isinstance call alone - it’s per-entity interpreter dispatch plus isinstance, plus getattr(e, "energy"), plus the attribute write back to a heap-allocated object. Predicate-per-entity is the structural cost; isinstance is its idiomatic embodiment.

Exercise 4 - The polymorphic-method trap

class Creature:
    __slots__ = ("energy",)
    def __init__(self, e): self.energy = e
    def update(self, dt): pass

class Hungry(Creature):
    def update(self, dt):
        self.energy -= HUNGER * dt

# rebuild ents with subclass instances
ents = [Hungry(e) if e < 10 else Creature(e) for e in (float(x) for x in energy)]

def polymorphic(ents, dt):
    for e in ents:
        e.update(dt)

t_p = timeit.timeit(lambda: polymorphic(ents, dt), number=3) / 3
print(f"polymorphic dispatch: {t_p*1e3:.1f} ms")

Typical: ~50-80 ms. The source-code branching disappeared (no if isinstance in the loop body), but the cost moved into Python’s method resolution. Each call:

1. Looks up update via the MRO chain (one for Creature, one for Hungry).
2. Sets up a Python frame for the method call.
3. Dispatches to a different code path depending on the runtime type - a cache miss every time the type changes.

The “cleaner code” form is more expensive than the visible-branch form - the predicate is consulted as often, and each consultation is more work than isinstance.

Exercise 5 - The list-comprehension filter

def list_comp_dispatch(creatures, dt):
    hungry_list = [c for c in creatures if isinstance(c, Hungry)]      # filter pass
    for c in hungry_list:                                              # work pass
        c.energy -= HUNGER * dt

# Two passes: one to filter, one to work. Plus a list allocation.

The cost is the filtered-iteration baseline plus the list allocation. At 1M entities with 10% hungry, expect ~30-40 ms - comparable to the isinstance chain, with extra allocation pressure.

The shape looks like EBP (a list containing only the hungry ones). The difference is when the filtering happens. EBP’s hungry table is built when the transition occurs (energy crosses below threshold) - once per creature per state change. The list-comp form rebuilds it every read - once per query, on the entire population.

For a simulator with multiple consumers of “the hungry creatures” per tick, this gap compounds: EBP pays 1× the build cost, list-comp pays K× (K = number of consumers).

Exercise 6 - A multi-state system

hungry = np.flatnonzero(energy < 10)
sleepy = np.flatnonzero(energy > 80)
dead   = np.flatnonzero(energy < 0)

def drive_hunger(hungry, energy, dt):
    energy[hungry] -= HUNGER * dt

def drive_sleep(sleepy, energy, dt):
    pass     # sleepy creatures are at rest; no energy change

def drive_death(dead, world):
    world.live_count -= len(dead)

# Each system reads its own table. Disjoint write-sets where possible.

Three EBP systems, three independent write-sets:

• drive_hunger reads hungry, writes energy[hungry slots]
• drive_sleep reads sleepy, writes nothing (or a separate rest_log)
• drive_death reads dead, writes world.live_count (or to_remove)

Now compare to the filtered alternative:

def drive_all_filtered(creatures, dt):
    for c in creatures:
        if c.is_hungry:    c.energy -= HUNGER * dt
        elif c.is_sleepy:  pass
        elif c.is_dead:    c.live = False

The filtered version is one loop with three shared write-sets (energy, live, etc.). The three EBP systems can run in parallel; the filtered loop cannot, because all three branches write through the same Python list.

The §31 multiprocessing pattern is the same systems, run on disjoint slices of hungry. The filtered version cannot be split that cleanly because the consumer can’t tell, before reading each creature, which branch it will take.

Exercise 7 - A naive EBP bug (stretch)

hungry = list(np.arange(5, dtype=np.uint32))     # five creatures hungry
energy = np.array([5.0, 8.0, 3.0, 1.0, 7.0], dtype=np.float32)

# anti-pattern: bad! mutating hungry while iterating it
for cid in hungry:
    energy[cid] -= 1
    if energy[cid] < 2:                          # crossed a deeper threshold
        hungry.append(cid + 100)                 # also become *very_hungry*

The bug: the for loop’s iteration is over hungry’s state at iteration start; appending to hungry mid-iteration may or may not extend the iteration depending on the iterator’s implementation. With a Python list, appending does extend the iteration; with a generator over a numpy slice, it does not. Either way, the behavior is fragile - and reasoning about which creatures end up processed depends on the iteration’s implementation detail.

The fix is the deferred-cleanup pattern from §15:

to_add: list[int] = []
for cid in hungry:
    energy[cid] -= 1
    if energy[cid] < 2:
        to_add.append(cid + 100)

# After the iteration completes, apply the queued changes
hungry.extend(to_add)

The iteration sees a consistent snapshot. Mutations are queued and applied at a clear boundary. This is exactly the §15 and §22 discipline scaling down to a single system.

20 - Empty tables are free

Concept node: see the DAG and glossary entry 20.

If a presence table is empty, the system that iterates it does nothing. No rows, no work. This is the consequence of §19 at the limit, and it is the property that lets the simulator scale gracefully under shifting state.

Concretely: a 1,000,000-creature simulation with zero hungry creatures right now spends zero cycles in drive_hunger. The system is wired into the DAG, runs every tick, takes a numpy array of hungry slots of length 0, executes one bulk op that operates on zero elements, returns. The overhead is one function call and one fancy-index of length zero - measured in microseconds, not milliseconds.

This is not “fast in the empty case as an optimisation”. It is free in the empty case as a structural consequence. The flag-based version runs through the entire creature table even when no flags are set, paying full memory bandwidth to discover that no work is needed. The EBP version is told there is no work by the simple fact of an empty table.

The Python-default failure: Optional fields on every entity

Python’s tutorial reflex when an attribute might be absent is disease: Optional[Disease] = None. Every Creature carries the field; healthy creatures carry None. This looks free - None is a singleton, after all - but every instance still pays one slot, every iteration still pays one getattr, and the storage still scales with population, not with prevalence.

From code/measurement/empty_tables.py, one million creatures with a disease field at four prevalence levels:

Read the 0% row first. With zero diseased creatures, the optional-field layout still costs 105.9 MB of RAM and 7.46 ms per tick of “process disease.” It pays full population price for state that does not exist. The presence layout pays 0.02 ms - function call plus an empty fancy-index - and an extra ~0 KB for the empty diseased array. At zero prevalence, the optional layout is 365× slower than the presence layout, and 4× heavier in memory. The optional layout is not paying for what is happening; it is paying for what might happen.

Read the 10% row. The presence layout pays 0.48 ms - proportional to the 100,000 active rows. The optional layout pays 19 ms - proportional to the full population of one million, because the loop walks every creature to check the is None predicate. The ratio shrinks from 365× to 40× as prevalence rises, but the presence layout always wins, and at typical sparsities (≪ 10% of population is in any specific state at any specific tick) the gap stays large.

The lesson generalises. For every condition you might think of as “optional state” - disease, held_item, target, cooldown_until, aimed_at, fingerprint, last_login_ip, parent_pointer - the disciplined Python form is a separate presence table that contains only the entities that have it right now, not an Optional[X] field on every entity.

Activity-based costs

The effect compounds across many states. A simulation with twenty possible behaviours, each represented as a presence table, pays for the fraction of creatures actually exhibiting each behaviour. Most ticks, most tables are nearly empty. The total work is proportional to the sum of active rows across all tables, not to population × number of behaviours. For a sparsely active world this is one or two orders of magnitude cheaper than the equivalent flag-based design.

A subtle case worth naming: an empty system is not the same thing as a missing system. A drive_hunger system that iterates an empty hungry is still in the DAG, still scheduled, still part of the program’s contract. It is just doing zero rows of work this tick. Removing it from the DAG entirely would change the contract; adding it back when the table next gains a row would require dynamic scheduling, which is harder than a no-op call. EBP gives you cheap idle systems, not absent ones.

Three implications

Activity-based costs. A simulator’s per-tick cost is set by what is active, not by what exists. A million dormant creatures cost nothing to ignore. Only behaving creatures consume budget. Most simulators in production rely on this - game worlds with hundreds of thousands of NPCs but only a few in active play, training simulations with millions of agents but few in critical phases, control systems with thousands of sensors but few in alarmed state.

Structural sparsity. The world is encouraged to be in mostly-resting states. Designs that scatter activity across many small presence tables (lots of cheap idle systems) outperform designs that concentrate activity in a single big “active creatures” flag. The data-oriented mindset is to multiply states (hungry, sleepy, mating, fighting, …) rather than gate behaviour through one master switch.

Persistence is also activity-based. A snapshot of an empty hungry table is one row in the schema and zero rows of data. A snapshot of an is_hungry: np.ndarray of length 1,000,000 is 1 MB regardless of how many bits are set. Backups, replication, and replay all benefit from the same property.

The flag-based mind sees idle objects as “still present, just inactive”. The data-oriented mind sees idle objects as not in the table. The difference is one of cost: the former pays for what exists; the latter pays for what is happening.

Exercises

1. Time the empty case. With your simulator from §19, run a tick where hungry is empty. Time drive_hunger. It should be in the microseconds range - function call plus empty fancy-index, no inner work.
2. Time the same case in flag form. Run the bool-mask version of drive_hunger against a 1,000,000-creature world where is_hungry.sum() == 0. Time it. Should be milliseconds - the mask scan still walks the whole column, even though nothing matches.
3. Run the exhibit. uv run code/measurement/empty_tables.py. Read the 0% row first. Note the absolute cost of the optional layout when nothing is diseased. Note the ratio of optional/presence widening as prevalence drops.
4. The cost-per-active-creature plot. Run the EBP simulator with hungry size ranging over 0, 100, 1,000, 10,000, 100,000, 1,000,000. Time drive_hunger at each. Plot. The line is roughly linear in K, starting at near-zero.
5. Add four more states. Add sleepy, mating, fighting, idle as presence tables, each with its own driver system. Run a tick where most tables are empty (most creatures are in idle). Confirm the per-tick cost is roughly the cost of the idle driver only, plus negligible per-system overhead.
6. Activity histogram. At each tick, log (tick, table_name, len) for every presence table. After 1000 ticks, plot len over time. The plot is the simulator’s activity profile; flat lines mean the world is at rest, bumps mean events are firing.
7. (stretch) Idle systems removed? Argue why removing an empty system from the DAG (rather than running it with zero work) is the wrong move. Hint: it changes the system DAG, breaks determinism if the table is non-empty next tick, and adds dynamic scheduling cost that exceeds the empty-call overhead.
8. (stretch) The Optional[X] sweep. Search any Python project you have. Count Optional[-typed fields on data classes. For each, ask: at runtime, what fraction of instances actually have it set? If the answer is “almost none,” that field is a candidate for a presence table.

Reference notes in 20_empty_tables_are_free_solutions.md.

What’s next

You have closed Existence-based processing. The next phase is Memory & lifecycle, starting with §21 - swap_remove. The simulator is about to start making structural changes to its tables - births and deaths, in production volumes - and the lifecycle phase makes those cheap.

Solutions: 20 - Empty tables are free

Exercise 1 - Time the empty case

import numpy as np, timeit

energy = np.zeros(1_000_000, dtype=np.float32)
hungry = np.empty(0, dtype=np.uint32)            # empty table

def drive_hunger_ebp(energy, hungry, dt):
    energy[hungry] -= 0.5 * dt

t = timeit.timeit(lambda: drive_hunger_ebp(energy, hungry, 1/30), number=10_000) / 10_000
print(f"drive_hunger on empty table: {t*1e6:.2f} µs")

drive_hunger on empty table: ~1-3 µs

A function call, a fancy-index of length zero, an __isub__ on a zero-length view. Microseconds. The system is “in the DAG” but pays almost nothing this tick.

Exercise 2 - Time the same case in flag form

is_hungry = np.zeros(1_000_000, dtype=bool)      # all False - nothing hungry

def drive_hunger_flag(energy, is_hungry, dt):
    energy[is_hungry] -= 0.5 * dt

t = timeit.timeit(lambda: drive_hunger_flag(energy, is_hungry, 1/30), number=1_000) / 1_000
print(f"drive_hunger on all-False mask: {t*1e6:.0f} µs")

drive_hunger on all-False mask: ~150-200 µs

~100× the EBP cost. The mask scan walks all 1M booleans to determine that none are set; numpy still has to materialise the (empty) result of energy[is_hungry]. The “zero work to do” is invisible to the dispatch - the predicate is consulted on every element regardless of the answer.

Exercise 3 - Run the exhibit

uv run code/measurement/empty_tables.py

 prevalence   layout                                  RSS (MB)   tick (ms)
--------------------------------------------------------------------------
     0.00%   list[Creature] with Optional[Disease]      106.4       8.88
     0.00%   numpy SoA + diseased presence              26.6       0.02
     0.10%   list[Creature] with Optional[Disease]      106.3       7.66
     0.10%   numpy SoA + diseased presence              26.7       0.04
     1.00%   list[Creature] with Optional[Disease]      107.1       8.65
     1.00%   numpy SoA + diseased presence              26.8       0.13
    10.00%   list[Creature] with Optional[Disease]      113.8      17.55
    10.00%   numpy SoA + diseased presence              26.5       0.56

The 0% row is the headline: zero diseased creatures, but the optional-field layout costs 8.88 ms per tick to discover this. The presence layout costs 0.02 ms - function-call overhead and an empty fancy-index. The optional layout pays full population price for state that does not exist.

The widening ratio at low prevalence (445× at 0.0%, 191× at 0.1%, 67× at 1%, 31× at 10%) shows that the optional cost is dominated by the iteration, not by the work - the loop walks all 1M creatures regardless of how few have a disease.

Exercise 4 - The cost-per-active-creature plot

import numpy as np, timeit
energy = np.zeros(1_000_000, dtype=np.float32)
results = []
for k in [0, 100, 1_000, 10_000, 100_000, 1_000_000]:
    hungry = np.arange(k, dtype=np.uint32)
    t = timeit.timeit(lambda: drive_hunger_ebp(energy, hungry, 1/30), number=200) / 200
    results.append((k, t * 1e6))
    print(f"K={k:>10}: {t*1e6:>8.1f} µs")

K=         0:       1.5 µs
K=       100:       2.4 µs
K=     1,000:       3.7 µs
K=    10,000:      14.2 µs
K=   100,000:     143.0 µs
K= 1,000,000:    1820.0 µs

Roughly linear in K above ~1000. Below that, the line is dominated by per-call overhead - the work itself disappears into noise. The plot is “y = a + b·K” with a ≈ 1.5 µs (overhead) and b ≈ 1.8 ns (per-active-creature work).

The line starts at near-zero because EBP’s cost depends on K, not N. A flag-based plot would be a flat line at ~150 µs (the mask-scan cost) regardless of K. The two strategies have different shapes.

Exercise 5 - Add four more states

hungry  = np.flatnonzero(energy < 10)
sleepy  = np.flatnonzero(energy > 80)
mating  = np.empty(0, dtype=np.uint32)
fighting = np.empty(0, dtype=np.uint32)
idle    = np.flatnonzero((energy >= 10) & (energy <= 80))    # the bulk

def tick(world, dt):
    drive_hunger(world.hungry, world.energy, dt)
    drive_sleep(world.sleepy, world.energy, dt)
    drive_mating(world.mating, world, dt)         # empty - near-zero cost
    drive_fighting(world.fighting, world, dt)     # empty - near-zero cost
    drive_idle(world.idle, world.energy, dt)

If mating and fighting are empty most ticks, the per-tick cost is:

• ~1 µs each for drive_mating and drive_fighting (empty tables)
• The actual work for hungry, sleepy, idle proportional to their sizes

Total: dominated by idle (which holds most of the population) plus small contributions from hungry/sleepy, plus negligible overhead from the empty tables. A simulator can have dozens of dormant systems without paying for them.

Exercise 6 - Activity histogram

activity_log: list[tuple[int, str, int]] = []

for tick_n in range(1000):
    tick(world, dt)
    for name in ("hungry", "sleepy", "mating", "fighting", "idle", "dead"):
        activity_log.append((tick_n, name, len(getattr(world, name))))

import collections
by_table = collections.defaultdict(list)
for t, name, n in activity_log:
    by_table[name].append((t, n))

# plot each name's series - flat lines = resting world, bumps = events

The activity profile is the simulator’s behaviour. A trace where hungry and dead stay flat near 0 means the population is well-fed and stable; bumps mean a food shortage hit; a stairstep up means births are outpacing deaths. The same numbers that drive the per-tick cost are also the simulator’s “vital signs.” Free observability.

Exercise 7 - Idle systems removed? (stretch)

Removing an empty system from the DAG sounds like a free optimisation. It is not. Three reasons:

1. Determinism breaks. The DAG is the contract; a system’s position in the DAG is part of its definition. Run A removes drive_mating because the table is empty; run B (one tick later, after a creature has entered mating) puts it back. The execution order has changed; the world hash changes; replay no longer reproduces.

2. Re-adding it has scheduling cost. When mating next gains a row, the system must be inserted back into the DAG and topo-sorted. Topo-sort is cheap (microseconds for a small DAG) but it is not free, and it pays this cost on every transition between empty and non-empty. The empty-call overhead it was supposed to save was also microseconds. The fix is more expensive than what it fixes.

3. The contract is now dynamic. Static DAG: every run executes the same sequence of systems in the same order. Dynamic DAG: the sequence depends on the run’s state. Reasoning about the simulator (which systems run when, what they read and write, what determinism property holds) becomes much harder. Empty calls are cheap; dynamic schedules are not.

The right move is to keep all systems in the DAG, accept the few microseconds of overhead per empty system per tick, and design states so most are sparse. A simulator with 30 systems and a 30 Hz tick budget can afford 30 µs of empty-call overhead - under 0.1% of the budget.

Exercise 8 - The Optional[X] sweep (stretch)

A quick sweep of any Python project for Optional[-typed fields:

grep -rE 'Optional\[|: [A-Z][a-zA-Z]* \| None|: None \|' src/

For each hit, ask: at runtime, what fraction of instances actually have it set?

• disease: Optional[Disease] - 0-2% of creatures. Strong candidate for a diseased presence table.
• held_item: Optional[Item] - 30-60%. Closer; the trade depends on access pattern. If most systems just need to know whether an item is held, presence wins. If they need the item type, a column might be simpler.
• parent: Optional[Self] - varies. Trees with many leaves and few internal nodes: presence wins. Balanced trees: column wins.
• last_login_at: Optional[datetime] - 99% of users have logged in. Column wins; the Optional wrapper is just defensive coding for the never-logged-in edge case.

The pattern: Optional fields with low fill-rate are presence tables waiting to be discovered. Optional fields with high fill-rate are columns with a sentinel that means “not yet” (a magic timestamp, 255 in a uint8, etc.).

21 - swap_remove

Concept node: see the DAG and glossary entry 21.

The presence-replaces-flags substitution from §17 raised a problem we deferred. When a creature stops being hungry, you remove its slot from hungry. When a creature dies, you remove its row from every table. Removing rows from the middle of an array is expensive - every later row has to shift left by one, costing O(N).

For a 1,000,000-creature simulator with 1,000 deaths per tick, naive remove costs roughly 10⁹ moves per tick - far past the budget of any real-time loop.

Python gives you four options. Two are wrong, two are right - and the right two are right in different situations.

Four options, ranked

# anti-pattern: bad!
lst.pop(i)            # O(N) - shifts every subsequent element left
np.delete(arr, i)     # O(N) plus a fresh allocation - usually the slowest

# disciplined: per-element swap_remove with an active counter
arr[i] = arr[n_active - 1]   # move last live element into the freed slot
n_active -= 1                # the "table" is now arr[:n_active]

# disciplined and faster: bulk filter when you have K indices in hand
keep_mask = np.ones(n_active, dtype=bool)
keep_mask[indices_to_remove] = False
arr = arr[keep_mask]         # one C-level pass; survivors keep original order

The mechanism for the per-element version is small: take the last live element, move it into the deleted slot, decrement the active count by one. Two memory writes and a counter decrement. O(1) regardless of N. The “active counter” pattern means you allocate a numpy column once at the maximum size you need, and n_active tells you how many rows are currently in use. The table is the prefix arr[:n_active]. Removing a row never resizes the backing storage; only the counter changes. Inserting a row writes to arr[n_active] and increments. (Insertion details in §24 - Append-only and recycling.)

The bulk-filter version takes a batch of indices and processes them in a single numpy call. It allocates a fresh column of size n_active - K, but pays the allocation only once for the whole batch instead of once per row. It is the natural pair to §22 - Mutations buffer, which is exactly the pattern of “collect K removes during the tick; apply them all at once at the boundary.” The batch is the unit of work; the single numpy call is the application.

The SoA reminder from §6 still applies. Both the per-element swap_remove and the bulk filter are single-column operations as shown above - and a creature table is six or eight columns, not one. Removing creature i is pos_x[i] = pos_x[-1]; pos_y[i] = pos_y[-1]; ...; n_active -= 1 across every column with the same i. The bulk-filter form is the same shape - one keep_mask computed once, applied to every column with the same indices, in lockstep. Apply it to half the columns and rows go out of alignment, exactly the bug from §9. The discipline is the same as it was for sort: every operation that reorders any column reorders all columns of that table together.

Cost, measured

From code/measurement/swap_remove.py, removing 100,000 mid-table rows from a 1,000,000-row table on this machine:

Four readings.

np.delete is the worst. This will surprise readers who reach for it because it sounds like the “numpy way” to remove a row. It is not - np.delete returns a new array with the element removed, allocating fresh memory and copying the surviving elements every call. At 100,000 sequential deletes from a 1M-row array, you allocate 100,000 progressively-shrinking arrays. The bytes are typed, the operation is C-level, and it is still ~5,700× slower than the bulk filter because the algorithmic shape is wrong.

list.pop(i) is the AoS middle ground, but only because Python lists are pointer arrays - shifting an N-element list is N pointer copies, which is faster than shifting and reallocating an N-element typed numpy array. Either way: O(N) per remove, ~950× slower than the bulk filter.

Sequential swap_remove processes ~7.8 million removes per second. Each remove is O(1), but the loop that drives it crosses the Python-numpy boundary 100,000 times - one bounds check, one assignment, one n_active -= 1 per iteration. That overhead is the only thing keeping it from being the fastest line in the table.

Bulk filter processes ~25.6 million removes per second - ~3× faster than sequential swap_remove. The boolean-mask pass and the compress are both single C-level operations over the whole array. The Python interpreter is touched once, not 100,000 times. This is the version the simulator’s cleanup pass should use whenever it has a buffer of indices to remove.

Reading the table together: per-element swap_remove is the right tool when you genuinely have one row to remove (rare). Bulk filter is the right tool when you have a buffer of K indices (the typical case once buffering is in place - §22). Both forms beat the AoS reflexes by orders of magnitude. The choice between them is set by whether the buffering pattern from §22 has happened upstream.

Cost paid

Order is sacrificed. If your code depended on rows being in any particular order, swap_remove reorders them. Two specific consequences:

• Iteration corruption. If you iterate the table and call swap_remove during iteration, the slot you just visited now holds a different row, but your loop counter has moved past it. Half the rows after a swap_remove get skipped or revisited inconsistently. (The same iterate-and-mutate footgun from §15.)
• Slot references break. The row that backfilled the hole used to live at the end; now it sits at slot i under an index nobody was told about. Every slot-keyed table from §17 - hungry, sleepy, the rest - now lists a slot that points at the wrong creature. This is the same bug as §9: rearrangement breaks slot-based references, and a slot-keyed hungry is nothing but slot-based references.

Both problems have fixes already named in the book. The iteration corruption is fixed by §22 - Mutations buffer: swap_remove never runs during iteration; it runs during cleanup at the tick boundary, when no system is iterating. The moved-slot problem is fixed in two steps by §23 - Index maps: the cleanup rewrites the moved slot wherever a slot-keyed table holds it (a reindex), and an id_to_slot map lets anything holding a stable id re-find the creature after the move. Then §24 asks the sharper question: if moving a slot is this much trouble, why move it at all? Mark the creature dead, recycle its slot later, and every reference stays standing. swap_remove on death is the honest first cut - the wrong way that earns the right one.

When the lifecycle phase matters

This whole phase - Memory & lifecycle - only matters for variable-quantity tables. Constant-quantity tables like the 52-card deck never grow or shrink, never need swap_remove, never need any of the machinery in this phase. The card game ran for ten chapters without it. The simulator from §11 onward needs all of it, because creatures are born and die every tick.

The constant vs variable distinction is what determines whether a programmer reaches into the lifecycle toolbox at all. Once you have a table whose row count varies at runtime, every tool in this phase becomes load-bearing.

Exercises

1. Compare timings, simple case. Build a list of length 1,000,000. Time 1,000 calls to lst.pop(0) (front delete, the worst case). Time the same with the swap_remove pattern (lst[0] = lst[-1]; lst.pop()). The ratio is roughly N.
2. Mid-table delete. Build a numpy int64 array of length 1,000,000. Time 1,000 calls to np.delete(arr, 500_000) (rebinding arr each time). Time 1,000 calls to the swap_remove pattern (arr[500_000] = arr[n_active - 1]; n_active -= 1). The ratio is enormous - np.delete allocates a fresh array each call.
3. Run the §21 exhibit. uv run code/measurement/swap_remove.py. Note the order of the four rows. Confirm np.delete is the slowest, not the fastest, despite being the “numpy way.” Note the gap between sequential swap_remove and bulk filter - both are O(K) algorithmically, but the bulk version pays the Python-loop overhead once instead of K times.
4. The iteration hazard. Build a numpy int64 array of length 100 with values 0 through 99 (np.arange(100)) and an n_active = 100. In a forward loop, iterate i in range(n_active) and apply swap_remove whenever arr[i] % 2 == 0. Compare with the expected output (only odd values remaining). What did you actually get? (Spoiler: you missed half the evens.)
5. The fix in one shape: iterate backwards. Repeat exercise 4, but iterate range(n_active - 1, -1, -1). Does it work now? Why does it work?
6. The fix in another shape: deferred cleanup. Repeat exercise 4, but instead of calling swap_remove inside the loop, append the index to to_remove. After the loop, sort to_remove in reverse order and apply swap_remove. This is the §22 pattern in miniature.
7. Aligned per-element swap_remove. Build the simulator’s six creature columns (pos_x, pos_y, vel_x, vel_y, energy, id). Write def delete_creature(world, slot) that calls swap_remove on every column in lockstep. Verify all columns remain aligned after a sequence of deletes.
8. Aligned bulk filter. Take the same six creature columns. Write def delete_batch(world, indices_to_remove) that builds one keep_mask and applies it to every column. Verify alignment by spot-checking row 17 (the (pos_x[17], pos_y[17], ..., id[17]) tuple) before and after the batch - its values should match the original row whose id is now at slot 17. Now write the broken version that applies the mask to only some columns; verify that row alignment is destroyed exactly as §9 predicted. The single-column bulk filter shown in the prose is for clarity; the table version always reads the mask once and uses it everywhere.
9. (stretch) The bandwidth cost. Compute the bytes moved by np.delete(arr, 0) on a 1 GB int64 array: roughly the whole 1 GB (the source array, copied minus the deleted element). Compute the same for the swap_remove pattern: roughly 8 bytes (one int64 move). The ratio is N / 1. Verify with tracemalloc or psutil.

Reference notes in 21_swap_remove_solutions.md.

What’s next

§22 - Mutations buffer; cleanup is batched is the rule that makes swap_remove safe to use: it never runs while any system is iterating.

Solutions: 21 - swap_remove

Exercise 1 - Compare timings, simple case

import time
N = 1_000_000

# pop(0) - worst case
lst = list(range(N))
t0 = time.perf_counter()
for _ in range(1_000):
    lst.pop(0)
t1 = time.perf_counter()
print(f"pop(0) × 1000:       {(t1-t0)*1000:.1f} ms")

# swap_remove at front
lst = list(range(N))
t0 = time.perf_counter()
for _ in range(1_000):
    lst[0] = lst[-1]; lst.pop()
t1 = time.perf_counter()
print(f"swap_remove × 1000:  {(t1-t0)*1000:.3f} ms")

pop(0) × 1000:       380 ms
swap_remove × 1000:    0.2 ms

~2000× ratio. pop(0) shifts every element down one slot - N pointer copies per call, K × N total. swap_remove writes one slot and pops - 2 ops per call, K total. The ratio scales with N.

Exercise 2 - Mid-table delete

import time, numpy as np
N = 1_000_000

# np.delete returns a fresh array each call
arr = np.arange(N, dtype=np.int64)
t0 = time.perf_counter()
for _ in range(1_000):
    arr = np.delete(arr, 500_000 if len(arr) > 500_000 else 0)
t1 = time.perf_counter()
print(f"np.delete × 1000:    {(t1-t0)*1000:.0f} ms")

# swap_remove
arr = np.arange(N, dtype=np.int64)
n_active = N
t0 = time.perf_counter()
for _ in range(1_000):
    i = 500_000 if i < n_active - 1 else 0
    arr[i] = arr[n_active - 1]
    n_active -= 1
t1 = time.perf_counter()
print(f"swap_remove × 1000:  {(t1-t0)*1000:.3f} ms")

np.delete re-allocates the entire array each call - N int64s = 8 MB copied per delete. After 1000 calls, 8 GB has been written. swap_remove writes 8 bytes per call. ~10⁵-10⁶× ratio.

Exercise 3 - Run the §21 exhibit

uv run code/measurement/swap_remove.py

Source: code/measurement/swap_remove.py. Removing 100,000 mid-table rows from a 1M-row table:

layout                                              time      remove rate
-------------------------------------------------------------------------
Python list, list.pop(i)                            3.59 s          28K ops/s
numpy, np.delete(arr, i)                           23.09 s           4K ops/s
numpy active counter, sequential swap_remove       0.011 s        8.0M ops/s
numpy bulk filter, arr[keep_mask]                  0.004 s       25.4M ops/s

Surprises that calibrate intuition:

• np.delete is the slowest - ~5,700× slower than the bulk filter. The “numpy way” sounds right but is algorithmically wrong: it reallocates on every call.
• Python list pop(i) beats np.delete at this scale, because pointer-shifts in a Python list are ~8 bytes each whereas reallocation copies the whole int64 array.
• Bulk filter is 3× faster than sequential swap_remove, even though both are O(K). The Python loop crossing the C boundary 100K times has measurable overhead; the bulk version pays the boundary cost once.

For a simulator’s cleanup pass: collect to_remove indices during the tick (cheap, append-only), then apply with one bulk-filter call at the boundary. This is the §22 pattern.

Exercise 4 - The iteration hazard

import numpy as np
arr = np.arange(100, dtype=np.int64)
n_active = 100

i = 0
while i < n_active:
    if arr[i] % 2 == 0:
        arr[i] = arr[n_active - 1]
        n_active -= 1
    else:
        i += 1

print(arr[:n_active].tolist())     # should be [1, 3, 5, ..., 99]

Without the else: i += 1 (i.e. plain for i in range(...)), the bug is: after a swap, the slot at i now holds a different value (the one that was at the end). The forward for loop has already moved past i and won’t re-check it. Half the evens get skipped.

The version above with the explicit while and conditional increment is correct: when a swap happens, don’t advance i - the slot has new contents that need re-checking. This is the canonical fix when iterating-while-mutating cannot be avoided.

Exercise 5 - The fix in one shape: iterate backwards

arr = np.arange(100, dtype=np.int64)
n_active = 100
for i in range(n_active - 1, -1, -1):
    if arr[i] % 2 == 0:
        arr[i] = arr[n_active - 1]
        n_active -= 1
print(arr[:n_active].tolist())     # all odds

Why it works: when you swap arr[i] = arr[n_active - 1], the slot at i now holds the old last element, but i is decreasing, so we move to i - 1 next - a slot we have not yet visited. We never re-encounter a swapped slot. The “old last” element gets to be checked for evenness at its new position because that position was never visited by the iteration.

Exercise 6 - The fix in another shape: deferred cleanup

arr = np.arange(100, dtype=np.int64)
n_active = 100

to_remove = []
for i in range(n_active):
    if arr[i] % 2 == 0:
        to_remove.append(i)

# Apply at end - reverse order so swap_remove indices stay valid
for i in sorted(to_remove, reverse=True):
    arr[i] = arr[n_active - 1]
    n_active -= 1

print(arr[:n_active].tolist())

The for loop is now read-only - no swap during iteration. Mutations are buffered and applied at the boundary (for i in sorted(to_remove, reverse=True)). This is the §22 pattern: filter (read-only) and apply (single batch) are separate phases.

For very large to_remove buffers, the bulk-filter form (arr = arr[~np.isin(np.arange(n_active), to_remove)]) is faster than per-index swap_remove. Both forms are correct; the bulk one wins on speed.

Exercise 7 - Aligned per-element swap_remove

class World:
    def __init__(self, n):
        self.pos_x   = np.zeros(n, dtype=np.float32)
        self.pos_y   = np.zeros(n, dtype=np.float32)
        self.vel_x   = np.zeros(n, dtype=np.float32)
        self.vel_y   = np.zeros(n, dtype=np.float32)
        self.energy  = np.zeros(n, dtype=np.float32)
        self.id      = np.arange(n, dtype=np.uint32)
        self.n_active = n

def delete_creature(world: World, slot: int) -> None:
    last = world.n_active - 1
    if slot != last:
        for arr in (world.pos_x, world.pos_y, world.vel_x,
                    world.vel_y, world.energy, world.id):
            arr[slot] = arr[last]
    world.n_active -= 1

Each column gets the same slot and last. Forgetting to apply this to one column produces the §9 misalignment bug. The discipline is: the function above is the only place that does swap_remove on a creature; no caller writes to one column without going through it.

Exercise 8 - Aligned bulk filter

def delete_batch(world: World, indices_to_remove: np.ndarray) -> None:
    keep = np.ones(world.n_active, dtype=bool)
    keep[indices_to_remove] = False
    for name in ("pos_x", "pos_y", "vel_x", "vel_y", "energy", "id"):
        col = getattr(world, name)
        # in-place compress: copy survivors to the front
        n_keep = int(keep.sum())
        col[:n_keep] = col[:world.n_active][keep]
    world.n_active -= len(indices_to_remove)

# spot-check alignment
row17_before = (world.pos_x[17], world.pos_y[17], int(world.id[17]))
delete_batch(world, np.array([5, 13, 87]))
# whichever creature is now at slot 17 - its row tuple should still be coherent
row17_after  = (world.pos_x[17], world.pos_y[17], int(world.id[17]))
# verify that row17_after matches the row in the original world whose id == world.id[17] now

The same keep mask is applied to every column. One boolean indexing pass per column, one mask shared across all columns. Forgetting one column lands the broken version: rows misaligned exactly as §9 predicted.

The broken version (apply mask to half the columns):

# anti-pattern: bad! demonstrates the bug
def delete_batch_broken(world, indices):
    keep = np.ones(world.n_active, dtype=bool)
    keep[indices] = False
    world.pos_x = world.pos_x[keep]
    world.pos_y = world.pos_y[keep]
    # forgot vel_x, vel_y, energy, id - they keep their old length and contents

Now pos_x[i] and vel_x[i] are from different rows. Reading “the velocity of the creature at slot i” returns garbage. The fix is structural: one function, all columns, one mask.

Exercise 9 - The bandwidth cost (stretch)

import numpy as np

# np.delete on a 1 GB int64 array - copies (1 GB - 8 bytes)
arr = np.zeros(1_000_000_000 // 8, dtype=np.int64)   # 125 M elements = 1 GB
arr2 = np.delete(arr, 0)                              # bytes moved ≈ 1 GB

# swap_remove - copies 8 bytes
n_active = len(arr)
arr[0] = arr[n_active - 1]
n_active -= 1                                          # bytes moved = 8

Bytes moved per delete:

Ratio: ~125,000,000×. At a 30 Hz tick rate, doing one np.delete per tick on a 1 GB array would mean moving 30 GB/s - past the bandwidth ceiling of most desktop systems. Doing one swap_remove takes 30 × 8 = 240 bytes per second.

The structural cost dominates the constant factors. This is why the chapter’s table shows np.delete losing to list.pop at this scale: the algorithmic shape is wrong regardless of how typed and contiguous the data is.

22 - Mutations buffer; cleanup is batched

Concept node: see the DAG and glossary entry 22.

This rule has been forward-referenced through ten chapters. Time to make it concrete.

Mutations during a tick do not apply immediately; they queue, and a cleanup pass applies them in a batch at the boundary rather than one at a time mid-tick. (When the compaction part of that batch actually runs - every tick, or on a slower cadence - is the refinement in “When the compress runs” below.) The shape:

@dataclass
class CleanupBuffer:
    to_remove: list[int]               # creature ids to delete this tick
    to_insert_pos_x: list[float]       # parallel arrays of inserted-row data
    to_insert_pos_y: list[float]
    to_insert_vel_x: list[float]
    to_insert_vel_y: list[float]
    to_insert_energy: list[float]
    to_insert_id: list[int]

(The insert side has one list per column. Per §6, a row is a tuple-at-index, and that’s true of the insert buffer too - it is an SoA buffer, not a list of Creature objects. The reason is the same reason the rest of the simulator is SoA: numpy gets to work on the bytes when cleanup runs.)

During the tick, every system that wants to delete appends an id to to_remove. Every system that wants to add appends one row’s worth of data to the parallel insert columns. No system mutates the live tables.

The cleanup pass

At the end of the tick, one system runs:

def cleanup(world: World, buffer: CleanupBuffer) -> None:
    # 1. Removals: build one keep_mask, apply to every column at once.
    if buffer.to_remove:
        ids_to_remove = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids_to_remove]              # see §23
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[: keep_mask.sum()] = col[: world.n_active][keep_mask]
        world.n_active = int(keep_mask.sum())
        # (Update id_to_slot - covered in §23.)
        buffer.to_remove.clear()

    # 2. Insertions: bulk concatenate parallel insert columns into the table.
    n_inserts = len(buffer.to_insert_id)
    if n_inserts:
        new_n = world.n_active + n_inserts
        # The columns were sized at maximum capacity at startup; we are
        # writing into the previously unused tail [n_active : new_n).
        world.pos_x[world.n_active : new_n] = buffer.to_insert_pos_x
        world.pos_y[world.n_active : new_n] = buffer.to_insert_pos_y
        world.vel_x[world.n_active : new_n] = buffer.to_insert_vel_x
        world.vel_y[world.n_active : new_n] = buffer.to_insert_vel_y
        world.energy[world.n_active : new_n] = buffer.to_insert_energy
        world.id[world.n_active : new_n] = buffer.to_insert_id
        world.n_active = new_n
        # (Append the new ids to id_to_slot - §23.)
        for lst in (buffer.to_insert_pos_x, buffer.to_insert_pos_y,
                    buffer.to_insert_vel_x, buffer.to_insert_vel_y,
                    buffer.to_insert_energy, buffer.to_insert_id):
            lst.clear()

Two passes, both bulk operations. The world is in a fully consistent state at the end. The keep_mask is built once and applied to every column; the insert tail is filled with one slice assignment per column. Per §21, the bulk-filter form is 5× faster than per-element swap_remove at K=100,000 mutations per tick - and per the editions-diverge framing in the prose of §10 and elsewhere, this is where the Python edition’s cleanup actually diverges from the Rust edition’s: Rust §22 uses a per-element swap_remove loop because compiled code pays no interpreter-boundary tax; Python §22 uses the bulk-mask form because we measured the boundary cost and it dominates at scale.

What this fixes

The iteration-corruption problem from §21 goes away because the table is never mutated while any system is iterating. By the time cleanup runs, every system has finished. There is no concurrent iteration to confuse. The list-during-iteration and dict-during-iteration footguns from §15 cannot happen - there is no creatures.remove(c) inside a for c in creatures loop, because nothing inside the tick mutates the live tables.

The race-condition problem from concurrent mutation goes away. Two systems may both want to remove a creature; both append to to_remove; cleanup deduplicates with np.unique. Neither system needs to coordinate.

The composition problem from §14 goes away. Systems read consistent snapshots; they read the world as it was at tick start, not the world as some other system half-rewrote it.

What it costs

Every mutation is one extra entry pushed to a side list. For a simulator with 1,000 deaths and 500 reproductions per tick, that is 1,500 entries of bookkeeping per tick - a few thousand bytes, completely negligible against the cost of running the systems themselves.

The cleanup pass is one additional system in the DAG. It is empty (no work) when no mutations are queued (§20); it runs the bulk filter and bulk concatenate when there are. The system is wired in once and never removed.

When the compress runs: mark dead now, compact on a cadence

The cleanup above does two jobs, and they want different cadences. The insert side is cheap - a slice-write into the tail. The remove side is the keep-mask compress, and it moves every surviving row to close the holes the dead rows left. At a million creatures that compress is ~13 ms (it rewrites every column), and now that subscriptions are keyed by slot (§26), every move also reindexes every subscription. Run every tick, it is the most expensive thing in the loop.

It does not have to run every tick. A death needs two things immediately: the creature must stop being processed, and references to it must stop resolving. Neither requires moving a row. So per tick a death marks its slot dead - it bumps the slot’s generation (§10/§24), the dirty marker, and unsubscribes from its subscription tables (§23’s sparse set, O(1)). The slot stays where it is. The compress - now a garbage-collection pass - runs on a slow cadence, every few dozen ticks, reclaiming all the accumulated dead slots in one sweep and reindexing id_to_slot and the subscriptions once. It is the same pass as §28’s cell-ordered compaction, at the cadence §26 recommends.

Between GC passes the dead slots sit in the columns as holes. A scan-all system like motion strides over them and updates a few dead rows harmlessly - they are not rendered or scored, and the generation check rejects any reference that reaches them. Measured, that wasted work is negligible.

The numbers settle the cadence. Per-tick compaction (A) against mark-dead-plus-GC (B), over GC interval G:

Ryzen 9 270, 1M creatures, 1,000 deaths/tick; reproduce with ebp_partition.py claim C5. At G = 1 the two collapse to the same compress plus the extra mark-dead, so per-tick wins; from G = 10 on, deferring the move pays. The recycle-versus-not distinction does not even show up - the dead-hole waste is too small to measure against the compress. The verdict holds on every reference machine: at G = 30 the deferred model is cheaper by 1.55x (Ryzen 9 270), 1.27x (i7-3610QM), 1.63x (i3-5010U), and 1.53x (Pi 4).

So the rule: mark dead every tick, compact on the GC cadence. The bulk-filter compress is not retired; it is relocated to the pass that runs every few dozen ticks instead of every one.

What it does not fix

Dedup is the system’s job. Two systems may both push the same id to to_remove if they independently detect the same death condition. The cleanup uses np.unique(to_remove) to reduce to distinct ids before computing slots. The cost is one O(K log K) sort on a small array - irrelevant against the bulk filter.

Order matters. Inside cleanup, deletions run first, then insertions. If you insert first, an inserted row might land in a slot you are about to delete. Deleting first frees up tail capacity that subsequent inserts can reuse - though slot recycling is its own decision (§24).

The pattern itself is universal. Database transactions buffer writes and commit at the boundary. Graphics pipelines render to a back buffer and swap. Version-controlled file systems collect changes and commit. They all solve the same problem: how do you let many independent operations modify shared state without stepping on each other? The answer is always the same - accumulate, then apply atomically.

Exercises

1. Implement the side buffers. Add to_remove: list[int] and the parallel insert lists (one per column) to your simulator’s world. They are empty at the start of every tick.
2. Push from apply_starve. Modify your starvation system to append to to_remove instead of any direct table mutation. Verify the system no longer touches the live creatures columns.
3. Push from apply_reproduce. Modify reproduction to append the parent’s offspring rows to the parallel insert lists. Verify reproduction no longer mutates creatures directly.
4. Implement bulk cleanup. Write the cleanup system as in the prose. Apply removals first (one keep_mask, applied to every column), then insertions (one slice-write per column). Run a tick with both kinds of mutations; verify the world is consistent after.
5. Compare cleanup forms. Implement a second cleanup that uses per-element swap_remove in a Python loop instead of the bulk mask. Time both at 1,000,000 creatures with 1,000 mutations per tick. The bulk form should win by ~5× per the §21 numbers - confirm on your machine.
6. The dedup question. Push id 42 to to_remove from two different systems in the same tick. Run cleanup without the np.unique step. What happens? (Hint: id_to_slot[42] is looked up twice; the second lookup may produce garbage if the first removal moved another row to that slot.) Now add the np.unique and re-run. The result is correct.
7. Tick-delayed visibility. A creature inserted in tick 5 (via the to_insert_* lists) does not appear in the live columns during tick 5’s systems - only at the end, in cleanup. Verify by adding an age_in_ticks column that increments at the end of each tick; the new creature’s value starts at 0 in tick 6, not tick 5.
8. (stretch) A graphics pipeline analogy. A rendering pipeline draws to a “back buffer” while the “front buffer” is being displayed. At the boundary of one frame to the next, the buffers swap. Argue why this is the same pattern as to_remove / to_insert plus cleanup. (Hint: it is the same atomic-commit shape; the back buffer is exactly the side table.)
9. (stretch) Cadence of the compress. Run the cleanup compress every tick (A); then run a version that marks each death dead - bump a generation, unsubscribe from its subscriptions - and compacts only every 30 ticks (B). Time both at 1M creatures with 1,000 deaths per tick. Reproduce the C5 crossover: B should win from a GC interval near 10 onward. Confirm the dead-hole waste in a scan-all motion pass is negligible.

Reference notes in 22_mutations_buffer_solutions.md.

What’s next

§23 - Index maps is the missing piece for swap_remove and bulk-filter cleanup to be useful: a parallel data structure that tracks where every id currently lives, updated whenever the columns move.

Solutions: 22 - Mutations buffer; cleanup is batched

Exercise 1 - Implement the side buffers

from dataclasses import dataclass, field

@dataclass
class CleanupBuffer:
    to_remove: list[int] = field(default_factory=list)
    to_insert_pos_x:   list[float] = field(default_factory=list)
    to_insert_pos_y:   list[float] = field(default_factory=list)
    to_insert_vel_x:   list[float] = field(default_factory=list)
    to_insert_vel_y:   list[float] = field(default_factory=list)
    to_insert_energy:  list[float] = field(default_factory=list)
    to_insert_id:      list[int]   = field(default_factory=list)

# tick boundary: clear everything
buffer = CleanupBuffer()

The insert side is parallel column lists, not a list of objects. The whole point of the simulator’s SoA discipline is that “a row to insert” is six values across six lists with the same index - exactly like the live tables, just on the side.

For tighter packing, the insert lists could be pre-allocated numpy arrays with their own n_pending counter; for typical mutation rates (hundreds to thousands per tick), Python lists are plenty fast.

Exercise 2 - Push from apply_starve

def apply_starve(world: World, buffer: CleanupBuffer) -> None:
    """Read-set: world.energy, world.id, world.n_active.
       Write-set: buffer.to_remove (only)."""
    starvers = np.where(world.energy[: world.n_active] <= 0)[0]
    starver_ids = world.id[starvers]
    buffer.to_remove.extend(starver_ids.tolist())

The system does not call world.delete_creature(). It does not modify world.energy or world.n_active. It writes only to buffer.to_remove - the live world is untouched until cleanup. A diff between this version and the previous shows: every line that mutated a live column is gone; one extend line replaces all of them.

Exercise 3 - Push from apply_reproduce

THRESHOLD = 100.0

def apply_reproduce(world: World, buffer: CleanupBuffer, rng) -> None:
    """Read-set: world.energy, world.pos_x, world.pos_y, world.id, world.n_active.
       Write-set: buffer.to_insert_* (only). Parent's energy unchanged here;
                  splitting energy is a separate consideration handled by cleanup
                  or a follow-on system."""
    parents = np.where(world.energy[: world.n_active] > THRESHOLD)[0]
    if parents.size == 0:
        return
    n = parents.size
    # offspring inherit parent pos with tiny jitter
    jitter_x = rng.uniform(-0.1, 0.1, n).astype(np.float32)
    jitter_y = rng.uniform(-0.1, 0.1, n).astype(np.float32)
    new_ids  = world.next_ids(n)                              # see §24

    buffer.to_insert_pos_x.extend((world.pos_x[parents] + jitter_x).tolist())
    buffer.to_insert_pos_y.extend((world.pos_y[parents] + jitter_y).tolist())
    buffer.to_insert_vel_x.extend([0.0] * n)
    buffer.to_insert_vel_y.extend([0.0] * n)
    buffer.to_insert_energy.extend([world.energy[parents].mean()] * n)  # half-energy variant in §13
    buffer.to_insert_id.extend(new_ids.tolist())

Reproduction has no direct effect on the world during the tick. The offspring exist as parallel entries in the buffer lists. Cleanup will materialise them.

Exercise 4 - Implement bulk cleanup

def cleanup(world: World, buffer: CleanupBuffer) -> None:
    # 1. Removals (deletes first so freed slots can host inserts in §24's recycling)
    if buffer.to_remove:
        ids = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids]                     # see §23
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False
        n_keep = int(keep_mask.sum())
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[:n_keep] = col[: world.n_active][keep_mask]
        world.n_active = n_keep
        buffer.to_remove.clear()
        # update id_to_slot - see §23

    # 2. Insertions (one slice-write per column)
    n_inserts = len(buffer.to_insert_id)
    if n_inserts:
        new_n = world.n_active + n_inserts
        world.pos_x[world.n_active : new_n]  = buffer.to_insert_pos_x
        world.pos_y[world.n_active : new_n]  = buffer.to_insert_pos_y
        world.vel_x[world.n_active : new_n]  = buffer.to_insert_vel_x
        world.vel_y[world.n_active : new_n]  = buffer.to_insert_vel_y
        world.energy[world.n_active : new_n] = buffer.to_insert_energy
        world.id[world.n_active : new_n]     = buffer.to_insert_id
        world.n_active = new_n
        # update id_to_slot for new ids - see §23
        for lst in (buffer.to_insert_pos_x, buffer.to_insert_pos_y,
                    buffer.to_insert_vel_x, buffer.to_insert_vel_y,
                    buffer.to_insert_energy, buffer.to_insert_id):
            lst.clear()

Two bulk ops. The world is consistent at the end. Spot-check after a tick:

assert len(set(world.id[: world.n_active].tolist())) == world.n_active     # no duplicates

Exercise 5 - Compare cleanup forms

import time, numpy as np
N, K = 1_000_000, 1_000

# Bulk cleanup: arr[keep_mask]
def bulk_cleanup(arr, indices_to_remove):
    keep = np.ones(len(arr), dtype=bool)
    keep[indices_to_remove] = False
    return arr[keep]

# Per-element swap_remove in a Python loop
def per_element_cleanup(arr, indices_to_remove):
    n = len(arr)
    for i in sorted(indices_to_remove, reverse=True):
        arr[i] = arr[n - 1]
        n -= 1
    return arr[:n]

arr = np.arange(N, dtype=np.int64)
indices = np.random.default_rng(0).choice(N, size=K, replace=False)

t = time.perf_counter()
for _ in range(100):
    bulk_cleanup(arr.copy(), indices)
print(f"bulk:        {(time.perf_counter()-t)*10:.2f} ms / call")

t = time.perf_counter()
for _ in range(100):
    per_element_cleanup(arr.copy(), indices.tolist())
print(f"per-element: {(time.perf_counter()-t)*10:.2f} ms / call")

Typical ratio at K=1000: bulk ~3-5× faster. At K=100,000: bulk ~5-10× faster (the boundary-crossing cost grows linearly with K for the per-element version, while the bulk form pays it once).

The bulk form is the right default for the Python edition. If you find yourself writing a per-element swap_remove loop inside cleanup, consider whether you have a buffer of indices in hand - if you do, use the mask.

Exercise 6 - The dedup question

# anti-pattern: bad! no dedup
buffer.to_remove.append(42)                      # apply_starve appends it
buffer.to_remove.append(42)                      # apply_disease appends it too
# both systems independently noticed creature 42 should die

# cleanup without np.unique:
slots = world.id_to_slot[buffer.to_remove]      # [slot_of_42, slot_of_42] - same slot twice
keep_mask = np.ones(world.n_active, dtype=bool)
keep_mask[slots] = False                         # idempotent - same slot zeroed twice is fine

For removals via mask, dedup happens to be implicit - assigning False to the same index twice is the same as once. So the boolean-mask form is robust to duplicate to_remove entries.

The risk is for per-element swap_remove: removing slot 42 once moves the last row into 42; removing it again moves the new last row into 42, deleting an unintended row. The cleanup function above protects via np.unique regardless of which deletion form is used.

Exercise 7 - Tick-delayed visibility

@dataclass
class World:
    age_in_ticks: np.ndarray = ...
    # ...

def end_of_tick(world):
    """Increment all live ages."""
    world.age_in_ticks[: world.n_active] += 1

# Tick 5: parent reproduces; offspring goes into to_insert with age_in_ticks=0
buffer.to_insert_age_in_ticks.append(0)
cleanup(world, buffer)                            # offspring now in live columns

end_of_tick(world)                                # offspring goes 0 → 1 (counts as full tick of life)

# Tick 6: age_in_ticks of newborn is 1 at start of tick
print(world.age_in_ticks[-1])                     # 1

The offspring did not live a partial tick of tick 5. It became part of the world between tick 5 and tick 6. Tick 6 is its first full tick; end_of_tick on tick 6 makes its age_in_ticks = 2.

Whether the increment happens before or after cleanup is a policy decision. The convention here: increment after cleanup, so newborns start at 0 and reach 1 at the end of their first tick. The choice should be written down once (in the simulator’s contract) and applied consistently.

Exercise 8 - A graphics pipeline analogy (stretch)

A double-buffered renderer:

• Front buffer: the framebuffer the display reads.
• Back buffer: the framebuffer the renderer writes.
• At vsync (the frame boundary), the buffers swap. The display now reads what the renderer just wrote; the renderer starts writing what the display previously had.

Map to the simulator:

The shapes are identical. Both solve “many independent operations want to mutate shared state; how do they not step on each other?” by accumulating in a side buffer and applying atomically at a boundary. Database transactions, version-controlled file systems, audio engines (frame buffers for samples), and real-time-safety control systems (double-buffered set-points) all share this pattern.

A simulator that buffers its mutations is a simulator that has discovered transaction processing without naming it. Once you see the shape, every “atomic commit” boundary in software is a tick-boundary in disguise.

23 - Index maps

Concept node: see the DAG and glossary entry 23.

The slot-keyed tables from §17 and §19 left two questions open, and §21 added a third.

1. Point membership. “Is slot i in hungry?” costs O(K) when answered by scanning the table (np.any(hungry == i)).
2. Unsubscribe. To swap_remove slot i out of hungry you first need its position in the table - the same O(K) scan.
3. The moved slot. When swap_remove relocates a row (§21), every slot-keyed table that listed the old position now points at the wrong creature.

All three are solved by one idea: an index map - a parallel array from a key to a position, with a sentinel for “absent”. It appears twice in the simulator, the same shape pointing at two different things: id_to_slot re-finds a creature after a move, and a sparse set makes membership and unsubscribe O(1).

Instance one: id_to_slot

Maps a stable entity to its current column slot. This is what re-finds a creature after a move, and what anything outside the columns - a save, the network, the UI (§26) - uses to turn an id back into a slot. Python gives you two reasonable shapes for it, and one trap.

Two shapes that work

A numpy array, when ids are dense. If your ids are integers in [0, N_max) and most are in use, a single typed column does the job:

INVALID = np.iinfo(np.uint32).max  # 4_294_967_295
id_to_slot = np.full(N_max, INVALID, dtype=np.uint32)

def slot_of(id_to_slot: np.ndarray, creature_id: int) -> int | None:
    slot = int(id_to_slot[creature_id])
    return None if slot == INVALID else slot

The sentinel value (np.iinfo(np.uint32).max) marks “no slot - this id has no current row”. 4 MB at 1,000,000 ids; a single C-level memory read per lookup; bulk lookups via fancy indexing (id_to_slot[ids_to_remove]) run at numpy speed and are exactly what cleanup uses (§22). One cache line per 16 ids; cleanup streams through it sequentially.

A dict[int, int], when ids are sparse. If the id space is large but few are in use - id is a hash of a string, an external system’s UUID-as-int, a timestamp truncated to a slot - a Python dict is the right pick:

id_to_slot: dict[int, int] = {}

def slot_of(id_to_slot: dict[int, int], creature_id: int) -> int | None:
    return id_to_slot.get(creature_id)

Dict lookup is O(1) amortised, ~30-40 million ops/sec for integer keys (per code/measurement/float_or_int_tuple.py - note that which integer matters; int keys are 2.4× faster than float-tuple keys at the same map size). Dict pays for hash machinery on every lookup and one pointer chase per access; numpy pays neither. But dict pays only for ids that actually exist, which is the right shape for a sparse id space.

The choice is set by id density, not by taste. The simulator’s surrogate ids from §10 are dense - a fresh integer per creature, recycled when slots are reused. The numpy array is the right pick. An audit log indexed by 64-bit hash would be sparse - the dict is the right pick.

One shape that is wrong

# anti-pattern: bad!
from scipy.sparse import csr_matrix
m = csr_matrix(...)            # built for sparse 2D matrix arithmetic
slot = m[creature_id, 0]        # used here as a 1D point-lookup map

The scipy.sparse family - CSR, CSC, COO - are not index maps. They are sparse-matrix data structures, optimised for matrix-vector products and slicing entire rows or columns. Used for individual point lookups, they are very slow. From code/measurement/csr_matrix or python dict.py at 1,000 × 1,000 with 1% density, a Python dict is roughly 108× faster than CSR at random scalar lookups.

The exhibit’s headline reads “CSR matrix is 108× slower than Python dict.” That is true for the access pattern in the file - and it is the wrong reading. The right reading is: scipy gave you a sparse matrix, not a sparse map. Pick the structure that matches your access pattern. CSR is excellent at SpMV (sparse-matrix-vector-product, the common dense-vector-multiplied-by-sparse-matrix operation in scientific computing). It is poor at point-and-shoot lookups because its internal layout - three indices, indptr, data arrays - is optimised for stride-skipping, not for O(1) random access. The lesson is not “CSR is slow”; it is “wrong tool for this job, every time, by design.”

Instance two: the sparse set

The two questions the chapter opened with - “is slot i in hungry?” and “remove slot i from hungry in O(1)” - are not id_to_slot’s job. They are answered by a second index map, keyed by slot, pointing into the membership table.

The reflex is a boolean column: hungry_membership = np.zeros(N_max, dtype=bool), True where present. Resist it. That boolean is exactly the flag §17 abolished - one byte per creature whether set or not - and it answers “present?” but not “where in hungry, so I can swap_remove it?”. The structure that answers both is the sparse set: a dense array of the present slots (what the hot loop walks) and a sparse array, indexed by slot, holding each present slot’s position in dense, or INVALID.

INVALID = np.iinfo(np.uint32).max
dense  = np.empty(N_max, dtype=np.uint32)         # present slots; the hot loop walks dense[:n]
sparse = np.full(N_max, INVALID, dtype=np.uint32)  # slot -> its position in dense, or INVALID
n = 0

# is slot i present?   sparse[i] != INVALID
# subscribe(i):        sparse[i] = n; dense[n] = i; n += 1
# unsubscribe(i):      p = sparse[i]; last = dense[n - 1]
#                      dense[p] = last; sparse[last] = p   # backfill the hole, O(1)
#                      sparse[i] = INVALID; n -= 1

sparse stores positions and a sentinel, not booleans - it is the index-map pattern again, pointing into the membership table instead of into the columns. It answers “present?” and “where, so I can remove it in O(1)?”, which a boolean could not. This pair - a dense list plus a sparse index - is the sparse set, the membership structure every ECS ships.

The per-element form shown is the definition. In Python a scalar subscribe/unsubscribe in a loop pays the interpreter cost on every call; when many entries change in one tick, rebuild the dense list with a mask (the §21 bulk filter) rather than K scalar swap-removes, and maintain incrementally only when a few change per tick. Same rebuild-versus-maintain judgment as everywhere else in the book.

Maintenance

The map must be updated whenever a row moves. The events that move rows in this book are exactly three:

• Bulk filter cleanup (§22). Every removed slot’s id is set to INVALID. Every surviving id whose slot changed has its entry rewritten - exactly the rows that moved during the keep-mask compress.
• Append. When a new row lands at slot n, set id_to_slot[new_row.id] = n. The cleanup pass writes this in lockstep with the insert tail.
• Sort or reshuffle (for locality, §28). When the table is reordered, every slot moves. The full map is rewritten in lockstep with the sort. In numpy this is one assignment: id_to_slot[ids[order]] = np.arange(n_active).

The cleanup system from §22 is the natural home for these updates. Every removal and every insertion goes through cleanup; cleanup keeps the map in step.

The same pass reindexes every slot-keyed membership table. When the keep-mask compress renumbers surviving slots, build the old_to_new slot map once and remap each table’s dense array through it (dense = old_to_new[dense]), rebuilding its sparse index - in lockstep with the id_to_slot rewrite. A slot-keyed table is one more set of references into the columns; every reference the cleanup owns gets remapped together. §24 is the chapter that asks whether to avoid the move - and the reindex - altogether.

Cost

The numpy array adds one uint32 per id ever issued, including ids that are currently dead but whose slots have not been recycled. For a simulator that issues a million ids over its lifetime but has 100,000 alive at any moment, the map is 4 MB. That is a real cost - bigger than the alive table itself if the table has narrow columns. Mitigations:

• Generational ids (§10) plus a separate id allocator that recycles dead ids bound the map’s size to the high-water mark of live ids, not the total ever issued. With recycling, the map stays at 100,000 × 4 = 400 KB.
• A dict-of-int-to-int trades a constant-factor lookup overhead for tighter memory; useful when ids are sparse, as named above.

For most simulators with recycling, the dense np.ndarray is the right shape. One cache line per 16 ids; the bulk lookup id_to_slot[ids] is bandwidth-bound at numpy speed.

The pattern in the wild

Every ECS engine ships an index map. Bevy’s Entity (Rust) is a 64-bit handle whose unpacking is essentially a slot lookup with a generation check. slotmap’s SlotMap keeps an internal map. Database engines maintain index maps as B-trees over primary keys. The shape - id-to-slot lookup, maintained on every move - is universal.

Combined with §10’s stable ids and §24’s slot recycling, the index map is the third piece of the generational arena - the canonical handle-based data structure in modern systems software.

Exercises

1. Build the map. Add id_to_slot = np.full(N_max, INVALID, dtype=np.uint32) to your simulator. When a creature is appended at slot N, set id_to_slot[id] = N. When a creature’s slot changes during cleanup, update accordingly.
2. Build the sparse set. Give hungry a sparse = np.full(N_max, INVALID, dtype=np.uint32) alongside its dense list. Implement subscribe(i), unsubscribe(i), and is_member(i) - each O(1), no boolean. Confirm is_member(i) always agrees with i in set(hungry[:n].tolist()) across a run of subscribes and unsubscribes. (A boolean column would answer is_member but not give you the O(1) unsubscribe - and it is the flag §17 abolished.)
3. Maintain on bulk-filter cleanup. Modify your §22 cleanup to update id_to_slot after the keep_mask compress. The fastest form: after id[: new_n] = id[: n_active][keep_mask], run id_to_slot[id[:new_n]] = np.arange(new_n, dtype=np.uint32) - one bulk write, every surviving id’s slot rewritten in one pass. Then reindex every slot-keyed membership dense array through the same old→new slot map, so hungry still lists the right rows after the compress.
4. Time the difference. Rerun the simulator at 1M creatures, calling is_member(random_slot) 100,000 times per tick. Compare the linear scan of the dense list (§17 exercise 6) with the sparse-set lookup. The ratio is roughly N - about a million.
5. Run the exhibit (honestly). uv run "code/measurement/csr_matrix or python dict.py". Read the file’s headline (“CSR matrix is 108× slower”). Then read the chapter’s reframing. Confirm with one small experiment of your own that scipy’s CSR is fast at its job - csr.dot(some_dense_vector) for a 1000×1000 matrix - and slow at the job the file gave it.
6. The bandwidth cost. At 1M ids, id_to_slot is 4 MB. Cleanup’s bulk update on a tick with 1,000 swap_removes and 500 inserts writes ~1,500 entries - 6 KB. Compute the cleanup cost in microseconds for those writes against a 30 Hz budget.
7. Compaction compatibility. When creatures is reordered by cell (a preview of §28’s compaction), every slot moves. Rewrite id_to_slot in lockstep with one bulk numpy assignment: id_to_slot[ids[order]] = np.arange(n_active). Verify external references (held as ids) are still correct after the sort.
8. (stretch) A from-scratch generational arena. Combine §10’s gens: np.ndarray, §22’s deferred cleanup, and §23’s id_to_slot map into a SlotMap class. Provide insert(row) -> CreatureRef, remove(ref), get(ref) -> int | None. Compare the shape with slotmap::SlotMap (Rust) - same machinery, organised differently.

Reference notes in 23_index_maps_solutions.md.

What’s next

§24 - Append-only and recycling names two strategies for what happens to a slot after it has been freed. The choice is decided by access pattern, not by taste.

Solutions: 23 - Index maps

Exercise 1 - Build the map

import numpy as np
INVALID = np.iinfo(np.uint32).max

class World:
    def __init__(self, capacity: int, n_ids: int):
        self.capacity = capacity
        self.n_active = 0
        self.id           = np.zeros(capacity, dtype=np.uint32)
        # ... other columns ...
        self.id_to_slot   = np.full(n_ids, INVALID, dtype=np.uint32)

    def append(self, new_id: int, **fields):
        slot = self.n_active
        self.id[slot] = new_id
        for k, v in fields.items():
            getattr(self, k)[slot] = v
        self.id_to_slot[new_id] = slot
        self.n_active += 1

Adding the map is one extra column and one extra line in append. Removal updates happen in cleanup (next exercise).

Exercise 2 - Build the sparse set

INVALID = np.iinfo(np.uint32).max

class SparseSet:
    """dense list of present slots + a slot-indexed map to their position in dense."""
    def __init__(self, n_max: int):
        self.dense  = np.empty(n_max, dtype=np.uint32)         # present slots; walk dense[:n]
        self.sparse = np.full(n_max, INVALID, dtype=np.uint32)  # slot -> position in dense
        self.n = 0

    def is_member(self, i: int) -> bool:
        return bool(self.sparse[i] != INVALID)

    def subscribe(self, i: int) -> None:
        if self.sparse[i] != INVALID:
            return
        self.sparse[i] = self.n
        self.dense[self.n] = i
        self.n += 1

    def unsubscribe(self, i: int) -> None:
        p = self.sparse[i]
        if p == INVALID:
            return
        last = self.dense[self.n - 1]
        self.dense[p] = last          # backfill the hole
        self.sparse[last] = p
        self.sparse[i] = INVALID
        self.n -= 1

All three are O(1), no scan, no boolean. dense[:n] is the iteration list the hot loop walks; sparse answers “present?” and hands back the position needed to swap_remove in O(1). A boolean column could do is_member but not the removal - and it is the flag §17 abolished.

The cost is one uint32 per slot for sparse plus the dense backing - more bytes than a boolean column, but the boolean cannot give you O(1) unsubscribe. Pay it when the membership is stable and only a few entries change per tick (maintain incrementally). When the membership churns almost completely each tick, skip sparse entirely and rebuild dense from a mask (np.flatnonzero(predicate)) - no index needed.

Exercise 3 - Maintain on bulk-filter cleanup

def cleanup(world, buffer):
    if buffer.to_remove:
        ids = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids]
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False

        # mark the removed ids as no longer in the table
        world.id_to_slot[ids] = INVALID

        # compress every column
        n_keep = int(keep_mask.sum())
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[:n_keep] = col[: world.n_active][keep_mask]
        world.n_active = n_keep
        # rewrite id_to_slot for survivors - one bulk numpy assignment
        world.id_to_slot[world.id[:n_keep]] = np.arange(n_keep, dtype=np.uint32)
        buffer.to_remove.clear()

    # ... insertions: append new ids and write id_to_slot[new_id] = slot ...

The id_to_slot[ids[:n_keep]] = np.arange(n_keep) line is the keystone. It rewrites every surviving id’s slot in one bulk numpy assignment - exactly the same shape as the column compress, applied to the index map.

Exercise 4 - Time the difference

import time, numpy as np

world = build_world(n=1_000_000, hungry_count=100_000)
slots = np.random.default_rng(0).choice(1_000_000, size=100_000)

# Linear scan version (§17 ex 6): scan the dense list for the slot
def is_member_scan(hungry, slot):
    return bool(np.any(hungry == slot))

t = time.perf_counter()
for s in slots:
    is_member_scan(world.hungry, int(s))
print(f"linear scan × 100K: {time.perf_counter()-t:.2f} s")

# Sparse-set version: O(1) membership
t = time.perf_counter()
for s in slots:
    world.hungry_set.is_member(int(s))
print(f"sparse set × 100K: {time.perf_counter()-t:.3f} s")

Typical: linear scan ~5-10 minutes (10⁵ × 10⁵ = 10¹⁰ ops). Sparse set: ~30 ms (one C-level read per call, plus Python loop overhead). Ratio: ~10⁵-10⁶×.

For a real simulator that does many membership queries per tick, the sparse set is the difference between workable and unsalvageable. Without it, presence-replaces-flags would only be defensible for whole-table operations, not individual queries.

Exercise 5 - Run the exhibit (honestly)

uv run "code/measurement/csr_matrix or python dict.py"

Benchmarking with a 1000x1000 matrix, 1.0% density (9954 non-zero elements).
Performing 10000 random lookups.

CSR Matrix lookup time:        0.0616 s
Python Dictionary lookup time: 0.00072 s

Python Dictionary is faster for lookups by approximately 85.62 times.

The headline (“Dict is 86× faster”) is true for the access pattern in the file (random scalar lookups). The right reading is that scipy gave you a sparse matrix, not a sparse map. CSR is excellent at:

import numpy as np
from scipy.sparse import csr_matrix

mat = csr_matrix((1000, 1000))
# ... populate ...
v = np.zeros(1000)
result = mat @ v               # SpMV - what CSR is actually for

For SpMV at 1000×1000 with 1% density, CSR is dramatically faster than naive dense or dict-based approaches - nine thousand multiplications instead of a million. That’s the operation it’s optimised for.

The lesson: pick the structure that matches your access pattern. A dict is a sparse point-lookup map. CSR is a sparse matrix. They share the word “sparse” and almost nothing else.

Exercise 6 - The bandwidth cost

1M id_to_slot entries × 4 bytes = 4 MB total
1500 cleanup writes per tick × 4 bytes = 6 KB written
At ~10 GB/s memory bandwidth: ~0.6 µs to write 6 KB
30 Hz tick budget: 33 ms

The cleanup map-update cost is 0.002% of the tick budget at typical mutation rates. The id_to_slot maintenance is invisible against the rest of the work. The 4 MB total memory cost is the dominant concern at scale, not the bandwidth - which mitigates to 400 KB once recycling caps the high-water id count.

Exercise 7 - Compaction compatibility

def sort_for_locality(world, key_col_name: str):
    """Sort the table in-place by some key (e.g., spatial bucket).
       Updates id_to_slot to reflect the new positions."""
    key = getattr(world, key_col_name)[: world.n_active]
    order = np.argsort(key, kind="stable")

    for col_name in world.column_names:
        col = getattr(world, col_name)
        col[: world.n_active] = col[: world.n_active][order]

    # the keystone again - one bulk update
    world.id_to_slot[world.id[: world.n_active]] = np.arange(world.n_active,
                                                              dtype=np.uint32)

After the sort, world.id[k] is some new id, and id_to_slot[world.id[k]] == k. External code holding a reference to id 42 looks up id_to_slot[42], gets the new slot, reads the (now-relocated) row.

The sort changed every slot. The map update changed every entry of id_to_slot. Both are O(N) bulk numpy operations - fast enough to do every tick if needed.

Exercise 8 - A from-scratch generational arena (stretch)

import numpy as np
from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

INVALID = np.iinfo(np.uint32).max

class SlotMap:
    """Generational arena: stable handles, O(1) lookup, slot recycling, generation checks."""

    def __init__(self, capacity: int = 65536, n_ids: int = 1_000_000):
        self.capacity = capacity
        self.n_active = 0
        self.id    = np.zeros(capacity, dtype=np.uint32)
        self.gens  = np.zeros(capacity, dtype=np.uint32)
        self.value = np.zeros(capacity, dtype=np.float32)
        self.id_to_slot = np.full(n_ids, INVALID, dtype=np.uint32)
        self.next_id = 0

    def insert(self, value: float) -> CreatureRef:
        if self.n_active >= self.capacity:
            raise MemoryError("SlotMap full")
        slot = self.n_active
        new_id = self.next_id
        self.next_id += 1
        self.id[slot]    = new_id
        self.gens[slot]  = 0
        self.value[slot] = value
        self.id_to_slot[new_id] = slot
        self.n_active += 1
        return CreatureRef(id=new_id, gen=0)

    def remove(self, ref: CreatureRef) -> bool:
        slot = self._slot_of(ref)
        if slot is None: return False
        last = self.n_active - 1
        moved_id = int(self.id[last])
        if slot != last:
            self.id[slot]    = self.id[last]
            self.gens[slot]  = self.gens[last]
            self.value[slot] = self.value[last]
            self.id_to_slot[moved_id] = slot
        self.id_to_slot[ref.id] = INVALID
        self.gens[last] += 1                      # bump generation for next reuse
        self.n_active -= 1
        return True

    def get(self, ref: CreatureRef) -> float | None:
        slot = self._slot_of(ref)
        return None if slot is None else float(self.value[slot])

    def _slot_of(self, ref: CreatureRef) -> int | None:
        slot = int(self.id_to_slot[ref.id])
        if slot == INVALID: return None
        if int(self.gens[slot]) != ref.gen: return None
        return slot

Compare with slotmap::SlotMap (Rust): same machinery, different organisation. Rust packs (index, generation) into one Key (a u64); we use a NamedTuple. Rust uses Vec<Slot> with an internal free list; we use an active counter and bump generations on remove. The structural pieces - id allocator, generation array, id_to_slot map, swap_remove on delete - are identical.

Combined with §22’s deferred cleanup, this SlotMap is the simulator’s table primitive. Once you have it, every variable-quantity table in the book reuses the shape - creatures, food, pending events, transition log entries - each one a SlotMap with different columns.

24 - Append-only and recycling

Concept node: see the DAG and glossary entry 24.

When a row is removed from a table, its slot is freed. There are two strategies for what happens to that slot.

Append-only. Old slots stay valid forever. The table grows monotonically. New rows always go to the end.

Recycling. Freed slots are reused. The table’s length stays bounded. New rows go into freed slots before the table grows.

Each is correct; they have very different access patterns and costs.

When you have to think about slot reuse

A short Python aside before the strategies. Most Python code never thinks about slot reuse because the language hides it: del obj lets the garbage collector reclaim the memory, and the next obj = something() may or may not land in the same address - you do not know and do not care. The runtime decides.

Numpy columns are the opposite. You allocated np.empty(N_max, dtype=...) once, at startup. The slots are positional: slot 17 is the bytes at offset 17 * 4. There is no GC to reclaim them; there is just n_active and a discipline about whether slot 17, once freed, gets reused or sits empty until the table is rebuilt. The Python edition’s lifecycle phase is exactly the work the runtime usually does for you, made explicit because numpy will not.

Append-only

Use append-only when:

• History matters. The simulator’s eaten, born, dead logs from code/sim/SPEC.md are all append-only - they record what happened. Removed entries would be lost history.
• Old references must remain valid forever. Some slot-as-pointer designs assume the table never shrinks.
• Total volume is bounded by elapsed time, not by population. A 30-second 30 Hz simulation produces at most 900 frames; an append-only frame log is at most 900 rows. No need to recycle.

The cost is monotonic memory growth. A long-running simulator with append-only eaten accumulates millions of rows over hours. Mitigations:

1. Periodic snapshot + truncate (the log is replaced by a recent slice).
2. Tiered storage - recent in memory, older streamed to disk (§30).
3. Just accept the memory, if the run is short.

Recycling

Use recycling when:

• Steady-state size is small even though total inserted is large. The simulator’s creatures table at 100,000 alive with 100,000 deaths and 100,000 births per second - net flow zero, but total ever issued grows linearly. Recycling keeps memory bounded.
• Memory matters. Recycling caps the table at the high-water mark of live rows.

The cost is reference-stability complications. A new row in a recycled slot has the same slot as a previous, removed row. Code holding an old slot reference would silently dereference the new row. The fix is generational ids (§10): each slot has a generation counter that increments on every recycle. References hold (id, gen); dereference checks the generation. A stale reference fails its check.

A slot allocator looks like:

class SlotPool:
    """Allocates fixed-capacity slot indices, recycling freed ones.
       Generation increments on every free, so old (slot, gen) refs
       can detect they are stale."""

    def __init__(self, capacity: int) -> None:
        self.capacity = capacity
        self.free_slots: list[int] = []          # stack of freed slots
        self.next_slot: int = 0                  # high-water mark
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int]:
        if self.free_slots:
            slot = self.free_slots.pop()         # reuse a freed one
        else:
            slot = self.next_slot                # grow
            self.next_slot += 1
            assert self.next_slot <= self.capacity, "pool exhausted"
        return slot, int(self.gens[slot])

    def free(self, slot: int) -> None:
        self.gens[slot] += 1                     # invalidate old refs
        self.free_slots.append(slot)

allocate pops a freed slot if any are available, otherwise grows. free bumps the generation and adds the slot back to the free list. Stale references (with the old generation) cannot dereference the recycled row.

The free list is a Python list used as a LIFO stack - append and pop are both O(1). The generation column is numpy because it is touched in lockstep with cleanup (§22) and benefits from bulk numpy ops when many slots are freed together.

That generation bump on free is also the dirty marker the deferred cleanup (§22) runs on. A death does not have to move a row: it bumps the generation - marking the slot dead in O(1), so stale (slot, gen) references fail their check - and leaves the slot in place. The periodic garbage-collection compress reclaims the dead slots in one sweep and reindexes. Per tick you mark; on the GC cadence you compact. Recycling and deferred compaction are the same discipline from two sides: a freed slot is either refilled by the next birth or reclaimed by the next GC pass, and until then it sits marked and harmless - which is why a scan-all system can stride over it without special-casing (§22 measured the waste as negligible).

Choosing between them

Match the strategy to the table’s role:

Mixing strategies in one simulator is normal. The discipline is to be explicit about which table is which, and apply the right machinery to each.

Exercises

1. Two append-only logs. Implement eaten and born as append-only numpy columns with their own n_active counters. After 1,000 ticks, examine the lengths and verify they grow monotonically.
2. A recycling pool. Implement the SlotPool above. Allocate 1,000 slots, free 500, allocate 500 more. Print the slot indices the second allocate batch returns. Did the pool reuse the freed slots, or grow?
3. Stale reference detection. Allocate a slot with (slot, gen=0). Free it. Allocate a new row in the same slot - its gen is 1. Try to dereference the old (slot, 0) against the live gens column; confirm the check fails.
4. Switch creatures to append-only. Run the simulator with creatures as append-only (no recycling, every birth grows the table). Run for 10,000 ticks with steady birth and death. Plot n_active and next_slot over time - n_active is roughly flat (deaths balance births), next_slot grows monotonically. Memory cost: next_slot * row_size.
5. Switch eaten to recycling. Run with eaten recycled. After 100 ticks, all “what did this creature eat at tick 50” queries fail because the rows were reused. The history is gone. This is the failure mode that makes append-only the right pick for logs.
6. (stretch) A capacity-aware allocator. Modify SlotPool.allocate to return None when the pool is full instead of asserting. The simulator now has to handle “no slot available” as a real condition - what does it mean? (Hint: the world has hit its population cap; either rebuild bigger, drop the new entity, or delete the oldest one to make room.)

Reference notes in 24_append_only_and_recycling_solutions.md.

What’s next

§25 - One writer, many readers is the rule that makes every other discipline in the phase work: each table has exactly one writer.

Solutions: 24 - Append-only and recycling

Exercise 1 - Two append-only logs

import numpy as np

class AppendLog:
    def __init__(self, capacity: int, dtype):
        self.tick      = np.zeros(capacity, dtype=np.uint32)
        self.creature  = np.zeros(capacity, dtype=np.uint32)
        self.value     = np.zeros(capacity, dtype=dtype)
        self.n_active  = 0
        self.capacity  = capacity

    def append(self, tick: int, creature_id: int, value):
        if self.n_active >= self.capacity:
            raise MemoryError("log full - snapshot and truncate")
        self.tick[self.n_active]     = tick
        self.creature[self.n_active] = creature_id
        self.value[self.n_active]    = value
        self.n_active += 1

eaten = AppendLog(capacity=1_000_000, dtype=np.float32)
born  = AppendLog(capacity=1_000_000, dtype=np.uint32)

# After 1000 ticks of the simulator
print(f"eaten: {eaten.n_active} entries (monotonic - never shrinks)")
print(f"born:  {born.n_active} entries (monotonic)")

Both n_active counters only ever increment. Once entries are written, they stay. Capacity is the high-water-mark of total events ever recorded, not of current population.

Exercise 2 - A recycling pool

class SlotPool:
    def __init__(self, capacity: int):
        self.capacity = capacity
        self.free_slots: list[int] = []
        self.next_slot: int = 0
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int]:
        if self.free_slots:
            slot = self.free_slots.pop()
        else:
            slot = self.next_slot
            self.next_slot += 1
        return slot, int(self.gens[slot])

    def free(self, slot: int):
        self.gens[slot] += 1
        self.free_slots.append(slot)


pool = SlotPool(capacity=10_000)
first_batch = [pool.allocate()[0] for _ in range(1_000)]
print(f"first 1000 slots: {first_batch[:5]}...{first_batch[-3:]}")

# Free 500
for slot in first_batch[:500]:
    pool.free(slot)

# Allocate 500 more
second_batch = [pool.allocate()[0] for _ in range(500)]
print(f"second 500 slots: {second_batch[:5]}...{second_batch[-3:]}")
print(f"all reused?      {set(second_batch).issubset(set(first_batch[:500]))}")
print(f"next_slot now:   {pool.next_slot}")  # still 1000 - no growth

first 1000 slots: [0, 1, 2, 3, 4]...[997, 998, 999]
second 500 slots: [499, 498, 497]...[2, 1, 0]
all reused?      True
next_slot now:   1000

The second batch reuses the freed slots in LIFO order (the most recently freed slot is allocated next). Total next_slot stays at 1000 - the pool did not grow.

Exercise 3 - Stale reference detection

pool = SlotPool(capacity=100)
slot, gen = pool.allocate()                           # slot=0, gen=0
old_ref = (slot, gen)                                  # save
pool.free(slot)                                        # gens[0] = 1
new_slot, new_gen = pool.allocate()                    # reuses slot 0, gen=1
new_ref = (new_slot, new_gen)

def deref(pool, ref):
    slot, gen = ref
    return None if int(pool.gens[slot]) != gen else slot

print(f"new ref deref:  {deref(pool, new_ref)}")      # 0
print(f"old ref deref:  {deref(pool, old_ref)}")      # None - stale!

The old reference’s generation is stale. Even though the slot is alive again, the generation check correctly identifies that the holder of old_ref is looking at a different row than they expect. The reference is rejected; the holder must re-fetch.

Exercise 4 - Switch creatures to append-only

# After 10,000 ticks, steady-state birth/death:
# n_active ≈ 100,000 (live population, oscillates around equilibrium)
# next_slot = total ever issued = (births_per_tick × 10,000)
#           ≈ 100 × 10,000 = 1,000,000 (10× the live population)

# Memory cost: next_slot × row_size = 1M × ~32 bytes = 32 MB
# Live data:    n_active × row_size = 100K × 32 bytes = 3.2 MB
# Wasted:       28.8 MB sitting in dead slots

The append-only creatures table has 90% of its memory occupied by tombstones - slots whose previous occupants are dead. Reading n_active is correct, but the table’s allocated bytes grow with elapsed time. For a 1-hour simulation, the wasted memory might be 100× the live data.

For history tables this is fine (the tombstones are the history). For the live population it’s a memory leak waiting to be named. Recycling is the structural fix.

Exercise 5 - Switch eaten to recycling

# eaten is now a SlotPool-managed table with capacity 10_000
# After 100 ticks at 50 eats/tick, 5000 events recorded into 10_000 slots
# After 200 ticks: free list starts being used; old eat events are overwritten
# After 300 ticks: ~10,000 events have been recycled into existing slots

# Query: "what did creature 42 eat at tick 50?"
# Search eaten.tick[:n] == 50 → finds it (tick 100)
# Search after tick 250: finds nothing (the row was recycled at ~tick 250)

The history is gone once a slot is recycled. There is no record that creature 42 ate at tick 50 - the slot now holds tick 273’s eat event for creature 91. Recycling for a history table is category error.

This is exactly the failure mode that makes append-only correct for logs. Logs grow forever; you handle that with snapshot-and-truncate or tiered storage, not by recycling slots.

Exercise 6 - A capacity-aware allocator (stretch)

class CapacityAwarePool:
    def __init__(self, capacity: int):
        self.capacity = capacity
        self.free_slots: list[int] = []
        self.next_slot: int = 0
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int] | None:
        if self.free_slots:
            slot = self.free_slots.pop()
        elif self.next_slot < self.capacity:
            slot = self.next_slot
            self.next_slot += 1
        else:
            return None                              # full!
        return slot, int(self.gens[slot])

Returning None from allocate is the simulator’s signal that the world has hit its population cap. Three reasonable policies:

1. Drop the new entity. “Sorry, no room.” A reproduction event silently fails. Simplest, hides the resource limit, may distort the simulation’s behaviour.
2. Delete the oldest one. A LRU-style eviction. The pool needs an oldest-tracking scheme (a tick column, a queue). The simulation continues at capacity but loses identifiers; not appropriate for logs but reasonable for active scenes.
3. Resize the pool. Allocate a larger backing buffer, copy, retry. Most flexible, but introduces a slow-path that may blow a tick budget; consider doubling the pool every time it fills, like Vec::push’s amortised growth.

The book’s simulator picks whichever fits the table:

• creatures: option 3 with periodic doubling. The scenario should never hit the cap in practice, but if it does, the simulation continues.
• pending_event: option 1. Events that don’t fit the cap are dropped; the simulation makes do with what it has.
• eaten/born (append-only): option 1, but with snapshot+truncate as the recovery, not silent drop.

The choice is per-table. Document it next to the table’s allocation.

25 - One writer, many readers

Concept node: see the DAG and glossary entry 25.

Every table has exactly one writer, and any number of readers.

The rule is small. Its consequences are everything.

Why it works. A row is a tuple (§6) - its fields are aligned by index. A table’s columns must be modified together to maintain alignment. A single writer guarantees this: only one place in the code mutates the table, so only one place can violate alignment, so testing one place is enough.

A table with two writers has two places where alignment can be violated. If they run concurrently, alignment is violated nondeterministically. If they run sequentially, the order matters and must be specified. Either way, the cost of getting it right grows superlinearly with the number of writers.

The Python-specific problem: nothing enforces it

Rust has a borrow checker. &mut [T] is the type-level expression of single-writer ownership; only one mutable reference can exist at a time, and the compiler rejects code that violates it. Python has no equivalent. There is no &mut, no exclusive-access type, no compile-time check. Anyone who has a reference to a numpy array can mutate it. The single-writer rule is a discipline you enforce by convention, not a constraint the language enforces for you.

This makes the rule more important in Python, not less. Without compile-time enforcement, the violations show up at runtime as the bugs the rule was supposed to prevent: intermittent, silent, late-binding. The discipline is what stands between the architecture and the bug.

The numpy view trap

The hardest version of the violation in Python is the numpy view. A slice of a numpy array is not a copy - it is a view into the same underlying bytes. Writing through the view mutates the parent:

# anti-pattern: bad!
arr = np.zeros(10)
view = arr[2:5]    # looks like a new array; is actually a view
view[0] = 42       # also writes arr[2] = 42

A function receiving view has no way to know from the variable’s name or its np.ndarray type that it shares memory with someone else’s table. There is no compile-time signal. Mutating view looks local; the side effect on arr is invisible until something else reads it. This is the single-writer rule violated at the byte level, hidden behind a slice that looks like a fresh allocation.

Three mitigations:

# explicit copy when handing data to a function that may mutate it
foreign_function(arr[2:5].copy())

# read-only flag on the parent (writes via any view raise ValueError)
arr.flags.writeable = False

# document the ownership in the function signature and let it live in the contract
def motion(pos_x: np.ndarray, vel_x: np.ndarray, dt: float) -> None:
    """Read-set: vel_x, dt.   Write-set: pos_x.
       pos_x and vel_x must not alias each other or any other column."""

The first two are runtime mechanisms. The third is the convention this book lives on. A function’s docstring declares the read-set and write-set (§13); the caller is responsible for not handing aliasing arrays into a function that assumes none. If the caller cannot guarantee non-aliasing, they pass a copy.

The disciplines that depend on it

All of these need single-writer ownership to work:

• §31 - Disjoint write-sets parallelize freely. Two systems with disjoint write-sets can run on different processes. The rule guarantees no shared mutation.
• §22 - Mutations buffer. A side-table writer (cleanup) is the only writer of creatures. All other systems push to to_remove and to_insert, which they own.
• §43 - Tests are systems. A test system reads everything and writes nothing. The ownership rule is what guarantees its reads see consistent state.
• The InspectionSystem pattern. A debug inspector holds read-only references to every table. Read-only access composes with single-writer ownership to make races structurally impossible.

What the rule looks like in practice

def motion(pos_x: np.ndarray, pos_y: np.ndarray,
           vel_x: np.ndarray, vel_y: np.ndarray, dt: float) -> None:
    """Read-set: vel_x, vel_y, dt.   Write-set: pos_x, pos_y."""
    pos_x += vel_x * dt
    pos_y += vel_y * dt

def next_event(pos_x: np.ndarray, food_x: np.ndarray,
               pending: np.ndarray) -> None:
    """Read-set: pos_x, food_x.   Write-set: pending."""
    ...

def apply_eat(pending: np.ndarray, food: np.ndarray,
              to_remove: list[int], energy: np.ndarray) -> None:
    """Read-set: pending, food.   Write-set: to_remove (append), energy."""
    ...

For each table, exactly one writer is allowed:

• pos_x, pos_y: written only by motion.
• pending: written only by next_event.
• to_remove, to_insert: written by many systems, but each system appends only its own queued mutations; no one reads them until cleanup.
• creatures, food: written only by cleanup, which materialises every other system’s queued changes.

Multiple systems may contribute to a table by appending to its side buffer; the actual single writer of the live table is cleanup. The architecture preserves the rule even as many systems propose mutations.

Bugs that arise from violations

Two systems writing the same column produce inconsistent state. The bug is usually intermittent (depends on schedule), silent (no error reported, just bad data), and late-binding (manifests far from the cause). They are among the hardest bugs in any concurrent system. The single-writer rule eliminates them by construction. In Python, where the language will not catch the violation, the rule is the only thing standing between you and the bug.

The rule applies recursively. A view table whose entries are derived from another table inherits the ownership rule: a hungry: np.ndarray is owned by the system that classifies hunger; no other system writes to it.

This is the rule that closes Memory & lifecycle. Without it, the buffering, swap_remove, index maps, and slot recycling are all unsafe in any concurrent or parallel context. With it, everything composes.

Exercises

1. Identify the writers. For each table in your simulator (creatures, food, food_spawner, pending_event, eaten, born, dead, hungry, to_remove, to_insert), name the one system that writes it. If you find a table with two writers, the rule is violated - investigate.
2. The view trap, in your fingers. Build arr = np.arange(10). Take view = arr[2:5]. Set view[0] = 999. Print arr. Confirm arr[2] == 999. Now take cpy = arr[2:5].copy(), set cpy[0] = 0, print arr - confirm arr is unchanged. The slice was a view; the .copy() was not.
3. The read-only-flag mitigation. Build arr = np.arange(10). Set arr.flags.writeable = False. Try to assign arr[3] = 42. Catch the ValueError. Now derive view = arr[2:5] from the read-only parent - note that view.flags.writeable is also False. Read-only-ness propagates.
4. A constructed violation. Write two functions that both mutate energy. Call them in sequence on the same array; the result is whatever the second one wrote. Now run them in two multiprocessing.Process workers sharing the array via multiprocessing.shared_memory; observe that no error is raised and the bug is silent. This is the failure mode the single-writer rule prevents - Python will not warn you.
5. Refactor with a buffer. Take one of the violations from exercise 4 and add a side buffer that one function writes and the other reads. The two functions are now writer-disjoint, even though they touch the same logical concept.
6. Build an InspectionSystem. Write a function that takes a World (a dataclass holding all the tables), reads every column, and returns a snapshot dictionary. Mark every input array read-only via arr.flags.writeable = False for the duration of the call. The system is read-only by construction and cannot violate the rule.
7. (stretch) The cleanup system as canonical writer. In your simulator, audit: every mutation of creatures, food, etc. flows through cleanup. Every other system writes only to to_remove, to_insert, or its own outputs. Verify the audit holds for the simulator end-to-end. Note this is harder in Python than in Rust because nothing checks it for you - write a unit test that asserts no system other than cleanup mutates the live tables.

Reference notes in 25_ownership_of_tables_solutions.md.

What’s next

You have closed Memory & lifecycle. The simulator’s machinery is now complete: it can grow, shrink, recycle, parallelise, and replay. The next phase is Scale, starting with §26 - Subscription tables. The simulator’s per-tick cost goes under the microscope.

Solutions: 25 - One writer, many readers

Exercise 1 - Identify the writers

Audit any simulator project for tables with two writers - that’s the rule violated. Common sources of the violation:

• An “update” function that also validates and corrects.
• A logging side-effect that mutates state.
• Two systems both setting a derived flag.

The fix is always one of: split into two systems with an intermediate buffer, or designate one as the writer and have the other request changes via a side buffer.

Exercise 2 - The view trap, in your fingers

import numpy as np

arr = np.arange(10)
view = arr[2:5]                      # view, NOT a copy
view[0] = 999
print(arr)
# [  0   1 999   3   4   5   6   7   8   9]   - arr was mutated through the view!

cpy = arr[2:5].copy()                # explicit copy
cpy[0] = 0
print(arr)
# [  0   1 999   3   4   5   6   7   8   9]   - arr unchanged

A slice of a numpy array is a view into the same backing buffer. view[0] = 999 writes to byte offset 16 of arr (since int64 × index 2). The .copy() allocates a new buffer; mutations there are isolated.

This is the classic memory-aliasing trap. The variable name (view vs cpy) gives no signal. The dtype gives no signal. The only ways to know: check arr.base (view.base is arr is True; cpy.base is None) or pass through the convention *.copy() whenever ownership transfers.

Exercise 3 - The read-only-flag mitigation

arr = np.arange(10)
arr.flags.writeable = False

try:
    arr[3] = 42
except ValueError as e:
    print(f"caught: {e}")
# caught: assignment destination is read-only

view = arr[2:5]
print(view.flags.writeable)          # False - read-only-ness propagates to views

Setting writeable = False is a runtime guard. Anyone with a reference to the array - including any view derived from it - can read but not write. This is the closest Python has to Rust’s &[T] (immutable borrow). It does not guarantee correctness across function calls (a careless caller can still set writeable = True back), but it catches accidental writes loudly.

For library functions that accept arrays from outside, locking the input via writeable = False for the function body is a defensive practice. The cost is one attribute set; the protection is real.

Exercise 4 - A constructed violation

import numpy as np

def system_a(energy):
    energy[:] += 1.0                # writer 1

def system_b(energy):
    energy[:] -= 0.5                # writer 2 - same column!

# Sequentially: result depends on order
energy = np.zeros(10)
system_a(energy); system_b(energy)
print(energy)        # [0.5, 0.5, ...] - A first, then B

energy = np.zeros(10)
system_b(energy); system_a(energy)
print(energy)        # [0.5, 0.5, ...] - same end state because additions commute
                      # but the per-step state would differ

Sequentially: order matters and must be specified. With multiprocessing/shared_memory:

# anti-pattern: bad! two writers, no synchronisation
from multiprocessing import Process
from multiprocessing.shared_memory import SharedMemory
import numpy as np

shm = SharedMemory(create=True, size=80)
energy = np.ndarray((10,), dtype=np.float64, buffer=shm.buf)
energy[:] = 0

def worker_a(shm_name):
    s = SharedMemory(shm_name)
    e = np.ndarray((10,), dtype=np.float64, buffer=s.buf)
    for _ in range(1_000_000): e[:] += 0.0001

def worker_b(shm_name):
    s = SharedMemory(shm_name)
    e = np.ndarray((10,), dtype=np.float64, buffer=s.buf)
    for _ in range(1_000_000): e[:] -= 0.0001

# Run them simultaneously
p1, p2 = Process(target=worker_a, args=(shm.name,)), Process(target=worker_b, args=(shm.name,))
p1.start(); p2.start(); p1.join(); p2.join()
print(energy)   # not [0, 0, ...] - race conditions ate some updates

Each += involves a read, an add, and a write. Two processes interleaving these without coordination produce lost updates: process A reads x; process B reads x; A writes x+1; B writes x-1; the result is x-1 (or x+1) instead of x. No ValueError, no warning. Just silently wrong arithmetic.

The single-writer rule is the structural prevention. Two writers to the same column means coordination is required, and Python provides no enforcement. The rule eliminates the need for coordination at the architectural level.

Exercise 5 - Refactor with a buffer

def system_a(energy, energy_delta):
    energy_delta[:] += 1.0           # writer of energy_delta only

def system_b(energy, energy_delta):
    energy_delta[:] -= 0.5           # writer of energy_delta only

def cleanup(energy, energy_delta):
    energy[:] += energy_delta        # the SOLE writer of energy
    energy_delta[:] = 0

Now system_a and system_b are writer-disjoint with respect to energy; both write to energy_delta (which is also a violation, but a contained one - energy_delta is a side buffer, not load-bearing world state).

The architectural fix is one more level of buffering: each system writes to its own delta column.

def system_a(energy_delta_a):  energy_delta_a[:] += 1.0
def system_b(energy_delta_b):  energy_delta_b[:] -= 0.5
def cleanup(energy, energy_delta_a, energy_delta_b):
    energy += energy_delta_a + energy_delta_b
    energy_delta_a[:] = 0; energy_delta_b[:] = 0

This is the canonical pattern for parallel mutation: each writer has its own column; the merge happens in cleanup, single-threaded, on disjoint inputs. §31 - Disjoint write-sets parallelize freely develops it further.

Exercise 6 - Build an InspectionSystem

from contextlib import contextmanager

@contextmanager
def read_only_world(world):
    """Locks every column read-only for the duration of the inspection."""
    columns = (world.pos_x, world.pos_y, world.vel_x, world.vel_y, world.energy, world.id)
    for c in columns:
        c.flags.writeable = False
    try:
        yield
    finally:
        for c in columns:
            c.flags.writeable = True

def inspect(world) -> dict:
    """A read-only system; returns a snapshot."""
    with read_only_world(world):
        return {
            "n_active": world.n_active,
            "energy_min": float(world.energy[: world.n_active].min()),
            "energy_max": float(world.energy[: world.n_active].max()),
            "centre_of_mass": (float(world.pos_x[: world.n_active].mean()),
                               float(world.pos_y[: world.n_active].mean())),
        }

The system reads everything, writes nothing, locks the world for the duration. Any accidental write inside the inspection raises ValueError immediately. The lock is dropped on exit, so subsequent (non-inspection) systems can mutate normally.

This is the §43 test as system shape. A test that “verifies the world is consistent” runs in the same shape: lock, read, assert, unlock.

Exercise 7 - The cleanup system as canonical writer (stretch)

def write_audit(world, system_func):
    """Record which columns each system wrote during one tick."""
    snapshot_before = {name: getattr(world, name).tobytes() for name in world.column_names}
    system_func(world)
    written = []
    for name, before in snapshot_before.items():
        after = getattr(world, name).tobytes()
        if after != before:
            written.append(name)
    return written

# After running each system, assert which ones it should have written
expected = {
    "motion":         {"pos_x", "pos_y"},
    "next_event":     {"pending_event"},
    "apply_eat":      {"to_remove", "energy_delta"},     # buffers, not live tables
    "apply_starve":   {"to_remove"},
    "cleanup":        {"pos_x", "pos_y", "vel_x", "vel_y", "energy", "id", "n_active",
                       "id_to_slot", "to_remove", "to_insert_pos_x", ...},  # cleanup writes everything
}

for name, func in systems:
    written = write_audit(world, func)
    assert set(written) <= expected[name], f"{name} wrote {written} - unexpected: {set(written) - expected[name]}"

The audit is itself a system. It runs once per tick (or in a CI-only build) and asserts the structural property: every system writes only what it claims to write. A drift between expected and actual is the signal that someone added a side-effect - exactly the violation the single-writer rule forbids.

In Python this is the closest you get to a borrow checker. It runs at runtime, with O(N) overhead per tick (the byte snapshots), and it catches violations at the smallest mutation. Disable it in production; keep it on in CI.

26 - Subscription tables, keyed by slot

Concept node: see the DAG and glossary entry 26.

A system rarely touches every entity. Motion moves all of them, but starvation only reads the hungry, reproduction only the well-fed, a sleep timer only the sleeping. §17 gave the tool for “which entities are in this set”: a membership table. §19 measured the payoff: walk the 100,000 hungry instead of scanning 1,000,000 and masking, and the work is proportional to the subset, not the population.

Call that membership table a subscription table, and the loop that walks it a system’s hot loop. A creature subscribes to hungry when its energy drops; it unsubscribes when it eats. The subscription is the system’s input; the hot loop is the system. This section settles a question §17 left open: what does a subscription table store, and how fast is the hot loop that reads it?

A wrong turn first: splitting fields

The instinct many readers arrive with - and the one this chapter used to teach - is to split the fields of a creature into hot and cold: put the columns an inner loop touches in one group, the rarely-read ones in another, so a load does not drag in bytes the loop ignores. In a row-oriented (array-of-objects) world this is a real technique: reading c.pos_x pulls the whole Creature object into cache, c.birth_t and c.id and all.

But this book has been structure-of-arrays since §7, and in numpy SoA the split is already done for you, more completely than a hot/cold grouping would manage. Every column is its own np.ndarray, its own allocation. Reading pos_x touches pos_x’s memory and nothing else; birth_t is a different array entirely. There is no row to drag a cold field along with, so there is nothing to split. The bandwidth win that motivates a field split in array-of-objects is the same win SoA already banked back in §7; it is not a separate technique to apply here.

So the attribute columns are never split. They stay whole: every column, every slot, reachable by index i. What a system changes is not the columns but how it reaches into them. Rather than scan the whole column and mask, a system keeps a subscription table - the slots i it cares about - and indexes straight in: energy[hungry], no scan, no field split. The rest of this section is about making that direct access fast.

What a subscription stores: slots or ids

A creature has a stable id and a current slot, its position in the columns. id_to_slot maps one to the other. A subscription table could hold either, and in numpy the choice is a measurable cost, not a style preference.

• Hold slots. The hot loop is one fancy-index gather: energy[hungry]. No redirection. But when cleanup moves entities, every slot in every subscription is stale and must be rewritten.
• Hold ids. The hot loop is two gathers: energy[id_to_slot[hungry]] - resolve each id to its slot, then gather the columns. The table survives relocation untouched; only id_to_slot changes.

The redirection is paid every tick. The rewrite is paid once per cleanup interval. Which loses?

Measured. At 1,000,000 creatures with a tenth subscribed, the id-keyed hot loop runs about twice as slow as the slot-keyed one on a modern desktop¹. The extra cost is the inner gather: id_to_slot is a four-megabyte array and the subscribed ids are scattered through it, so id_to_slot[hungry] is a hundred-thousand-element random read before the column gather has even started. The slot key skips that gather entirely - energy[hungry] indexes the columns once. (In a compiled language the same gap is a scattered cache miss per element; numpy turns it into a second fancy-index pass, but the verdict is the same.)

The rewrite the slot key pays in return is small and bounded: when cleanup compacts, remap each subscription’s dense array through the old→new slot map (§23), once per cleanup interval, not once per tick. Across the realistic range - a handful of subscriptions, cleanup every few dozen ticks - the per-tick saving buries the per-interval rewrite. The benchmark is ebp_partition.py; numbers below.

So subscription tables hold slots. This is also why the lifecycle keeps stable slots and lets cleanup own the reindex: slot keys are only safe when one system is responsible for rewriting them when entities move. The cleanup can do that for any reference it owns - a subscription, or a cross-entity link stored in a column - remapping them all in one pass.

So what is the id for?

For every reference cleanup cannot reach to rewrite. A save file (§36), a replay log (§37) whose events are (tick, id, name) - exactly the §18 boundary - a packet on the wire, an entity the UI has selected, a snapshot a slow background system is still reading (§39): a slot is meaningless to all of them, because the next compaction moves it. They hold the id and resolve it through id_to_slot once, at the boundary (§35), never per element. Slots are an internal, momentary fact; the id (with its generation, §10) is the identity that survives a relocation, a save, and a network hop.

Locality: a slot-keyed gather is fast only when its slots are dense

A slot-keyed hot loop gathers columns at the slots the subscription lists. If those slots are scattered through the column - which is what churn produces as deaths and births leave holes - energy[hungry] is a random-access gather. If they are contiguous, it streams. Compacting the live, subscribed entities to the front of the columns turns the scattered gather into a sequential one.

How much that buys depends on the machine, and the spread is wide. In numpy the gather’s per-element machinery - bounds-check, dereference, copy into the output - is large, and on a fast desktop it dominates the cache-miss difference, so compaction buys only about 1.3x on the hot loop². On hardware where memory latency dominates that machinery instead - the Pi 4 - the win is several-fold (4.9x; see Measurements). The reorder pass is about a millisecond at a million rows on the desktop, so on hot-loop locality alone it pays back in tens of ticks, on the order of a second at 30 Hz³; on the memory-bound machines the per-tick saving is larger and the payback shorter.

That payback sounds slow until you notice you were going to pay it anyway. The same pass reclaims dead slots (§24), which the simulator must do regardless as births and deaths leave holes, and the locality gain rides along on a move you already owed. So the rule for Python is: compact on the GC cadence; the locality it also delivers ranges from a modest desktop bonus to a real win on memory-bound hardware. §28 - Proximity is a property of position is that pass.

The one case a split would help, in full view

There is a single scenario where grouping columns would still pay. A hot loop that gathers several columns at scattered slots issues one fancy-index per column, each its own random walk; interleaving those columns into one structured array would gather once. That case is real, and worth stating plainly rather than hiding behind the principle.

We keep the columns separate anyway. The book’s answer to scatter is to remove it: compaction (§28) makes the subscribed slots dense, a dense gather streams each column, and the per-column cost the structured array would have saved is gone. A structured array would also forfeit what SoA bought in §7 - whole-column vectorised ops, and the per-column single-writer parallelism of §31 - on every loop that is not scattered, to win the one that is. So the rule stands with its exception in the open: keep the columns separate, and compact when the gather scatters.

Name the subscription before you build it

A subscription is earned by a system that genuinely processes a subset. “Most creatures are not hungry on most ticks, so hungry is far smaller than the population” is a sound reason to build one. “Every creature is always in alive, but other engines keep an alive-set” is not. A subscription that holds the whole population is a scan-all with extra bookkeeping, and the measurement says so: at full participation the gather is slower than a plain vectorised pass over the column - it is the same crossover §19 measured, where presence loses to the bool mask once nearly everyone is a member. The subscription wins in proportion to how much it excludes, and not otherwise.

The payoff is not only speed; it is extensibility

An entity’s character is just the set of subscriptions that hold it - a grazer is “the herd-motion table plus the graze-forage edge,” no more - so a new kind of entity is a new subscription, not a new class threaded through the code. The reference simulator adds a predator exactly this way: name a predators subscription in the registry, seed it, and wire two systems (herd-motion and a forage edge onto grazers). The join that maintains subscriptions never learns a predator exists, because it loops the registry and maintains every entry the same way. Diff sim2b.py against sim1b.py and a whole trophic level costs a handful of lines, with nothing existing edited. Composition, not surgery - the extensibility the architecture keeps promising, made literal and measurable in a diff.

Measurements

The prose quotes the modern-desktop figure; the spread across the reference machines is below. The keying verdict (row 1, slot beats id) holds on every machine. The locality win (row 2) varies widely - modest on the desktop, several-fold on the Pi - because it depends on how much memory latency dominates numpy’s fixed gather overhead. Reproduce any column by running ebp_partition.py on that machine.

Exercises

These extend the simulator’s creature columns and the id_to_slot map from §23.

1. Build a slot-keyed subscription. Add hungry = np.empty(0, dtype=np.uint32) holding the slots of hungry creatures. Build it each tick with np.flatnonzero(energy < HUNGER_THRESHOLD). Write the hot loop: energy[hungry] -= burn * dt. Verify it touches only the subscribed creatures.
2. Key it by id instead, and time both. Build a second version where hungry holds entity ids and the hot loop resolves each through id_to_slot[hungry] before the gather. At 1M creatures with 10% subscribed, time both hot loops with timeit. Reproduce the ~2x gap. Where does the id version’s time go? Compare id_to_slot’s size with one cache line.
3. Unsubscribe in O(1). When a creature stops being hungry, remove its slot from hungry. What do you need alongside hungry to find the slot’s position in the table without scanning it? (It is the §23 sparse set, one level up.)
4. Reindex on compaction. Relocate the live creatures to the front of the columns (a stand-in for the §24/§28 cleanup), producing an old_to_new slot map. Rewrite the slot-keyed hungry with hungry = old_to_new[hungry]; confirm the hot loop still processes the same creatures. Now do the same for the id-keyed version: what has to change? Time both reindex passes.
5. Dense vs scattered. Time the slot-keyed hot loop with the subscription’s slots scattered through the column, then again after sorting them to the front. Reproduce the ~1.3x speedup. How many ticks of hot-loop saving pay back one compaction pass? (Answer near 30 - and note this is not the reason to compact; reclaiming dead slots is.)
6. The subscription that holds everyone. Subscribe every creature and time the hot loop against a plain energy -= burn * dt * mask scan. The subscription should be no faster, and at full participation slower. Explain why, and state the rule for when a subscription is worth building.
7. (stretch) Two subscriptions, one entity. Put creatures in both hungry and sleepy. On compaction, both dense arrays need remapping through old_to_new. Measure how the reindex cost grows with the number of subscriptions an entity sits in, and argue why it stays cheaper than the id key’s per-tick redirection for any realistic cleanup interval.

Reference notes in 26_subscription_tables_solutions.md.

What’s next

§27 - Working set vs cache puts numbers on the question this section kept leaning on: how big is the hot loop’s footprint, and what cache level does it fit in?

Solutions: 26 - Subscription tables, keyed by slot

Exercise 1 - Build a slot-keyed subscription

import numpy as np
HUNGER_THRESHOLD = 10.0

def classify_hungry(energy: np.ndarray) -> np.ndarray:
    return np.flatnonzero(energy < HUNGER_THRESHOLD)   # the slots that pass

def drive_hunger(hungry: np.ndarray, energy: np.ndarray, burn: float, dt: float) -> None:
    energy[hungry] -= burn * dt                        # one gather; slots index the columns directly

The subscription holds slots, so the hot loop indexes the columns with no redirection. Only the K subscribed rows are touched - the work is proportional to the subset, exactly as §19 promised.

Exercise 2 - Key it by id instead, and time both

import numpy as np, timeit
N = 1_000_000
rng = np.random.default_rng(0)
energy = rng.random(N).astype(np.float32)
slots  = rng.choice(N, size=N // 10, replace=False).astype(np.uint32)   # 10%, scattered

# id-keyed: hungry holds ids; id_to_slot resolves id -> slot
ids = rng.permutation(N)[:N // 10].astype(np.uint32)
id_to_slot = np.empty(N, dtype=np.uint32)
id_to_slot[ids] = slots

slot_keyed = lambda: energy[slots]                # one fancy index
id_keyed   = lambda: energy[id_to_slot[ids]]      # two fancy indexes

t_slot = timeit.timeit(slot_keyed, number=200) / 200
t_id   = timeit.timeit(id_keyed,   number=200) / 200
print(f"slot {t_slot*1e6:.0f} us   id {t_id*1e6:.0f} us   id/slot {t_id/t_slot:.2f}x")

slot 149 us   id 310 us   id/slot 2.08x      (Ryzen 9 270)

The id version’s time goes into id_to_slot[ids] - a 100,000-element random read of a 4 MB array, a complete gather before the column gather starts. id_to_slot is 4 MB; one cache line is 64 bytes, so the subscribed ids are scattered across tens of thousands of cache lines. The slot key never touches it. Four-machine spread: ebp_partition.py claim C1, and the §26 Measurements table.

Exercise 3 - Unsubscribe in O(1)

You need the §23 sparse set alongside hungry’s dense list: a sparse array, indexed by slot, holding each present slot’s position in dense.

def unsubscribe(dense, sparse, n, i):          # remove slot i in O(1), no scan
    p = sparse[i]
    last = dense[n - 1]
    dense[p] = last                            # backfill the hole
    sparse[last] = p
    sparse[i] = INVALID
    return n - 1

Without sparse you would scan hungry to find slot i’s position - O(K). The sparse set makes both “is i subscribed?” and “remove i” O(1), and it is the same index-map shape as id_to_slot, one level up: pointing into the subscription table instead of into the columns.

Exercise 4 - Reindex on compaction

# compaction relocates the live creatures; build the old -> new slot map once
order = np.argsort(~alive_mask, kind="stable")[:n_live]   # live slots, front to back
old_to_new = np.full(N, INVALID, dtype=np.uint32)
old_to_new[order] = np.arange(n_live, dtype=np.uint32)

# slot-keyed subscription: one bulk remap
hungry = old_to_new[hungry]

# id-keyed subscription: hungry (holding ids) is untouched; only id_to_slot is rewritten
id_to_slot[ids_in_new_order] = np.arange(n_live, dtype=np.uint32)

The slot-keyed reindex is K writes (one per subscribed slot) per cleanup. The id-keyed table needs no reindex - that is the id key’s one advantage, and it buys it by paying the per-tick redirection from exercise 2. Across realistic cleanup intervals the slot key wins the trade (exercise 7). Both reindex passes are bulk numpy assignments; time them and you will find both are a small fraction of one tick, so the contest is decided by the per-tick hot loop, not the per-interval reindex.

Exercise 5 - Dense vs scattered

scattered = rng.choice(N, size=N // 10, replace=False).astype(np.uint32)
compacted = np.arange(N // 10, dtype=np.uint32)          # subscribed slots at the front

t_sc = timeit.timeit(lambda: energy[scattered], number=200) / 200
t_co = timeit.timeit(lambda: energy[compacted], number=200) / 200
print(f"scattered {t_sc*1e6:.0f} us   compacted {t_co*1e6:.0f} us   ratio {t_sc/t_co:.2f}x")

scattered 149 us   compacted 114 us   ratio 1.31x      (Ryzen 9 270)

Compaction buys ~1.3x on the Ryzen 9 270 - the gather’s per-element machinery dominates and masks most of the cache-miss difference (the same reason numpy mutes the cache cliffs in §27 and false sharing in §33). The reorder pass is ~1 ms at a million rows, so payback on locality alone is ~30 ticks, about a second at 30 Hz. (On memory-bound hardware the cache-miss difference dominates the gather overhead instead: the Pi 4 sees ~4.9x with payback ~7 ticks - see the §26 Measurements table.) That is acceptable - but on the desktop it is not the reason to compact. The same pass reclaims dead slots (§24), which the simulator owes regardless; the locality rides along free.

Exercise 6 - The subscription that holds everyone

all_slots = np.arange(N, dtype=np.uint32)
mask = np.ones(N, dtype=bool)

t_sub  = timeit.timeit(lambda: energy[all_slots], number=200) / 200   # gather all N
t_scan = timeit.timeit(lambda: energy * mask,     number=200) / 200   # branchless scan
print(f"gather-all {t_sub*1e6:.0f} us   scan {t_scan*1e6:.0f} us")

At 100% participation the gather is several times slower than the branchless scan: the gather still pays per-element machinery across all N, while energy * mask streams the whole contiguous column at full bandwidth. A subscription that holds everyone is a scan-all with extra bookkeeping. Rule: build a subscription only when it excludes most of the population. Its win is proportional to what it leaves out - the same crossover §19 measured, where presence loses to the bool mask once nearly everyone is a member.

Exercise 7 - Two subscriptions, one entity (stretch)

The reindex cost scales with S, the number of subscriptions an entity sits in: each compaction remaps S dense arrays through old_to_new. The id key pays its redirection every tick regardless of S. Amortized over a cleanup interval G ticks:

• slot key: S × reindex / G per tick
• id key: S × redirect per tick

Since one redirect (a full id_to_slot[ids] gather) is far more expensive than one reindex amortized over tens of ticks, the slot key wins at every S tested. From ebp_partition.py claim C4 (S=2, G=30) on the Ryzen 9 270, the id key costs ~30x the slot key per tick, amortized. The slot key’s reindex burden grows with S, but it grows slowly and only at the cleanup cadence; the id key’s redirection is paid in full every tick.

27 - Working set vs cache

Concept node: see the DAG and glossary entry 27.

The working set of a loop is the data it touches per pass. The cache hierarchy (§1) is what holds that data. The two together decide the loop’s speed - once you are in numpy. In pure Python, the interpreter-dispatch tax dominates and the cliff is invisible. The moment your inner loop drops into a bulk numpy op, the cliff is real and exactly where the hardware says it is.

Sequential numpy access stays bandwidth-bound and cheap at every size - the prefetcher reaches forward and amortises the cost. The cliff is in random (gather) access, where each element is an unpredictable jump the prefetcher cannot hide. The exact ns depend on the chip; the numbers below are what this machine does, measured in §1’s cache_cliffs.py exhibit (3-run medians):

The cliff is in the gather column. The 10K and 100K rows fit in L1 / L2 (gather ratio ~5×); 1M sits in L3 (~17×); the 10M and 100M rows spill to RAM (ratio 66-80×). The numpy sequential row stays flat under ~0.7 ns/element throughout - that is what bandwidth-bound looks like on this machine. (The sub-nanosecond seq column is noisy run-to-run; the gather column and the RAM ratios are the stable claims.)

Computing your working set

The arithmetic is mechanical. Motion’s inner loop reads pos_x: float32 = 4 bytes, pos_y: float32 = 4 bytes, vel_x: float32 = 4 bytes, vel_y: float32 = 4 bytes, energy: float32 = 4 bytes. Total: 20 bytes per creature. At N creatures, working set = 20 × N bytes.

Each transition costs roughly 3-5× in per-element time when the access pattern is random. Sequential access is largely insulated by the prefetcher, but only up to RAM bandwidth - at 10M creatures and beyond, the prefetcher is no longer hiding latency, just keeping pace with what RAM can deliver.

This is what §4’s “cliff” was about, made concrete for your simulator. The transition points are not magic - they are arithmetic over your cache sizes. From §1 exercise 1 you have those numbers written down.

Why this lesson still matters when numpy hides it

Most numpy code never thinks about cache size because the inner loops are bandwidth-bound and “fast enough.” That intuition holds until the working set leaves L3 - at which point per-element cost rises 5-10× with no change to the source code. A simulator written for 1M creatures and tested at 100K never notices the cliff; it shows up the day the simulator is sized to 10M and the deadline is missed.

What sets the working set is which columns a system reads, and SoA already decides that for you (§26). Motion reads px, py, vx, vy, energy - five columns, 20 bytes per creature - and never touches birth_t or species, because those are different arrays entirely. There is no full row to trim and no hot/cold split to apply: in numpy SoA a system’s footprint is its read-set, already. The levers that move the cliff are reading fewer columns and storing them in narrower dtypes (§2), not regrouping fields.

Design discipline

• Decide the target N before the schema. The schema must fit the cache that fits N.
• Audit the inner loops. Sum the bytes per row touched. Compare to your cache sizes.
• When you cross a transition, measure - do not assume. The prefetcher and the OS will sometimes save you, sometimes not. Numpy’s bulk-op threshold also shifts with version; benchmark on the exact stack you ship.
• The narrowest dtype that holds the value (§2) is not aesthetic; it is the cliff’s distance. np.float32 over np.float64 doubles the headroom; np.uint8 for indices from 0 to 255 packs 64 to a cache line.

This is not premature optimisation. It is layout-aware design - making the schema fit the machine that will run it. A schema that ignores the cache works for small N and breaks at the scales the simulator was meant for.

Exercises

1. Compute your working sets. For each system in your simulator, compute bytes per row × N for N = 1K, 10K, 100K, 1M, 10M. Note which cache level each falls into on your machine (use lscpu | grep -i cache from §1 exercise 1).
2. Find your cliff. uv run code/measurement/cache_cliffs.py (the §1 exhibit) gives you ns/element across sizes for sequential and gather access. Plot the gather column. The transitions should match your cache sizes.
3. The unused column costs nothing. Add two columns motion never reads (birth_t: float64, species: uint8). Recompute motion’s working set and re-time motion at the cliff size from exercise 2. It should not move: in SoA the extra columns are separate arrays motion never loads. This is why there is no hot/cold split to apply in numpy (§26) - a system’s footprint is already just the columns it reads.
4. A wider dtype. Change energy: float32 to energy: float64. Recompute the working set. Time motion. The cliff should move inward (closer to smaller N).
5. Random vs sequential, your machine. Re-read the gather/seq ratio in the cache_cliffs table for your output. The factor ~5× → 80× growth across sizes is your machine’s cache-vs-RAM cost gap. Memorise this number; it is the answer to “how much does a random access cost compared to a sequential one on this hardware?”.
6. (stretch) The L1 sweet spot. Find the N at which motion’s working set fills L1 to roughly 75%. Run the motion loop in tight repetition (call it 1,000 times in a row, no other work between calls). The L1-resident loop should run at a stable ~0.2 ns/element for the entire run. The closest L2-only neighbour should be 3-5× slower.

Reference notes in 27_working_set_vs_cache_solutions.md.

What’s next

§28 - Proximity is a property of position computes neighbour structure from the position stream, and its cell-ordered compaction (with §26) makes the spatial gather sequential.

Solutions: 27 - Working set vs cache

Exercise 1 - Compute your working sets

For the motion system (pos_x, pos_y, vel_x, vel_y, energy at float32):

The cliff is at the L3 → RAM transition. The exact size depends on your CPU’s L3 (run lscpu from §1 exercise 1 to confirm).

For the starvation system (reads energy only - 4 bytes per creature):

The starvation system fits more creatures per cache level than motion, because it touches fewer bytes per row. Smaller working set, larger headroom.

Exercise 2 - Find your cliff

uv run code/measurement/cache_cliffs.py

From §1 - gather column (random access):

N           gather (ns/elem)
10,000          1.62
100,000         2.24
1,000,000       3.69
10,000,000      7.60
100,000,000     7.78

Transitions visible: 10K → 100K (L1 → L2, ~1.4×), 100K → 1M (L2 → L3, ~1.6×), 1M → 10M (L3 → RAM, ~2.1×). The cliff is shallowest at the L1/L2 boundary and steepest at L3/RAM on this machine.

Exercise 3 - The unused column costs nothing

Add birth_t: float64 and species: uint8 - two columns motion never reads. Motion’s working set is unchanged: 20 bytes per creature (pos_x, pos_y, vel_x, vel_y, energy = 5 × float32). The new columns are separate np.ndarrays; pos += vel * dt and energy[...] -= ... never load their bytes.

• Motion’s working set at 1M: 20 MB, with or without the extra columns. The cliff does not move.
• This is the SoA dividend §26 names: a system’s footprint is its read-set, already. There is no hot/cold split to apply, because the cold columns were never in motion’s working set to begin with.

The one layout where unused fields do cost you is a numpy structured array (one combined dtype for the whole row): there arr['pos_x'] strides past every other field’s bytes on every read. The fix is not a split; it is SoA, which this book has used since §7. Keep the columns separate and the question never arises.

Exercise 4 - A wider dtype

energy = np.zeros(n, dtype=np.float64)         # was float32 - doubles the bytes

Working set per creature: 20 → 24 bytes (one column doubled). Cliff moves inward by ~20%. At N=1M, working set 24 MB → still fits typical L3. At N=1.5M, 36 MB → starts to spill. The motion timing rises proportionally to the bytes read (sequential access is bandwidth-bound; bytes moved is the cost).

This is §2’s narrowest-dtype discipline re-applied at scale. Choosing float32 over float64 doubles your population headroom in cache. The choice is not aesthetic - it is “how many creatures can my simulator host at L3-resident speed?”