An Introduction to Programming using entity-component-systems & existence-based processing in python

written by Bjorn Madsen updated: 2026-05-09

Read online: Codeberg · GitHub Pages

Clone source: git clone https://codeberg.org/root-11/intro-book-python.git · git clone https://github.com/root-11/intro-book-python.git

Issues: Codeberg · GitHub

This book teaches programming from first principles of data-oriented design, entity-component-systems (ECS), and existence-based processing (EBP). It uses Python and numpy as the only languages.

The book is structured around forty-three concepts (the DAG) and their canonical wording (the glossary). Sections are short — two to three pages of prose followed by four to twelve compounding exercises. Concepts are named only after they are built: every section earns its vocabulary through working code, not the other way around.

The through-line is a small ecosystem simulator built in stages from one hundred wandering creatures to a hundred million streamed ones. The simulator’s specification is at code/sim/SPEC.

This is the Python edition — a sister volume to the Rust edition of the same book. Same forty-four sections, same DAG, same simulator. The variation is per-chapter commentary on what Python’s defaults push the reader into, and why ECS and EBP win even in a slow language. The thesis the edition carries: ECS and EBP beat OOP because they process more efficiently (operations grouped over arrays), they extend more cleanly (data-oriented composition over class graphs), and they have smaller memory footprint (typed columns over object graphs).

What carries this edition is the evidence. Every load-bearing claim is backed by a measurement the reader can reproduce on their own laptop in under a minute. The exhibits live in code/measurement/ and run via uv run code/measurement/<file>.py.

This is a work in progress. Section ordering is by the DAG; reading order can be linear (front to back) or by following the cross-links wherever they lead.

Who this book is for

You used Python last week. You wrote a class, put instances in a list, iterated over them. Your code worked, but it was slower than you expected, and you have started wondering whether the standard idioms are the bottleneck.

This book is for people who want to find out. The premise is that they are — and that the architecture this book teaches is what Python is fast in, when Python is fast at all.

Many online books include a playground that runs the code in your browser. This one does not, on purpose: the measurements only mean something when they come from your hardware.

Background

You should be comfortable with high-school algebra and a command line — running a command, changing directories, reading error messages without panic. A laptop with internet is enough; the book uses Python 3.11+, numpy, and uv for environment management. Everything else is standard library.

You do not need prior expertise in numerics, parallel computing, or game development. The book teaches numpy and the simulator together; the language is a vehicle, not the subject.

A first taste

Before any vocabulary is named, here is what an ECS world looks like in fifteen lines of Python. One hundred creatures, each with a position and a velocity, moving for thirty ticks of simulated time. No classes, no instances, no method calls — four numpy arrays indexed in lockstep, and a function (the per-tick update) that advances every creature in one stride.

import numpy as np

n = 100
x  = np.arange(n, dtype=np.float32) * 0.1
y  = np.sin(np.arange(n, dtype=np.float32))
vx = ((np.arange(n) * 7) % 11).astype(np.float32) * 0.01 - 0.05
vy = ((np.arange(n) * 13) % 7).astype(np.float32) * 0.01 - 0.03

for tick in range(30):
    x += vx
    y += vy
    if tick % 10 == 0:
        print(f"tick {tick}: creature 17 at ({x[17]:.2f}, {y[17]:.2f})")

Run it locally. Three lines print, the script stops. That is the entire shape of what the rest of the book grows: tables (the four arrays), a tick (the outer loop), a system (the per-tick update). Everything that follows is the discipline that lets this same shape carry a hundred million creatures without falling apart.

The familiar Python shape — a Creature class, a list of instances, a step() method — works at this size too. It stops working at a million, and the reason is in §2: an order of magnitude more memory per creature, an order of magnitude slower per tick. The book teaches the layout that survives the next zero.

Running the code

Python has no equivalent of the Rust Playground — there is no browser-hosted runner that reproduces the numbers a chapter quotes. Every measurement and exhibit in this book runs locally, using uv to manage the Python toolchain and environment. To run anything, you will want a clone of the book’s repo:

git clone https://codeberg.org/root-11/intro-book-python.git
cd intro-book-python
uv run code/measurement/cache_cliffs.py

Each code/measurement/<name>.py file is one exercise group, runnable in isolation. The numbers it prints are yours — they come from your hardware. The exercise asks “how fast does your machine run this?”, and that question only has a real answer locally.

From the simulator chapters onward (§11+), the exercises stop being self-contained scripts. They build the through-line: a Python program that grows from one hundred wandering creatures to a hundred million streamed ones. That program holds state between runs, which is what uv run and the project layout buy you.

The companion edition

If you already know Python well and want compile-time enforcement of the discipline this book teaches by convention, the Rust edition covers the same forty-four sections in Rust. The architecture is identical; the language differs. Many readers find that watching the borrow checker enforce in Rust what this edition asks for as discipline is a useful calibration in the other direction too.

Nomenclature

Quick reference for symbols, notation, and abbreviations the book uses. Concept definitions live in the glossary; this page covers the shorthand only.

Symbols

Text formatting

Variables you will see across chapters

Python types and their numpy counterparts

This book uses numpy’s typed dtypes for hot data. The mapping the reader will see most often:

Conventions for code blocks

1 — The machine model

Concept node: see the DAG and glossary entry 1.

Most explanations of “how a computer works” use a diagram with a CPU and a single big block called memory. The diagram is wrong. Memory is many things at different speeds, and which one your data sits in decides whether your program is fast or slow.

Inside the CPU there is L1 cache — small, sometimes only 32 KB per core, but a read from it costs about one nanosecond. Around it sits L2 — a few hundred KB, around 3-4 ns. Then L3 — measured in megabytes, around 10 ns. Outside the CPU sits main memory (RAM) — gigabytes, around 100 ns per read. The numbers vary by chip; the ratios are stable. L1 is roughly a hundred times faster than RAM.

When your code reads arr[17], the CPU does not pull just byte 17. It pulls a whole 64-byte chunk — a cache line — and keeps that line in L1. The next read of arr[18] is then almost free. Reading sequentially is fast because every line that gets loaded is mostly used before it gets evicted. Reading at random is slow because every read costs a fresh trip to RAM.

A pointer is an address in memory. Following one is one memory read at an address the CPU does not get to predict. If the address is in cache, the read is fast; if not, you wait the full ~100 ns. A program with many objects and many pointers between them is a program with many of those waits.

Why you have not had to think about this

If you used Python last week, none of the above came up. The interpreter ran your code, the operating system handed it memory, and it worked. You felt no cliff at 100 KB or 100 MB. You wrote a for loop, the loop ran, and the cost per element was whatever it was.

That experience is real, and it is hiding the machine from you. The cost of one iteration of a Python for loop — PyObject_Add, the refcount increment, the PyLong boxing, the bytecode dispatch — is around 5 nanoseconds per element on this machine. That number is higher than an L3 cache miss. So when you iterate over a Python list, the cache hierarchy is invisible to you: you spend so long in the interpreter on every step that whether the next byte was in L1 or had to come from RAM is rounding error.

This is the missing piece of the machine model in Python. The hierarchy is still there; the bottleneck just moved. To see the machine, you have to look in places where the interpreter dispatch isn’t dominating. Two such places, both measurable on your laptop:

1. Sum a million int64s, three ways. code/measurement/cache_cliffs.py walks N from 10K to 100M and times: sum(lst) on a Python list, arr.sum() on a contiguous numpy array, and arr[idx].sum() where idx is a shuffled permutation. On this machine:

Read the columns. The Python list is flat at ~4.6 ns/element across five orders of magnitude. From inside the interpreter the cache hierarchy does not exist. The numpy sequential column is 25-30× faster and reveals the bandwidth — the inner loop is C, the bytes are typed, the prefetcher works. The numpy gather column is the same data accessed in a shuffled order; once the working set leaves L1 (between 10K and 100K), the per-element cost climbs, and by 100M the gap to sequential is 72×. That ratio is the L1-to-RAM cost gap on this machine, measured.

2. Take an exception once vs a million times. code/measurement/try_except.py compares try/except ZeroDivisionError against an explicit if value != 0 check, across hit rates from 0.0001% to 99.9999%. At 50/50 the try/except form is 4× slower; at 99.9999% (almost no exceptions raised) the try/except form is faster than the if. The difference is the CPU’s branch predictor: a taken branch with high frequency is essentially free; a mispredicted one costs ~10-20 cycles. The lesson is not “use try/except” or “use if” — it is that constant factors are rate-dependent, and even Python inherits this.

3. Constant factors leak through. code/measurement/string_methods.py compares %-format, f-strings, and .format for the same output. On this machine %-format is ~20% faster than f-strings, which are ~5% faster than .format. None of this matters in a one-off log line. All of it matters in a tight loop. The “modern idiomatic” choice is not automatically the cheap choice.

What this chapter is asking you to do

The dominant fact about modern CPUs is that arithmetic is virtually free; the cost is getting the data to the arithmetic. A program that respects this is fast. A program that ignores it can be a hundred times slower than a program that does the same work, with the same number of additions, in a layout the cache likes.

In Python this fact wears a disguise: the interpreter is so slow that the machine appears to have no cliff. The disguise comes off the moment you leave pure Python — and almost everything this book teaches involves leaving pure Python for typed contiguous columns where the cliff is right where it always was.

This is also what makes “complexity class” misleading on its own. An O(N log N) algorithm that hits the cache hard can outrun a “faster” O(N) algorithm that scatters reads across RAM. Big-O describes how cost grows with N; layout describes the constant factor that gets multiplied in. At the scales this book targets, the constant factor often wins.

Exercises

These exercises are calibrations. Run them on your machine and write the numbers down — the rest of the book references them.

1. Look up your cache sizes. On Linux, lscpu | grep -i cache lists L1d, L1i, L2, L3 per core. (On macOS: sysctl -a | grep cache.) Write them down. These are the budgets §27 will hold you to later.
2. Run the cache-cliffs exhibit. uv run code/measurement/cache_cliffs.py. Read the output. Note the size at which the numpy gather column starts climbing — that is where you spilled out of L1. Note where it climbs again — L2, L3.
3. Confirm the interpreter mask. Modify the exhibit to print arr.tolist() sum at every size step alongside the existing measurements. Confirm that the Python list cost is still flat — the cliffs do not appear, even though the data is the same.
4. Run the try/except exhibit. uv run code/measurement/try_except.py. Note the cross-over: at what hit-rate does try/except become faster than if? On most machines it lands above 99%.
5. Run the string-format exhibit. uv run code/measurement/string_methods.py. Note the ranking on your machine. The order can shift across CPython versions — measure, do not memorise.
6. A linked list of pointers. Build a chain of 1,000,000 nodes as class Node: __slots__ = ("value", "next"), then sum value by walking .next from the head. Compare against the same sum on a numpy int64 array of the same length. The ratio you see is roughly the L1-to-RAM ratio for one level of indirection in Python — note that this ratio compounds when objects nest deeper.
7. (stretch) Read your lscpu output to your benchmarks. With your cache sizes from exercise 1 and your timings from exercise 2, identify which level of cache each step in the gather column is leaving. The transitions are not always clean — annotate where they are noisy.

Note — Numbers in this chapter were measured on this author’s machine. The shape — flat Python list, staircase numpy, widening gather/seq ratio — is robust across hardware. The exact ratios shift with CPU generation: older or smaller chips (Raspberry Pi 4, 2012-era Intel) show a graded staircase across L1/L2/L3, while modern desktop chips often show one big cliff at the L3-to-RAM boundary. Measure on your own machine; reproduce shapes, not specific numbers.

Reference notes for these exercises in 01_the_machine_model_solutions.md.

What’s next

The cache sizes you wrote down in exercise 1 and the cliffs you found in exercise 2 are the constants behind the whole book. §2 — Numbers and how they fit takes the next step: how big is each unit of data, and how many fit in a cache line?

Solutions: 1 — The machine model

These exercises are about measuring your machine. Numbers vary; ratios are stable. Run them and write down what you see.

Exercise 1 — Cache sizes

Linux: lscpu | grep -i cache. macOS: sysctl -a | grep cache.

Typical desktop x86-64 in 2026: L1d 32-48 KB per core, L2 1-2 MB per core, L3 16-128 MB shared. Apple Silicon: larger L1, very large shared L2. Older or smaller chips (Pi 4, 2012-era Intel) show a graded L1 → L2 → L3 → RAM staircase; modern desktops often show one big cliff at L3 → RAM.

Write the numbers down. §27 refers back.

Exercise 2 — Run the cache-cliffs exhibit

uv run code/measurement/cache_cliffs.py

Source: code/measurement/cache_cliffs.py.

           N     Python list     numpy seq    numpy gather   gather/seq
-----------------------------------------------------------------------
      10,000         5.72 ns      0.420 ns         1.62 ns         3.9×
     100,000         6.01 ns      0.234 ns         2.24 ns         9.6×
   1,000,000         4.78 ns      0.203 ns         3.69 ns        18.2×
  10,000,000         4.46 ns      0.196 ns         7.60 ns        38.7×
 100,000,000         4.59 ns      0.152 ns         7.78 ns        51.3×

Read the columns:
  Python list — roughly flat across sizes; interpreter dispatch dominates.
  numpy seq   — staircase; cliffs reveal L1/L2/L3/RAM transitions.
  numpy gather — random access; gap to seq widens as working set spills caches.

The L1 → L2 step in the gather column is shallow (2-3×). The L3 → RAM step is the dramatic one. The Python list column is the chapter’s whole point: from inside the interpreter the cache hierarchy is invisible.

Exercise 3 — Confirm the interpreter mask

Add to the per-N loop in cache_cliffs.py:

lst = arr.tolist()
t0 = time.perf_counter()
sum(lst)
ns_lst = (time.perf_counter() - t0) * 1e9 / n
print(f"  list cost: {ns_lst:.2f} ns/elem")

Same data, same arithmetic. The number stays in the 4-6 ns/elem band at every N. The cliff is not in the data; it’s in what is touching the data.

Exercise 4 — Run the try/except exhibit

uv run code/measurement/try_except.py

Source: code/measurement/try_except.py. Four points along the rate axis from one author’s run:

At 0% hits (every call raises), try/except costs ~11× more. At 50/50, ~4×. Around 96% hits the two cross over. At ~100% hits, try/except is the cheaper form because no exception is raised and the path is straight-line; the if form pays the comparison every time. The branch predictor does the rest: a branch with a stable outcome predicts ~100% and costs ~0 cycles; a flipping one costs 10-20.

The lesson is not “use one or the other” — it is that constant factors are rate-dependent.

Exercise 5 — Run the string-format exhibit

uv run code/measurement/string_methods.py

Source: code/measurement/string_methods.py. Median over seven runs, one author’s machine:

%-format wins by ~14%. f-string and .format are within 1% of each other on this run; their order flips between CPython versions and between integer-only vs string-heavy payloads. Measure on yours; do not memorise.

Exercise 6 — A linked list of pointers

import time
import numpy as np

class Node:
    __slots__ = ("value", "next")
    def __init__(self, value, nxt=None):
        self.value = value
        self.next  = nxt

def build(n):
    head = Node(1)
    for _ in range(n - 1):
        head = Node(1, head)
    return head

def walk_sum(head):
    s = 0
    while head is not None:
        s += head.value
        head = head.next
    return s

n = 1_000_000
head = build(n)
arr  = np.ones(n, dtype=np.int64)

t0 = time.perf_counter(); walk_sum(head); t1 = time.perf_counter()
arr.sum();                                 t2 = time.perf_counter()

print(f"linked list: {(t1 - t0) * 1e9 / n:.1f} ns/elem")
print(f"numpy array: {(t2 - t1) * 1e9 / n:.2f} ns/elem")

linked list: 18.4 ns/elem
numpy array: 0.36 ns/elem

Ratio ~50×. That is not the full L1-to-RAM ratio, and the reason matters: nodes built in a tight loop land contiguously in memory because the allocator reuses freshly-freed slots. Walking the chain accidentally inherits some of the array’s locality; the prefetcher catches part of it.

To see the cost without that accident, link nodes in shuffled order:

import random, gc
nodes = [Node(1) for _ in range(n)]
order = list(range(n))
random.shuffle(order)
for i in range(n - 1):
    nodes[order[i]].next = nodes[order[i + 1]]
head = nodes[order[0]]
del nodes; gc.collect()

linked list: 107.7 ns/elem
numpy array: 0.36 ns/elem

Now each head.next is an unpredictable jump — close to a full RAM round-trip per node, ~300× slower than the numpy sum.

The structural label “linked list” doesn’t tell you the cost. The layout in memory does. __slots__ is the floor here, not the ceiling — without it, every Node carries a __dict__ and the numbers worsen further.

Exercise 7 — Reading lscpu against your benchmarks

The transitions are noisy because:

• Cache levels overlap (a hot line stays in L1 after spilling to L2).
• Hardware prefetchers help even shuffled accesses up to a point.
• The OS may evict pages between runs.
• The shuffle is fixed across runs; some indices land near recently-touched lines and amortise.

If your noise is worse than your signal: median of five runs. If transitions still don’t line up with lscpu (e.g. L2 is 1 MB but the cliff appears at 200 KB), convert byte budgets to elements — the gather array is 8 bytes per int64, so 1 MB of L2 holds 128K elements, not 1M.

2 — Numbers and how they fit

Concept node: see the DAG and glossary entry 2.

A cache line is 64 bytes. That is the unit of memory the CPU loads at a time. Everything you do with data is, in part, a question of how many things fit in 64 bytes.

What an int actually costs

You wrote x = 1 last week and that was the end of the question. What sat in memory was a PyLong object: a header, a refcount, a length, and one or more 32-bit “digit” limbs holding the value. The minimum size, even for 0, is 28 bytes. As the value grows past one digit, the object grows by four bytes per additional digit. From code/measurement/number_footprint.py on this machine:

int 0                          28 bytes
int 1                          28 bytes
int 256 (last interned)        28 bytes
int 257                        28 bytes
int 1_000                      28 bytes
int 2**31                      32 bytes
int 2**63                      36 bytes
int 2**127                     44 bytes
float 0.0                      24 bytes
float 3.14                     24 bytes
float 1e300                    24 bytes

A PyFloat is 24 bytes, fixed. A PyLong is at least 28 bytes and grows with magnitude. A bool is also a PyLong. A complex is 32 bytes. The header alone is bigger than the value in every case.

This is the first part of the chapter’s question. Picking the narrowest type that holds your range — the discipline that defines whether a cache line packs 8 things or 64 things — does not exist in pure Python. There is no uint8. There is no int32. Every Python int is the same costly object regardless of whether it holds the value 0 or 2**63. You cannot trade range for cache lines, because you cannot pick the range.

Note — CPython caches small integers in [-5, 256] as singletons (the small-int cache). A list of zeros does not allocate a million PyLong(0) objects — it allocates a million pointers, all to the same one. Once the values escape that range, every value is a fresh allocation. Confirm this with id(0) == id(0) (true) versus id(257) == id(257) (sometimes true, sometimes not, depending on the parser’s caching of literal constants in the same compilation unit — but never reliable). Treat the small-int cache as a CPython implementation detail you cannot lean on.

What numpy gives you back

numpy makes the width budget exist again. np.int8 is one byte, range -128 to 127. np.int16 is two bytes, np.int32 is four, np.int64 is eight. np.float32 is four bytes (~7 decimal digits of precision); np.float64 is eight (~15 digits). The signed/unsigned and integer/float variants compose freely.

A np.zeros(N, dtype=np.uint8) is N bytes — flat, contiguous, no per-element header. A cache line packs 64 of them. A np.zeros(N, dtype=np.int64) is 8N bytes; one cache line packs 8. If your loop touches one element per cache line, the int64 version makes 8× as many memory loads as the uint8 version. The width budget is back.

Same exhibit, the data column tells the story at N=1,000,000:

The Python-list-vs-numpy ratio at this scale: 40× more bytes in the list compared to numpy int8, 20× vs int16, 10× vs int32, 5× vs int64. Choosing the narrowest numpy width that holds your range gives you up to 8× additional shrink on top of the list-to-numpy step. Sum times collapse from milliseconds to fractions of a millisecond — two orders of magnitude.

Pick the narrowest type that holds your range, and write down why. A 52-card deck’s suits need 4 values, ranks need 13, locations need maybe 8 — all fit in np.uint8. A creature’s pos needs about ten kilometres of grid resolved to centimetre precision; that fits in np.float32. A timestamp in microseconds for a year-long simulation needs something like 3×10¹³, which does not fit in np.uint32 (4×10⁹) but fits comfortably in np.uint64. Choose, and write the choice down.

Floats are not real numbers

They look like real numbers but are not. There are only about 4 billion float32 values; there are only about 18 quintillion float64 values; that is finite. Operations have edges: 1.0 / 0.0 = inf, 0.0 / 0.0 = nan, and nan != nan — yes, equality is broken on purpose for nan, because there is no reasonable answer. But == is also unreliable for ordinary floats: 0.1 + 0.2 == 0.3 is False, because 0.1 and 0.2 cannot be represented exactly in binary and the rounding error happens to land just past 0.3. This is why math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0) exists — it is the standard library’s acknowledgement that == is the wrong tool for floats and that comparing them needs a tolerance you choose deliberately. Subtracting two nearly equal floats loses most of their precision (this is catastrophic cancellation). Adding a tiny float to a large one quietly drops the tiny one (this is absorption). None of this is a problem if you know it is there; all of it is a problem if you assume floats are mathematics.

code/measurement/sums.py demonstrates the consequences across five pathological datasets — random balanced, large-plus-many-small, alternating signs, tiny increments, and arrays containing NaNs — using six summation algorithms (sum, math.fsum, Kahan, Neumaier, pairwise, decimal reference). Run it; read the discrepancies. The same input data summed in different orders gives different answers, and the “naive” answer is sometimes off by orders of magnitude. The fix is not “use float64 instead of float32” — it is picking a summation algorithm aware of the data shape. math.fsum and Neumaier are usually the right defaults for a single-pass sum where you cannot bound the input.

Most of this book uses np.uint8, np.uint16, np.uint32, np.float32, and np.uint64 for time. int* and float64 appear when the range or precision genuinely demands it. The choice is documented at every column declaration.

Exercises

1. Per-value cost. Print sys.getsizeof(0), sys.getsizeof(2**31), sys.getsizeof(2**127), sys.getsizeof(0.0), sys.getsizeof(True). Confirm that even a bool costs 28 bytes (bool is a subclass of int). Now print np.array([0, 2**31, 0], dtype=np.int64).nbytes. Three int64s = 24 bytes total, no headers, no per-value pointers.
2. Cache-line packing. For each numpy dtype — int8, int16, int32, int64, float32, float64 — compute how many fit in a 64-byte cache line. A np.array(_, dtype=np.int32) of 16 elements is exactly one line; a np.array(_, dtype=np.float64) of 8 elements is exactly one line.
3. Width and speed. Sum a np.ones(100_000_000, dtype=np.int8), then a np.ones(100_000_000, dtype=np.int64). The ratio in time should be smaller than the ratio in bytes (8×) because compute is not the bottleneck — memory bandwidth is. Note also that the int8 sum overflows; this is a hint about why the book picks widths with the maximum value in mind.
4. Float weirdness. Compute 0.0 / 0.0, 1.0 / 0.0, (-1.0) ** 0.5, math.sqrt(-1.0). Print them. Then nan = float("nan"); assert nan != nan — confirm it does not raise.
5. == is the wrong tool. Print 0.1 + 0.2 == 0.3. Observe False. Print 0.1 + 0.2 to see the rounding error: 0.30000000000000004. Now use math.isclose(0.1 + 0.2, 0.3) and observe True. Read the math.isclose docs — note that the default rel_tol=1e-9 is a choice you should be making explicitly when the problem demands a tighter or looser tolerance. The standard library has isclose because the language designers know == is unreliable here; lean on it.
6. Catastrophic cancellation. Compute np.float32(1e10) - (np.float32(1e10) - np.float32(1.0)). The result should be 1.0; on float32 it usually is not. Repeat with np.float64 and observe it gets closer (but not always exactly 1.0).
7. Run the summation exhibit. uv run code/measurement/sums.py. Read the discrepancies between the algorithms across the five datasets. Note the dataset where the spread is largest. That dataset is the one that decides which summation routine you should reach for in production.
8. Choose a width. For each of these columns, write down the dtype you would pick and why: a creature’s age in ticks at 30 Hz over a year-long simulation; a card’s suit; the pixel count of a 4K screen; the user id in a system with up to 100 million users; an audio sample value in 16-bit PCM.
9. (stretch) The eps of a float. np.finfo(np.float32).eps is the smallest x such that 1.0 + x != 1.0 in float32. Compute the value, then compute np.float32(1.0) + np.float32(0.5) * np.finfo(np.float32).eps — is the result 1.0 or 1.0 + eps/2? What does this say about a sum of small numbers added one at a time to a large running total?

Reference notes in 02_numbers_and_how_they_fit_solutions.md.

What’s next

§3 — The Vec is a table takes the next step: now that you know how big the elements are, what does an np.array do with them, and what shape does the rest of the book expect them to be in?

Solutions: 2 — Numbers and how they fit

Exercise 1 — Per-value cost

import sys, numpy as np
print(sys.getsizeof(0))         # 28
print(sys.getsizeof(2**31))     # 32
print(sys.getsizeof(2**127))    # 44
print(sys.getsizeof(0.0))       # 24
print(sys.getsizeof(True))      # 28  — bool is a subclass of int
print(np.array([0, 2**31, 0], dtype=np.int64).nbytes)  # 24

A single bool costs 28 bytes — same as a small int. Three int64s in a numpy array cost 24 bytes total: no per-element header, no per-element pointer. That ratio (28 bytes per Python int each, vs 8 bytes per int64 in a column) is the size budget the rest of the book leans on.

Exercise 2 — Cache-line packing

A cache line is 64 bytes:

A np.array(..., dtype=np.int32) of 16 elements is exactly one cache line. A np.array(..., dtype=np.float64) of 8 elements is exactly one. A Python list of anything is one pointer (8 bytes) per element plus the elements as separate objects elsewhere — at most 8 list pointers per line, with the actual values at unpredictable addresses.

Exercise 3 — Width and speed

import numpy as np, time
n = 100_000_000
a8  = np.ones(n, dtype=np.int8)
a64 = np.ones(n, dtype=np.int64)

t0 = time.perf_counter(); int(a8.sum());  t1 = time.perf_counter()
t2 = time.perf_counter(); int(a64.sum()); t3 = time.perf_counter()
print(f"int8  sum: {(t1-t0)*1000:.1f} ms")
print(f"int64 sum: {(t3-t2)*1000:.1f} ms")

int8  sum: 20.8 ms
int64 sum: 14.8 ms

The result is counterintuitive: int64 is faster than int8 despite reading 8× more bytes. The reason is how numpy reductions work. arr.sum() does not accumulate in the array’s dtype; it widens to a 64-bit accumulator by default to avoid silent overflow. That widening means each int8 is read as one byte then promoted to eight bytes inside the loop, so the int8 case pays bandwidth-savings plus per-element widening — and on this machine the widening cost dominates.

To force overflow (the chapter’s “hint about why the book picks widths with the maximum value in mind”), pin the accumulator:

a8.sum(dtype=np.int8)  # now wraps; result is some int8 value, not 100_000_000

The book’s discipline therefore has two parts. Pick the narrowest dtype that holds your values (storage) and be explicit about the accumulator (arithmetic). The two are different choices.

Exercise 4 — Float weirdness

In pure Python, three of the five prompts raise — Python’s defaults protect you from the IEEE 754 edges:

>>> 0.0 / 0.0
ZeroDivisionError: float division by zero
>>> 1.0 / 0.0
ZeroDivisionError: float division by zero
>>> math.sqrt(-1.0)
ValueError: math domain error
>>> (-1.0) ** 0.5
(6.123233995736766e-17+1j)        # promoted to complex, not nan
>>> nan = float("nan"); nan != nan
True

The IEEE 754 behaviour the chapter prose describes — nan and inf from division — surfaces through numpy:

>>> import numpy as np, warnings
>>> with warnings.catch_warnings():
...     warnings.simplefilter("ignore")
...     print(np.float64(0.0) / np.float64(0.0))   # nan
...     print(np.float64(1.0) / np.float64(0.0))   # inf
...     print(np.sqrt(np.float64(-1.0)))           # nan
nan
inf
nan

nan != nan works in pure Python because float("nan") constructs the IEEE bit pattern directly; the generation of nan from arithmetic is what numpy provides and pure Python guards against. Both views matter: when you leave the interpreter for numpy columns you trade exception protection for IEEE behaviour, and you need to know the rules of the side you’re on.

Exercise 5 — == is the wrong tool

>>> 0.1 + 0.2 == 0.3
False
>>> 0.1 + 0.2
0.30000000000000004
>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True

0.1 and 0.2 are not exactly representable in binary; their sum lands one ulp past 0.3. math.isclose exists because the standard library acknowledges == is the wrong tool for floats. The default rel_tol=1e-9 is a choice — make it deliberate when the problem demands a tighter or looser tolerance. The pattern you’ll learn to reach for:

math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0)   # near-zero values need abs_tol too

Exercise 6 — Catastrophic cancellation

import numpy as np
a32 = np.float32(1e10); b32 = a32 - np.float32(1.0)
print(a32 - b32)                          # 0.0  — should be 1.0

a64 = np.float64(1e10); b64 = a64 - np.float64(1.0)
print(a64 - b64)                          # 1.0

float32 has ~7 decimal digits of precision; 1e10 already exhausts them, so 1e10 - 1.0 cannot be distinguished from 1e10 and the subtraction returns 0.0. float64 has ~15 digits, room to spare for this size. The lesson is not “use float64” — it is that the right precision depends on the dynamic range of the values you’ll subtract. A simulation that subtracts two large nearly-equal positions to compute a small velocity needs the wider type even if the final answer fits in a narrower one.

Exercise 7 — Run the summation exhibit

uv run code/measurement/sums.py

Source: code/measurement/sums.py. Five datasets × three orders × five algorithms. The dataset where the spread is largest is large_plus_small (a few values of size 10⁶ added to many values of size 1):

=== DATASET: large_plus_small (N=2000002) ===
-- Order: original --
Reference: 2000000
builtin_sum        | time_s: 0.0085 | result: 2000000     | abs_err: 0
math_fsum          | time_s: 0.0077 | result: 2000000     | abs_err: 0
kahan_sum          | time_s: 0.0918 | result: 2000000     | abs_err: 0
neumaier_sum       | time_s: 0.1427 | result: 2000000     | abs_err: 0
pairwise_sum       | time_s: 0.3901 | result: 1999998     | abs_err: 2

pairwise_sum — usually the recommended general-purpose stable summation — is off by 2 absolute on this dataset. Two of the three large values get absorbed during a partial-sum step where they are paired with a million-and-something accumulated 1s. builtin sum, math.fsum, kahan_sum, and neumaier_sum all return the exact integer answer. The lesson: stability across reorderings is not the same as stability across magnitude mixtures. math.fsum is the safest single-pass default when you cannot bound the data; pairwise wins only when magnitudes are uniform.

Exercise 8 — Choose a width

The discipline is to write down why. Two years later, when someone changes the budget (10M users → 1B users), the column type’s reasoning is the diff that matters.

Exercise 9 — The eps of a float

import numpy as np
eps = np.finfo(np.float32).eps            # 1.1920928955078125e-07
print(np.float32(1.0) + np.float32(0.5) * eps)   # 1.0
print(np.float32(1.0) + eps)                     # 1.0000001

Half an eps added to 1.0 is absorbed — the result is still exactly 1.0. One full eps added to 1.0 produces the next representable float above 1.0. This is the unit in the last place rule: floats near 1.0 have a spacing of eps; smaller additions cannot be represented and are silently dropped.

The implication for summation: adding 10⁹ values each of size 0.5 * eps to a running total of 1.0 produces a final total of 1.0, not 1.0 + 5×10². Every step rounds away the contribution. This is the failure mode kahan_sum and neumaier_sum correct: they keep a compensation term that accumulates the dropped bits across iterations and folds them back in. The book uses math.fsum (which keeps full precision via a list of exact partials) when input magnitudes are unbounded.

3 — The Vec is a table

Concept node: see the DAG and glossary entry 3.

A list in Python is a header object on the heap that stores three things: a length, a capacity (over-allocated by a small fraction), and a pointer to a contiguous run of PyObject* pointers. That last word is the lesson. The list does not contain your integers; it contains pointers to integer objects, each allocated separately on the heap. lst[i] reads a pointer from the contiguous run, then dereferences it to find the actual PyLong (28 bytes per int, 24 per float) somewhere else in memory.

If you used Python last week, this is the container you reached for, and it is the right shape for some problems. It is also the wrong shape for almost everything the trunk of this book teaches, which is “process all the rows of a table.” A list of N rows-as-tuples is one big jump table sitting in front of N+10N small objects scattered across the heap. Walking it is pointer-chasing, not sequential reading.

A numpy array — np.array(..., dtype=...) — is the same three-things-on-the-heap shape, but the contiguous run holds values, not pointers. Ten million int64s in a numpy array is 80 MB of contiguous bytes; ten million ints in a list is 280 MB of PyLong objects plus 80 MB of pointers, scattered. arr[i] computes base + i * 8 and reads — once. No object dereference. No allocation per element.

The trunk of this book uses two containers: list for the small bookkeeping (the names of your tables, the schedule of your systems) and numpy.ndarray for the rows. There are no dicts of objects, no class hierarchies, no dataclasses with __slots__ for the things that need to scale. Not because they don’t exist, but because every container that wraps a PyObject per row pays the pointer-chase tax on every read, and the rest of the book is about not paying that tax.

The flip, measured

Take the same data — N rows, K integers per row — and lay it out five ways. The first two are what the official tutorial teaches. The middle two are stdlib-only flips. The fifth is the disciplined endpoint.

code/measurement/aos_vs_soa_footprint.py builds each, in a fresh subprocess so RSS readings don’t bleed, with N=1,000,000 and K=10. Values past the small-int cache so PyLong objects aren’t shared singletons across rows. Three numbers per layout: peak RSS, construction time, time to sum column 0.

Note — Measured on this author’s machine; reproduce on yours with uv run code/measurement/aos_vs_soa_footprint.py. Order-of-magnitude is the durable claim. Numbers will shift with K, N, value range, and CPython version, but the shape — that the AoS-to-SoA flip and the boxed-to-typed flip and the Python-loop-to-C-loop flip are three independent wins — is stable across machines.

The five rows separate three independent decisions that the four-row version conflated.

The mutable AoS is worse than the immutable AoS. Replacing the inner tuples with lists costs ~60 MB of additional list-header overhead at this scale. The “list of lists” pattern is the most-taught layout in introductory Python and the most-expensive one in this comparison.

Step one — AoS → SoA — is the speed flip. Tuple-of-lists is the same code an intermediate Python programmer might write without ever touching numpy. It saves only ~12% on memory but sums column 0 about 10× faster than the AoS forms. The win is the access pattern: walking one contiguous list of 1M PyLong pointers instead of walking 1M tuple objects and dereferencing through each one to reach row[0]. Storage is barely better; the loop is dramatically better.

Step two — boxed list → typed bytes — is the memory flip. Going from list[int] to array.array('q', …) shrinks each column from ~38 MB of pointers-and-PyLong-objects to ~8 MB of contiguous int64 bytes. The whole structure drops to ~77 MB total, smaller than numpy in this run (numpy carries ~20 MB of one-off import overhead). But the column-sum slows down — 2.5 ms → 11.6 ms — because Python has to unbox each int64 into a temporary PyLong before adding it. The unboxing tax buys back about a third of the SoA speed win. Typed storage saves bytes; it does not save the inner loop.

Step three — Python loop → C loop — is the order-of-magnitude move. np.sum walks the same typed bytes that array.array stored, but the loop is in C and the interpreter is stepped out of the way. 11.6 ms → 0.4 ms; about 30× speedup on the same bytes, no further memory saving (and a small import-overhead cost). This is the layout the simulator (§11+) and every system after it depends on.

Read the three steps together: the SoA flip is the speed move, the typed-storage flip is the memory move, the C-vectorisation flip is the speed move again at a larger scale. Each is a separate decision; each can be taken without the others. Numpy happens to bundle the second and third into one library, which is why most teaching collapses them into “use numpy.” The exhibit shows they are separate wins.

The Python-default trap, named

The official tutorial is not wrong. It’s optimised for teaching the language, not for teaching layout. The path it teaches looks like this:

1. Make a class for the row.
2. Put instances in a list.
3. Reach for dataclass when the class gets noisy.
4. Reach for __slots__ when memory pressure shows up.

Each step is a local improvement and a global trap. Step 1 commits you to AoS. Step 2 puts pointers between the rows. Step 3 makes the AoS more ergonomic. Step 4 saves a per-instance __dict__ but does nothing about the fundamental shape — every row is still its own heap object reached through a pointer. The __slots__ win is real and small; the SoA win is the same data costing 4-5× less memory, and you don’t need a class at all.

There is no such thing as a cost-free abstraction. Every pointer has a cost, and in a list of rows that cost multiplies linearly with the row count. The four-step path stacks pointers: an outer list of N row-pointers, each row pointing to K field objects, each field a separately allocated value somewhere else on the heap. __slots__ removes one layer (the per-instance __dict__); the SoA flip removes the rest. The next several phases of this book teach the alternative.

Exercises

1. Pointer-chase or value-read. Print sys.getsizeof(0), sys.getsizeof(1000), sys.getsizeof(10**100). Note that even a small Python int costs 28 bytes. Now print np.array([0, 1000, 10**18], dtype=np.int64).nbytes. Three int64s = 24 bytes, and there are no per-element headers.
2. The interning trap. Repeat exercise 1 with values 0 and 1, then again with values 257 and 1000. Use id() to confirm that [0] * 1_000_000 shares one PyLong object across all positions, but [1000 + i for i in range(1_000_000)] does not. The “list of small ints is cheap” intuition only holds inside CPython’s small-int cache [-5, 256].
3. Capacity vs length. Build lst = []. In a loop, append 0..1000 and print len(lst) and sys.getsizeof(lst) after each step. Observe the over-allocation pattern — list grows in chunks, like Vec::push, but the chunks are CPython implementation detail (currently ~1.125 × growth).
4. Run the §3 exhibit. uv run code/measurement/aos_vs_soa_footprint.py. Read the output. The sum-c0 column matters: even if you ignore the memory line, the column-sum cost gap between layouts 1 and 4 is two orders of magnitude on the same data.
5. The dict trap. Build d = {i: i*i for i in range(1_000_000)} and time looking up 100,000 random keys. Build arr = np.arange(1_000_000) ** 2 and time the same access pattern via arr[idx]. Note that you have replaced “look up by integer” with “index by integer,” and the structures cost different amounts.
6. swap-remove vs remove. Build lst = list(range(1_000_000)). Time removing 100 elements from the middle by lst.pop(500_000) (slow — every pop shifts ~half the list). Time the equivalent via lst[i] = lst[-1]; lst.pop(). Note the orders-of-magnitude difference. This trick will earn its keep at §21.
7. (stretch) Read your own array. Use np.frombuffer(arr.tobytes(), dtype=np.int64) and confirm that arr.data.tobytes() is exactly arr.size * 8 bytes long. The bytes you would write to disk are the bytes already in memory. This is what §36 — persistence means by “tables serialise themselves.”

Reference notes in 03_the_vec_is_a_table_solutions.md.

Applied reference

If you want to see this discipline carried through a real piece of code, read .archive/simlog/logger.py. It is a 700-line columnar logger that parks dict payloads into pre-allocated numpy columns, with a double-buffered design that lets the simulation write to one buffer while a background thread dumps the other to disk. The book does not require you to read it now. It’s the destination this chapter and the next several point at.

What’s next

§4 — Cost is layout, and you have a budget takes the layout reasoning into per-tick territory: how many bytes can you actually move per tick on your machine, and what does that buy you in entities? After that, §5 — Identity is an integer is where the through-line simulator gets its first concrete shape.

Solutions: 3 — The Vec is a table

Exercise 1 — Pointer-chase or value-read

>>> import sys, numpy as np
>>> sys.getsizeof(0)            # 28
>>> sys.getsizeof(1000)         # 28
>>> sys.getsizeof(10**100)      # 72  — large ints grow per limb
>>> np.array([0, 1000, 10**18], dtype=np.int64).nbytes
24

Three int64s in a numpy array: 24 bytes, no per-element headers. Three PyLongs in a list: 28 + 28 + 72 = 128 bytes for the values plus 8 × 3 = 24 bytes of pointers in the list’s backing array plus the list header. The numpy column is the values; everything else in the list version is bookkeeping.

Exercise 2 — The interning trap

>>> a = [0] * 1_000_000
>>> b = [1000 + i for i in range(1_000_000)]
>>> len(set(id(x) for x in a[:100]))         # 1   — all the same object
>>> len(set(id(x) for x in b[:100]))         # 100 — every value is its own PyLong

[0] * 1_000_000 does not allocate a million PyLong(0)s; it allocates a million pointers, all to one shared 0 object. The list weighs 8 MB of pointers + one 28-byte int. The intuition “a list of small ints is cheap” is true inside CPython’s small-int cache ([-5, 256]) and false everywhere else.

id(257) == id(257) and id(1000) == id(1000) may both return True within a single statement because the parser caches literal constants in a compilation unit. Across statements, identity is not guaranteed for values outside [-5, 256]. Don’t lean on that — it’s an implementation detail of how the bytecode compiler stores literals, not a runtime property of integers.

Exercise 3 — Capacity vs length

import sys
lst = []
prev = sys.getsizeof(lst)
sizes = []
for i in range(1001):
    lst.append(i)
    s = sys.getsizeof(lst)
    if s != prev:
        sizes.append((len(lst), s))
        prev = s
print(sizes[:8])
print(f"growth points up to N=1000: {len(sizes)}")

[(1, 88), (5, 120), (9, 184), (17, 248), (25, 312), (33, 376), (41, 472), (53, 568)]
growth points up to N=1000: 28

list over-allocates and re-allocates in chunks, like Rust’s Vec::push. The growth pattern (currently ~1.125 × capacity) is a CPython implementation detail — different versions and pypy/micropython will pick different multipliers. The principle is identical to Vec: amortised O(1) push, occasional copy. The takeaway is the same as in any growable container: if you know the final size, pre-allocate (np.zeros(N, ...) for numpy; [None] * N then assign for lists) instead of pushing.

Exercise 4 — Run the §3 exhibit

uv run code/measurement/aos_vs_soa_footprint.py

Source: code/measurement/aos_vs_soa_footprint.py. N=1,000,000 rows, K=10 ints per row, values past the small-int cache, each layout in a fresh subprocess:

layout                                          build (s)   RSS (MB)  sum c0 (s)
--------------------------------------------------------------------------------
1. list of tuples              (AoS)                0.744      437.0      0.0249
2. list of lists               (AoS)                0.615      498.2      0.0269
3. tuple of lists              (SoA stdlib)         0.463      382.9      0.0025
4. tuple of array.array        (SoA typed)          0.660       76.7      0.0116
5. tuple of numpy int64 arrays (SoA numpy)          0.092       93.8      0.0004

Ratios vs layout 5 (numpy SoA):
  1. list of tuples              (AoS)             4.7× memory     8.1× build    69.2× sum-c0
  2. list of lists               (AoS)             5.3× memory     6.7× build    74.7× sum-c0
  3. tuple of lists              (SoA stdlib)      4.1× memory     5.1× build     6.9× sum-c0
  4. tuple of array.array        (SoA typed)       0.8× memory     7.2× build    32.2× sum-c0

The five rows separate three independent wins:

• AoS → SoA (1/2 → 3): the speed flip. ~12% storage win, 10× speedup on column-sum. Walking one contiguous list of pointers beats walking N tuples and dereferencing through each one to reach row[0]. No numpy required.
• SoA-list → SoA-typed (3 → 4): the memory flip. 5× storage win (~383 MB → ~77 MB) from dropping the PyLong boxes. But the sum slows down (2.5 ms → 11.6 ms) because Python unboxes each int64 to a PyLong before adding it. Typed storage saves bytes; it does not save the inner loop.
• SoA-typed → SoA-numpy (4 → 5): the C-vectorisation flip. Same bytes, 30× speedup on the same sum. The loop moves into C; the interpreter is stepped out.

The four-row form of this exhibit collapsed steps 2 and 3 into “use numpy.” The five-row form shows they are separate. Numpy happens to bundle them; array.array lets you take the memory win without the C-loop win, which is sometimes the right trade for a project that wants stdlib-only deps.

Exercise 5 — The dict trap

import time, random, numpy as np

d   = {i: i*i for i in range(1_000_000)}
arr = np.arange(1_000_000) ** 2
idx = np.array([random.randrange(1_000_000) for _ in range(100_000)])

t0 = time.perf_counter()
for k in idx: d[int(k)]
t1 = time.perf_counter()
arr[idx]
t2 = time.perf_counter()

print(f"dict 100K lookups:  {(t1-t0)*1000:.1f} ms")
print(f"numpy 100K gather:  {(t2-t1)*1000:.2f} ms")
print(f"ratio: {(t1-t0)/(t2-t1):.0f}×")

dict 100K lookups:  34.6 ms
numpy 100K gather:   0.75 ms
ratio: 46×

Both look up “by integer.” The dict pays a hash, a probe, and a PyObject* dereference per access — all in pure Python. The numpy gather is one indirection through a typed buffer in C. Same operation, 46× cost gap. When the keys are dense integers, dicts are not the right tool — the only thing they buy you is sparse indexing, and a dense column gets you indexing for free.

Exercise 6 — swap-remove vs remove

import time
lst1 = list(range(1_000_000))
t0 = time.perf_counter()
for _ in range(100):
    lst1.pop(500_000)
t1 = time.perf_counter()
print(f"100 pop(middle):    {(t1-t0)*1000:.2f} ms")

lst2 = list(range(1_000_000))
t0 = time.perf_counter()
for _ in range(100):
    i = 500_000
    lst2[i] = lst2[-1]; lst2.pop()
t1 = time.perf_counter()
print(f"100 swap_remove:    {(t1-t0)*1000:.3f} ms")

100 pop(middle):    3.9 ms
100 swap_remove:    0.019 ms

~200× difference. lst.pop(i) for i in the middle costs O(N) because every element after i shifts down one slot; 100 mid-list pops at N=1M is ~50M element moves. The swap-remove pattern is O(1): overwrite the gap with the last element, then truncate. It changes the order of remaining elements, which is fine wherever order doesn’t carry meaning. §21 builds the rest of the discipline around it.

Exercise 7 — Read your own array

>>> import numpy as np
>>> a = np.arange(10, dtype=np.int64)
>>> raw = a.tobytes()
>>> len(raw)                                  # 80 — exactly 10 × 8 bytes
>>> b = np.frombuffer(raw, dtype=np.int64)
>>> (a == b).all()                            # True

The bytes you would write to disk are the bytes already in memory. There is no serialization step. A typed numpy column is its own on-disk format up to byte-order and dtype. §36 builds on this directly: the persistence layer stores (N, dtype, raw_bytes) and round-trips losslessly, with no encoder/decoder pair to maintain.

4 — Cost is layout — and you have a budget

Concept node: see the DAG and glossary entry 4.

A program runs at some target rate. A game runs at 30 Hz or 60 Hz; an audio loop at 48 kHz; a control loop at 1 kHz; a web request handler at “as fast as a human is willing to wait”. The target rate sets a budget — the time available for one tick of work.

Every operation the program does in one tick spends from that budget. Operations have very different costs. From the numbers you measured in §1:

The bolded row is the one most explanations leave out. Inside a Python for loop, every step pays for PYTHON_NEXT_INSTR, refcount work, PyObject boxing — about 5 ns even when you do nothing. That cost is higher than an L1 read and competitive with an L3 read. It is the dominant fact about pure-Python performance, and it does not appear in any C-style cost table.

Three regimes — and a fourth

A loop is compute-bound when its cost is dominated by arithmetic — typically when the data fits in L1 and the inner work is heavy (dot products, transcendentals, integer divides). It is bandwidth-bound when its cost is dominated by how fast the memory subsystem can deliver bytes — typically when the working set is bigger than L3 but the access pattern is sequential, so the prefetcher can fill lines ahead of demand. It is latency-bound when its cost is dominated by individual memory round-trips — typically when the access pattern is random, so the prefetcher cannot help.

Python adds a fourth: interpreter-bound. From the §1 cache-cliffs exhibit, summing 100 million int64 values cost 4.59 ns per element in a Python list and 0.15 ns per element in a numpy array. The Python list run was not bandwidth-bound, nor latency-bound — the bytes were the same bytes. It was interpreter-bound. The CPU spent most of its cycles inside the bytecode dispatcher and the PyLong arithmetic path, not on the data. The fix is not “buy faster RAM”; the fix is leave pure Python for the inner loop.

The four regimes have very different time budgets:

A loop processing 1,000,000 entities in a 30 Hz tick costs 0.6% of the budget if it is bandwidth-bound, 36% if it is latency-bound, and 14% if it is interpreter-bound. The same algorithm, the same data, four ways of running it, four orders of magnitude apart. Complexity-class reasoning cannot tell these regimes apart.

Cost is layout, not just complexity

The same algorithm that costs 0.2 ms on a sequential numpy column may cost 27 ms on a list-of-tuples carrying the same data, because every row read is a pointer chase to a separately allocated tuple, and every column read inside the row is another pointer chase to a PyLong. From the §3 exhibit, summing column 0 of one million ten-int rows took 30 ms as a list of tuples and 0.4 ms as a numpy SoA — a 75× spread on the same payload. Two programs with the same big-O, same input data, and the same machine differ by almost two orders of magnitude on the inner loop, just because of where their data sits.

This gives you a design rule. Decide your target rate before you decide anything else. That sets the budget. Then when you choose data structures, ask whether the resulting working set fits in cache; ask how many memory loads per row your inner loop does; ask whether any single operation in the loop dominates the budget; ask whether you are running inside the interpreter or outside it. Most decisions become forced once the budget is named.

The reverse direction is also useful. If you find yourself wanting to add something to the inner loop — a dictionary lookup, a getattr against a class, a Python-level callback, an exception handler — count its cost in microseconds against the budget. Often the answer is “this single addition uses 80% of my tick”, and the right move is not to optimise it but to lift it out of the inner loop entirely.

The engineering analogy

The shape of this thinking is familiar to engineers in other domains. An electrical engineer designs a circuit by counting milliamps against a current budget. A structural engineer counts kilonewtons against a load budget. The data-oriented programmer counts microseconds against a tick budget. Good design is measured in millivolts and microamps — and in nanoseconds and microseconds. Pick the unit, write the budget down, count against it. Programming has no special exemption from accounting.

Note — Time is one budget. Power is another. Cache hits are energetically nearly free — the data is already next to the arithmetic units. Cache misses fire up the memory controller, the bus drivers, sometimes a DRAM refresh; that is where the watts go. A loop that fits in L2 spends most of its time on cheap arithmetic; a loop that pointer-chases through RAM spends most of its time waiting, and during the waiting the CPU drops clocks and the chip stays cool. The same SoA-and-sequential-access discipline that fits the time budget also fits a power budget. For embedded, mobile, control, and battery-powered work, power is the primary budget; time is downstream of it. The “millivolts and microamps” line above is literal, not metaphor. One Python-specific addendum: an interpreter-bound loop is also relatively power-hungry per useful operation, because the CPU is running flat-out doing dispatch work instead of arithmetic. Moving to numpy improves time and energy at the same time. There is no trade-off here — the disciplined choice is also the cheap one.

Exercises

1. Pick your rates. For each of these systems, name a plausible target rate and the resulting per-tick budget: a card game; a real-time strategy game; a market data feed; an embedded sensor controller; a web API endpoint a user is waiting for; an offline batch job that processes a billion rows.
2. Count an operation. Time a single dict[k] lookup on a dict of 1,000,000 entries (use timeit for a million repeats and divide). Note its cost in microseconds. How many can you fit in a 30 Hz tick (33 ms)? In a 1 kHz tick (1 ms)?
3. The layout difference. Sum 1,000,000 int64 values in a numpy array. Sum 1,000,000 ints in a Python dict with integer keys (use sum(d.values())). What is the per-element time difference (in nanoseconds)? Where did it go? Map the answer back to the regime table above.
4. The cliff. With your numbers from §1 exercise 2, pick a numpy array size that just fits in L2 and one that just doesn’t. Time a arr.sum() at each size. The cliff is real.
5. Working backwards from the budget. You target 60 Hz; your inner loop runs over 100,000 entities; each entity touches one cache line of state. Estimate the cost of the loop in microseconds in each of the four regimes (compute, bandwidth, latency, interpreter). Compare to your 60 Hz budget (16,666 µs). Note which regime gives you headroom and which blows the budget.
6. A bad design. Construct a Python design that is “obviously fast” by big-O reasoning but blows the 30 Hz budget on a million entities. (Hint: list of dataclass instances with a per-tick for entity in entities: entity.update() is the canonical example. Estimate its cost from the interpreter-bound row of the regime table.)
7. Find your CPU’s TDP. Look up your CPU’s rated thermal design power on the manufacturer’s spec sheet, or read it locally on Linux with sudo dmidecode -t processor | grep -i 'power\|TDP'. Note the value. TDP is what the chip can dissipate sustained without thermal throttling — burst can be 1.5-2× higher for tens of seconds; sustained settles back to TDP.
8. Battery budget. A typical laptop battery holds about 50 Wh. Your simulator runs at 30 Hz and draws an average of 8 W (mostly memory bandwidth on the inner loop). How many hours of simulation does a full charge buy? If a layout change pushes more loads to RAM and raises the average draw to 14 W, how many hours then? Express the cost of the layout change as a percentage of battery life.
9. Measure delta power. In one terminal, run a sustained sequential numpy sum loop:

   import numpy as np
   arr = np.arange(10_000_000, dtype=np.int64)
   while True: _ = int(arr.sum())
   

   In another terminal: sudo perf stat -a -e power/energy-pkg/ -- sleep 30 reads the package-energy counter over 30 seconds. Run the same measurement with a random gather version (arr[idx].sum() with a shuffled idx) and an idle baseline. Convert each to average watts. The random-access run should draw more watts than the sequential one, which should draw more than idle. The gap between them is the energy cost of breaking the prefetcher.
10. (stretch) Joules per access. Approximate energies per memory read: L1 hit ≈ 0.1 nJ, L2 ≈ 1 nJ, RAM ≈ 30 nJ (rough; published numbers vary by chip and process). Estimate the total energy of summing 10 million int64s sequentially (mostly prefetched, near-L1 cost) versus by random indices (mostly RAM misses). Convert both to milliwatt-hours and express as a fraction of a 50 Wh battery. The absolute numbers are tiny; the ratio is what your battery life and your data-centre electricity bill care about.

Reference notes in 04_cost_and_budget_solutions.md.

What’s next

You now have the machine model (§1), the data widths (§2), the table primitive (§3), and the budget calculus (§4). The next section is the conceptual heart of the book: §5 — Identity is an integer. The card game is waiting.

Solutions: 4 — Cost is layout, and you have a budget

Exercise 1 — Pick your rates

The point of writing these down is that “this should be fast” is not a budget. “33 ms” is. The instant you have a number, every line of code in the inner loop is either spending bytes of that budget or it isn’t.

Exercise 2 — Count an operation

import timeit
d = {i: i*i for i in range(1_000_000)}
t = timeit.timeit("d[42]", globals={"d": d}, number=10_000_000)
print(f"dict[k] lookup: {t/10_000_000*1e9:.1f} ns")

dict[k] lookup: 15.1 ns

At 30 Hz (33 ms): ~2.2 million lookups per tick. At 1 kHz (1 ms): ~66,000 lookups per tick.

A 1-million-entity update that does one dict lookup per entity would cost 15 ms — half a 30 Hz budget on a single bookkeeping op. Two dict lookups per entity blows the budget on bookkeeping alone, with no actual simulation work done yet.

Exercise 3 — The layout difference

import time, numpy as np
n = 1_000_000
arr = np.arange(n, dtype=np.int64)
d   = {i: i for i in range(n)}
arr.sum(); sum(d.values())                     # warmup

t0 = time.perf_counter(); int(arr.sum());    t1 = time.perf_counter()
t2 = time.perf_counter(); sum(d.values());   t3 = time.perf_counter()

print(f"numpy sum:        {(t1-t0)*1e9/n:.2f} ns/elem")
print(f"sum(d.values()):  {(t3-t2)*1e9/n:.1f} ns/elem")

numpy sum:        0.20 ns/elem
sum(d.values()):  3.6 ns/elem
ratio: 18×

The dict version is interpreter-bound: the inner loop is a pure-Python for v in values: total += v, which pays bytecode dispatch + PyLong arithmetic + refcount work per element — about 3-6 ns. The numpy version is bandwidth-bound: a tight C loop reading int64s sequentially, the prefetcher loaded ahead, the L1 line warm. Same 1M int64 payload, two regimes apart, 18× cost gap.

Exercise 4 — The cliff

import time, numpy as np
for size in [100_000, 200_000, 1_000_000, 10_000_000]:
    a = np.ones(size, dtype=np.int64)
    a.sum()                                       # warmup
    best = float("inf")
    for _ in range(3):
        t0 = time.perf_counter(); a.sum(); t1 = time.perf_counter()
        if t1 - t0 < best: best = t1 - t0
    print(f"  N={size:>10,} ({size*8/1024:>7.0f} KB): {best*1e9/size:.2f} ns/elem")

  N=   100,000 (    781 KB):  0.11 ns/elem
  N=   200,000 (   1562 KB):  0.12 ns/elem
  N= 1,000,000 (   7812 KB):  0.11 ns/elem
  N=10,000,000 (  78125 KB):  0.20 ns/elem

On this machine the cliff between L2-fitting (200 KB - 1 MB) and L3-spilling (10 MB+) is shallow on the sequential sum (0.11 → 0.20 ns/elem, less than 2× slowdown). The prefetcher is doing its job: even with the working set in RAM, sequential-access numpy hovers near memory bandwidth limits. The dramatic cliff is on the gather column from §1; sequential numpy is forgiving.

This is why the chapter distinguishes bandwidth-bound from latency-bound: same N, same array, very different cliff depending on access pattern. The cliff exists; sequential numpy hides most of it.

Exercise 5 — Working backwards from the budget

Target 60 Hz (16.67 ms = 16,666 µs); 100,000 entities; one cache line touched per entity.

100K is small enough that even a Python-loop version fits comfortably. Scale to 10M:

At 10M entities, latency-bound and interpreter-bound layouts blow the budget by 3-7×. Bandwidth-bound finishes with 88% headroom. Same algorithm, same data, same machine.

Exercise 6 — A bad design

from dataclasses import dataclass

@dataclass
class Entity:
    x: float
    y: float
    vx: float
    vy: float

entities = [Entity(0.0, 0.0, 0.1, 0.1) for _ in range(1_000_000)]

# per tick:
for e in entities:
    e.x += e.vx
    e.y += e.vy

This is the canonical “obviously fast” Python design. Big-O is O(N); the inner work is two floating-point adds. Estimating from the regime table: interpreter-bound at ~5 ns × 4 attribute touches ≈ 20 ns/entity × 1M = 20 ms per tick, ~60% of a 30 Hz budget on simulation work alone. The exhibit tick_budget.py confirms this empirically:

  1,000,000  Python dataclass list      27.525 ms     30 Hz: 82.6%     60 Hz: 165% OVER
  1,000,000  numpy SoA                   0.278 ms     30 Hz:  0.8%     60 Hz:  1.7%

100× cost gap. The big-O is the same. The constant factor — the per-element interpreter dispatch through four attribute accesses on a heap-allocated dataclass — is what blows the budget.

Exercise 7 — Find your CPU’s TDP

Linux:

sudo dmidecode -t processor | grep -i 'power\|TDP'

Or look up the CPU model on the manufacturer’s spec sheet (Intel ARK, AMD product page, Apple silicon spec). Typical 2026 figures:

Burst can run 1.5-2× higher for tens of seconds; sustained settles back to TDP. The number matters because it’s the ceiling for energy per tick on your machine — useful when budgeting battery life or cooling.

Exercise 8 — Battery budget

50 Wh laptop battery, simulator at 30 Hz:

• 8 W draw: 6.25 hours runtime.
• 14 W draw (after a layout change): 3.57 hours runtime.

The layout change cost 2.68 hours, or 43% of battery life. A change that adds memory loads to the inner loop is a change that shortens battery life by nearly half. For mobile, embedded, or any battery-powered work, this matters more than the wall-clock tick time.

Exercise 9 — Measure delta power

# Terminal 1: sustained sequential numpy sum
python3 -c "
import numpy as np
arr = np.arange(10_000_000, dtype=np.int64)
while True: _ = int(arr.sum())
"

# Terminal 2: read package energy over 30 seconds
sudo perf stat -a -e power/energy-pkg/ -- sleep 30

Repeat for the random-gather version (arr[idx].sum() with shuffled idx) and for an idle baseline. Convert each to average watts (J/30s = W).

Expected ordering: idle < sequential < gather. The gap between sequential and gather is the energy cost of breaking the prefetcher — same arithmetic, same data volume, but more memory-controller and DRAM activity per useful operation.

This exercise needs root for perf access to RAPL counters, and works on x86 Linux. On macOS, powermetrics is the analog. On bare-metal embedded, an external power meter is the honest answer.

Exercise 10 — Joules per access

Approximate energies per memory read:

For 10M int64 reads:

• Sequential (mostly prefetched): assume mostly L1-equivalent cost. 10⁷ × 0.1 nJ = 1 mJ.
• Random gather (mostly RAM misses): 10⁷ × 30 nJ = 300 mJ.

300× more energy. Convert: 1 mJ = 0.28 µWh; 300 mJ = 83 µWh. As a fraction of a 50 Wh battery: 5.6 × 10⁻⁹ vs 1.7 × 10⁻⁶ — both tiny in absolute terms. The ratio is what compounds across millions of ticks per day across millions of laptops, or across the lifetime power bill of a data centre. The disciplined layout is also the cheap one, twice over: faster and cooler per useful operation.

5 — Identity is an integer

Concept node: see the DAG and glossary entry 5.

Hand a Python programmer fifty-two cards and tell them to write code that shuffles, sorts, and deals. Ask how long.

Most will start drawing classes. The “official” Python tutorial path leads here: define class Card with __init__(self, suit, rank), then class Deck holding a list[Card], then class Hand, then probably class Player and class Game. By the time the type hints are right and the __repr__ methods print nicely, an evening has passed. There will be debates about whether Hand should contain Card instances or hold references to a shared Deck, whether Deck.shuffle() should mutate or return a new deck, whether Card should be a @dataclass(frozen=True) for hashability. None of these debates are wrong; all of them are work that has nothing to do with cards.

The whole problem fits in three lines of numpy. The way it fits is the lesson of this section.

A deck of cards has three pieces of information per card: its suit (♠ ♥ ♦ ♣), its rank (A, 2, …, K), and its current location (in the deck, in someone’s hand, in the discard pile). That is three columns. The deck itself is fifty-two rows.

import numpy as np

suits     = np.zeros(52, dtype=np.uint8)  # 0..3
ranks     = np.zeros(52, dtype=np.uint8)  # 0..12
locations = np.zeros(52, dtype=np.uint8)  # 0=deck, 1..N=hands, 255=discard

That is the deck. The whole thing is 156 bytes — three contiguous columns of 52 unsigned bytes. There is no Card class. There is no Deck class. The card at index 17 has its suit at suits[17], its rank at ranks[17], and its current location at locations[17]. The card is the index.

Filling the columns with a fresh, ordered deck is one assignment per column:

suits[:] = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks[:] = np.tile(np.arange(13, dtype=np.uint8), 4)
locations[:] = 0

Dealing card 17 to player 1 is one element write:

locations[17] = 1

Asking what’s in player 1’s hand is one numpy primitive:

hand = np.where(locations == 1)[0]

hand is a numpy array of indices into the deck — a list of card identities — not a copy of any card data. Asking how many cards are in each location is also one primitive:

counts = np.bincount(locations, minlength=2)  # counts[0] = deck, counts[1] = player 1, ...

Shuffling — the move students expect to be hard — is shuffling the order of indices. 0..52 becomes [7, 32, 1, 19, ...], and you read your way through the cards in that order:

order = np.random.permutation(52)

Look at what just happened. Nothing about the cards changed. suits[17], ranks[17], and locations[17] are exactly the values they were before. The shuffle moved indices, not data.

Sorting works the same way. To sort by suit then rank, you sort the indices by (suits[i], ranks[i]):

order = np.lexsort((ranks, suits))  # last key is primary; sort by suit first, then rank

The cards do not move. Their identifiers are reordered.

That’s the deck of cards in maybe fifteen lines of Python. It includes shuffle, sort, deal, and several queries. It is not a stylistic shortcut; it is what a deck of cards is. The class-hierarchy version’s evening of work was the cost of pretending a card was an object that owned its suit and rank, when actually a card is one number — an index — and its suit and rank are values stored in arrays at that index.

We call this identity-is-an-integer, and it is the precondition for every economy the rest of this book buys you. Persistence will work because tables are easy to serialise — three np.save calls. Parallelism will work because indices are cheap to partition. Replay will work because a deck is just three arrays in a state. None of it works if you reach for class Card.

Even which integer matters

Not every integer is the same integer for performance. From code/measurement/float_or_int_tuple.py, looking up keys in a Python dict of 10,000 entries:

A two-tuple of ints hashes and compares 2.4× faster than a three-tuple of floats. Identity-is-an-integer is not just “use a number”; it is “use a small unsigned integer, ideally in a contiguous typed array.” A np.uint8 index packs 64 to a cache line and hashes in one CPU instruction. A (float, float, float) “identity” — the kind a Python tutorial might suggest for a 3D point in a dict — pays the price three times: more bytes, slower hash, slower compare.

The card-deck columns above use np.uint8 deliberately: 0..255 covers everything (4 suits, 13 ranks, up to 254 locations), one byte per value, 64 cards per cache line. The width budget from §2 meets the identity choice from §5: a np.uint8 column is the cheapest possible identity, the cheapest possible storage, and the cheapest possible lookup, all in one decision.

Note — The strong form, which we will return to later: sometimes you do not even need the index. The pair (suit, rank) already uniquely identifies a playing card — there are only fifty-two such pairs. The index is a surrogate key; the pair is a natural key. For variable-quantity tables (creatures that come and go) you usually need a surrogate, because two creatures can be identical. For a constant-quantity 52-card deck, you do not. The book uses surrogates throughout because the simulator is variable-quantity, but knowing when you can drop the index is its own discipline.

Exercises

The first time through, write everything from scratch in deck.py. Resist the urge to add a Card class or helper methods. Three numpy arrays.

1. Build the deck. Write def new_deck() -> tuple[np.ndarray, np.ndarray, np.ndarray] that returns the suits, ranks, and locations for a fresh, ordered deck (all 52 in location 0 = deck). All three arrays are dtype=np.uint8.
2. Print a card. Write def card_to_string(suit: int, rank: int) -> str that returns strings like "A♠", "10♥", "K♦". Use it to print the whole deck.
3. Shuffle. Use np.random.default_rng(seed).permutation(52) to produce a shuffled order. Print the deck in shuffled order. Confirm by inspection that the suits, ranks, and locations arrays are unchanged.
4. Sort by suit then rank. Use np.lexsort((ranks, suits)) to produce an order such that suits come out grouped, ranks ascending within each suit. Print again. Once again, the deck arrays are unchanged.
5. Deal a hand. Move the first 5 cards from the deck (location 0) to player 1 (location 1). Print player 1’s hand using card_to_string.
6. Hand query. Write def cards_held_by(locations: np.ndarray, player: int) -> np.ndarray returning all card indices currently held by a given player. The body is one line.
7. Count by location. Write a function that returns counts grouped by location using np.bincount. Confirm counts[0] + counts[1:].sum() == 52.
8. Deal four hands. Deal 5 cards to each of players 1, 2, 3, 4. Print all four hands.
9. (stretch) Drop the index. Rewrite cards_held_by to return an (N, 2) numpy array of (suit, rank) pairs directly — no indices. What does this make easier? What does it make harder? (Hint: you cannot move the cards back to the deck without knowing which i they were.)
10. (stretch) The sort hazard. While player 1 is holding indices [3, 17, 21, 28, 41], sort the deck arrays themselves in place by suit (order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]). What does player 1 think they hold now? Print the cards at the indices [3, 17, 21, 28, 41] after the sort. This is the bug §9 — sort breaks indices was written for. Don’t fix it yet — observe it.

Reference solutions for exercises 1-3 in 05_identity_is_an_integer_solutions.md. Solutions for the rest follow the same shape.

What’s next

Exercise 10 leaves you with a bug. The next several sections build the discipline that prevents it: §6 — A row is a tuple is the next vocabulary lesson, and §9 — sort breaks indices is the fix — keep a stable id alongside the position so external references survive reordering.

Solutions: 5 — Identity is an integer

The exercises ask you to write three columns and a handful of small functions. The whole deck — shuffle, sort, deal, query — fits in about 50 lines. No Card, no Deck, no Hand.

Exercise 1 — Build the deck

import numpy as np

def new_deck() -> tuple[np.ndarray, np.ndarray, np.ndarray]:
    suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)   # 0,0,...,1,1,...,3,3
    ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)     # 0,1,..,12,0,1,..,12,...
    locations = np.zeros(52, dtype=np.uint8)                  # all in 'deck' (=0)
    return suits, ranks, locations

Total bytes: 156. The deck is three contiguous arrays of 52 unsigned bytes.

Exercise 2 — Print a card

SUIT = ['♠', '♥', '♦', '♣']
RANK = ['A','2','3','4','5','6','7','8','9','10','J','Q','K']

def card_to_string(suit: int, rank: int) -> str:
    return f"{RANK[rank]}{SUIT[suit]}"

suits, ranks, _ = new_deck()
for i in range(52):
    print(card_to_string(suits[i], ranks[i]))

The string-rendering layer is outside the deck. It looks up into two small lookup tables. The deck itself never deals in symbols.

Exercise 3 — Shuffle

rng = np.random.default_rng(seed=42)
order = rng.permutation(52)
for i in range(52):
    j = order[i]
    print(card_to_string(suits[j], ranks[j]))

order is a permutation of [0, 1, ..., 51]. Reading the deck through order reads the cards in shuffled order. suits and ranks are byte-for-byte unchanged after the shuffle — (suits == new_deck()[0]).all() is True. The shuffle moved indices, not data.

Exercise 4 — Sort by suit then rank

order = np.lexsort((ranks, suits))   # last key is primary; suit groups, ranks ascending within
for i in range(52):
    j = order[i]
    print(card_to_string(suits[j], ranks[j]))

np.lexsort returns indices that would sort by the keys (last key dominates). (ranks, suits) means: primary sort by suit, secondary by rank. Once again, suits and ranks are unchanged.

Exercise 5 — Deal a hand

locations[:5] = 1                                   # first 5 cards → player 1
hand = np.where(locations == 1)[0]                  # indices held by player 1
for i in hand:
    print(card_to_string(suits[i], ranks[i]))

One element write per card moved. The card data does not move; only the location markers change.

Exercise 6 — Hand query

def cards_held_by(locations: np.ndarray, player: int) -> np.ndarray:
    return np.where(locations == player)[0]

One line. Returns indices, not card data. The caller looks up the card data through those indices.

Exercise 7 — Count by location

def location_counts(locations: np.ndarray) -> np.ndarray:
    return np.bincount(locations, minlength=2)

counts = location_counts(locations)
assert counts[0] + counts[1:].sum() == 52
print(f"in deck: {counts[0]}, in hands: {counts[1:].sum()}")

np.bincount is the right primitive for “count by integer category” — one C-level pass over the locations array. For 52 cards the cost is negligible; the same primitive scales to 100M creatures with hunger states without changing shape.

Exercise 8 — Deal four hands

suits, ranks, locations = new_deck()
order = rng.permutation(52)

for player in range(1, 5):
    take = order[(player - 1) * 5 : player * 5]
    locations[take] = player

for player in range(1, 5):
    hand = cards_held_by(locations, player)
    cards = [card_to_string(suits[i], ranks[i]) for i in hand]
    print(f"player {player}: {cards}")

Twenty cards dealt; four arithmetic slices into a permutation; one assignment per slice. No object construction, no per-card branching.

Exercise 9 — Drop the index (stretch)

def cards_held_by_pairs(suits: np.ndarray, ranks: np.ndarray,
                        locations: np.ndarray, player: int) -> np.ndarray:
    mask = locations == player
    return np.column_stack([suits[mask], ranks[mask]])     # shape (N, 2)

What this makes easier: returning a self-contained snapshot of the hand. The caller can inspect (suit, rank) without holding a reference to the deck arrays. For constant-quantity tables (a 52-card deck never grows), this is fine.

What it makes harder: putting the cards back. To move a card from a hand to the discard pile you need to know the index, not the value — there are 52 distinct cards but no general way to invert from (suit, rank) to “which row in the deck arrays held this.” For variable-quantity tables (creatures that are born and die), the index is what survives mutations to the table; the (suit, rank) “natural key” is brittle to anything that adds rows.

The book uses indices throughout because the simulator is variable-quantity. For constant-quantity domain (a fixed 52-card deck), dropping the index is a real option.

Exercise 10 — The sort hazard (stretch)

import numpy as np
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)

# Shuffle the arrays in place so positions are non-trivial
rng = np.random.default_rng(42)
order = rng.permutation(52)
suits[:] = suits[order]
ranks[:] = ranks[order]

# Player 1 holds indices [3, 17, 21, 28, 41]
held = [3, 17, 21, 28, 41]
locations[held] = 1
print("Player 1 holds at indices", held, "→",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# → ['K♦', '3♣', 'A♦', '4♥', 'A♠']

# Now sort the deck arrays in place by suit
order2 = np.argsort(suits, kind='stable')
suits[:]     = suits[order2]
ranks[:]     = ranks[order2]
locations[:] = locations[order2]

print("After in-place sort, player 1 looks at the SAME indices", held, "→",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# → ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Player 1 holds at indices [3, 17, 21, 28, 41] → ['K♦', '3♣', 'A♦', '4♥', 'A♠']
After in-place sort, player 1 looks at the SAME indices [3, 17, 21, 28, 41] → ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Player 1 recorded indices [3, 17, 21, 28, 41] and stashed them somewhere outside the deck arrays. The sort moved cards around. Player 1’s stored indices now point at whichever cards happened to land at those positions. They are not the cards player 1 was holding.

The locations column was reordered alongside suits and ranks, so internally np.where(locations == 1) correctly identifies player 1’s cards at their new positions ([8, 20, 27, 32, 44]). The bug is in the external index list — the one the player code held outside the table. Indices are not stable across reorderings.

This is the bug §9 — sort breaks indices addresses. The fix is to issue every card a stable id (a number that travels with the card across reorderings) and let external code refer to cards by id, not by current position. The deck arrays then carry an id column whose contents are reordered along with the card data; np.where(ids == card_id) finds a card no matter how the rows have been shuffled.

6 — A row is a tuple

Concept node: see the DAG and glossary entry 6.

In §5 you built a deck of 52 cards as three numpy columns. The card at index 17 is the triple (suits[17], ranks[17], locations[17]). Together those three values are the row. There is no Card class. There is not even a tuple object — the row exists implicitly in the alignment: the same index, used in every column, recovers all the data about one card.

This is what we call a row throughout the rest of the book — a coherent set of values that belong to the same entity. In a creature table the row is (pos[i], vel[i], energy[i], birth_t[i], id[i], gen[i]). In a food table it is (pos[i], value[i], id[i]). The fields belong to the same entity by virtue of all sharing index i. There is no dataclass holding them; there is no NamedTuple instance; there is no dict. There is only the discipline that whatever index i you used to read one column, you also use to read every other column of the same table.

Why “implicit” matters in Python

Python’s tutorial reflex when it sees the word row is to reach for a class — @dataclass class Row or class Row(NamedTuple) or, if performance is mentioned, class Row: __slots__ = (...). Each of these constructs the row as an object, with a header, a refcount, and field pointers. None of them are free. From code/measurement/classes_or_tuples.py, the time to materialise 1,000,000 two-field “rows” on this machine, ordered fastest to slowest:

Two readings of this table.

First reading: the bare tuple is ~16× faster than a slotted class and ~23× faster than a frozen+slots dataclass for per-row construction. The named alternatives all pay for an object header and per-field descriptor lookup that the tuple skips. From code/measurement/simple_namespace.py, even a dict ({'x': 10.0, 'y': 20.0}) constructs faster than any of the named-class options — about 0.036 s for the same million. Naming the row is the cost; the tuple is the cheapest row that is still recognisable as a row.

Second reading — and the one this book cares about — is the top line: two bulk numpy column allocations construct 1,000,000 rows-worth of data faster than a million individual tuple literals. Bulk allocation is roughly 30× faster than the named alternatives and is not even slower than the cheapest per-row option. The shape that lets you do this — pre-allocate a column once, fill it with values, and treat row i as the implicit tuple (col0[i], col1[i], ...) — has no per-row construction cost at all. The tuple at index i only exists when you ask for it explicitly; until then it lives in contiguous bytes inside numpy columns. From the §3 footprint exhibit, one million ten-field rows cost 99 MB as numpy SoA columns and 437 MB as a list of tuples — and the SoA version pays zero per-row construction cost on top of that, because there are no row objects.

A row is a tuple, but in Python the most useful version of that statement is: a row is a tuple you do not have to build.

Alignment is the discipline

The cost of implicit binding is that you must keep the indices aligned. If you sort suits without also sorting ranks and locations, the row at every index is corrupted — the deck still has 52 entries in 52 slots, but each slot now holds the suit of one card, the rank of another, the location of a third. This is not a hypothetical bug; you produced it deliberately in §5 exercise 10, and §9 will hand you the structural fix. The rule is simple: every operation that reorders any column of a table must reorder all columns of that table together.

The discipline that makes alignment maintainable is single-writer-per-column. If only one function writes to locations, and that function writes consistently, alignment is never violated. Multiple writers to the same column race against each other and produce inconsistent rows. This is what §25 (ownership of tables) enforces: each table has exactly one writer, and a row is a tuple precisely because that one writer kept all its columns in step.

A row is a tuple — assembled from columns indexed by the same entity, kept aligned by discipline rather than by any container holding it together.

Exercises

These extend your deck.py from §5.

1. Print row 17. Write def row(suits, ranks, locations, i) returning (int(suits[i]), int(ranks[i]), int(locations[i])). Use it to print the suit, rank, and location of card 17.
2. Mishandle the alignment. Sort only suits in place: suits.sort(). Print row 17 again. The values are now from three different cards — exactly the bug.
3. Lockstep sort. Reset the deck. Now sort all three columns together using an order array: order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]. Print row 17 again. The values are from one card. (The [:] matters — it is an in-place assignment that keeps the same backing array; suits = suits[order] would rebind the name to a new array and break aliases held elsewhere.)
4. Add a fourth column. Add dealt_at = np.full(52, 255, dtype=np.uint8) (when a card is dealt at tick t, write t into dealt_at[i]; the sentinel 255 means “not yet dealt”). Modify your lockstep sort to also reorder this column. Verify by spot-check that a row is still consistent after a sort.
5. The single-writer rule. Write def reorder_deck(suits, ranks, locations, dealt_at, order). This function is the only one that should ever reorder any column of the deck. Document that contract in a docstring above the function. Refactor your shuffle and sort to call it.
6. The construction cost, your machine. Run uv run code/measurement/classes_or_tuples.py on your machine. Note the ratios. Confirm that the slotted-dataclass row, the canonical “right” answer in modern Python, is the slowest of the named options at construction.
7. (stretch) When alignment is moot. A query that uses only (suits[i], ranks[i]) to identify a card — for instance, “is this the Ace of Spades?” — does not depend on locations or dealt_at. Write such a query (one line, using np.where). The natural-key view from §5’s strong form means this query survives reorderings of unrelated columns; only suits and ranks need to be aligned with each other.

Reference notes in 06_a_row_is_a_tuple_solutions.md.

What’s next

§7 — Structure of arrays (SoA) names the layout choice you have been making implicitly: each field its own column. The next section defends that choice against its alternative.

Solutions: 6 — A row is a tuple

These exercises extend the deck.py from §5. They demonstrate one rule: every operation that reorders any column must reorder all columns together.

Exercise 1 — Print row 17

def row(suits, ranks, locations, i):
    return (int(suits[i]), int(ranks[i]), int(locations[i]))

print(row(suits, ranks, locations, 17))
# (1, 4, 0)   — card 17 is suit 1 (♥), rank 4 (5), in deck (0)

The row is the implicit tuple (col0[i], col1[i], col2[i]). Casting to int strips the numpy dtype wrapper for cleaner printing — the underlying data is unchanged.

Exercise 2 — Mishandle the alignment

suits.sort()                                         # sorts only `suits`
print(row(suits, ranks, locations, 17))
# (1, 4, 0)  — but the (1, ...) is now from one card and (4, 0) from another

After suits.sort(), position 17 contains the 17th-smallest suit value but ranks[17] and locations[17] still hold the rank and location of whichever card originally sat at index 17. Row 17 is now a Frankenstein composite of three different cards. Reading any row gives nonsense; the per-column data is internally consistent, but the table no longer has rows.

Exercise 3 — Lockstep sort

suits, ranks, locations = new_deck()                 # reset

order = np.argsort(suits, kind='stable')             # one permutation, used for all
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

print(row(suits, ranks, locations, 17))
# (1, 4, 0)  — values from one card again

A single order array, applied identically to every column, preserves alignment. The row at any new index is still a coherent tuple from one card.

The [:] matters. suits = suits[order] rebinds the local name suits to a new array; any other code holding the original suits array (a function parameter, an attribute, an element of a tuple) keeps the unsorted array. suits[:] = suits[order] writes through the existing buffer, so all aliases see the sort. Aliasing pitfalls live or die on the difference.

Exercise 4 — Add a fourth column

suits, ranks, locations = new_deck()
dealt_at = np.full(52, 255, dtype=np.uint8)          # 255 = not yet dealt

# example: deal card 17 at tick 7
locations[17] = 1
dealt_at[17]  = 7

# lockstep sort, now over four columns
order = np.argsort(suits, kind='stable')
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]
dealt_at[:]  = dealt_at[order]

# spot-check: find where card 17 ended up via dealt_at = 7
moved_to = int(np.where(dealt_at == 7)[0][0])
print(row(suits, ranks, locations, moved_to), dealt_at[moved_to])
# (1, 4, 1) 7   — same card, new index, all four columns aligned

Adding a column adds one line to every place that reorders the table. That repetition is exactly what the next exercise factors out.

Exercise 5 — The single-writer rule

def reorder_deck(suits, ranks, locations, dealt_at, order):
    """The ONLY function permitted to reorder any column of the deck.

    Applies `order` (a permutation array) identically to every column,
    in place, so external references to these arrays continue to see
    aligned rows.
    """
    suits[:]     = suits[order]
    ranks[:]     = ranks[order]
    locations[:] = locations[order]
    dealt_at[:]  = dealt_at[order]


def shuffle(suits, ranks, locations, dealt_at, rng):
    reorder_deck(suits, ranks, locations, dealt_at,
                 rng.permutation(len(suits)))


def sort_by_suit_then_rank(suits, ranks, locations, dealt_at):
    reorder_deck(suits, ranks, locations, dealt_at,
                 np.lexsort((ranks, suits)))

The contract is in the docstring; future-you (or any other reader) sees in one place what every reordering must do. Adding a fifth column means editing one function. Forgetting to update one column at the call site stops being possible — there is only one call site.

This is the §25 ownership-of-tables discipline applied at the smallest scale: one writer per column, one reorder function per table.

Exercise 6 — The construction cost, your machine

uv run code/measurement/classes_or_tuples.py

Source: code/measurement/classes_or_tuples.py. One million two-field rows, ordered fastest to slowest:

0.004 s  numpy SoA: two np.full(1_000_000, 10.0) calls (bulk)
0.011 s  bare tuple (10.0, 20.0) × 1M individual constructions
0.117 s  class with __slots__
0.157 s  typing.NamedTuple subclass
0.167 s  collections.namedtuple
0.178 s  @dataclass

Two readings:

• The slotted dataclass — the canonical “right” answer in modern Python — is the slowest of the named options. The slots win is real but small (it removes the per-instance __dict__); the dataclass overhead at construction (descriptor lookup, __init__ call) dominates.
• Bulk numpy column allocation finishes 1M rows-worth of data in 3 ms, half the time of a million bare-tuple constructions. The shape with no per-row construction cost is the cheapest shape even when measured against the cheapest per-row option.

A row is a tuple. The most useful version of that statement is: a row is a tuple you do not have to build.

Exercise 7 — When alignment is moot (stretch)

def is_ace_of_spades(suits, ranks):
    return np.where((suits == 0) & (ranks == 0))[0]

# returns the index (or indices, if duplicates) of the Ace of Spades
print(is_ace_of_spades(suits, ranks))

This query reads only suits and ranks. It is correct as long as those two columns are aligned with each other. It does not care about the alignment of locations or dealt_at. If a future reorder swaps two columns alongside suits and ranks — but for some reason fails to update dealt_at — this query still finds the Ace of Spades correctly.

This is the strong-form observation from §5: a (suit, rank) natural key uniquely identifies a card without an index. For constant-quantity tables (52 cards, fixed) this alternative works. For variable-quantity tables (creatures coming and going) you usually need a stable surrogate id, because the natural key may collide or fail to identify a row that has been re-issued. The book uses surrogates throughout because the through-line simulator is variable-quantity; this exercise is a reminder that not every table needs one.

7 — Structure of arrays (SoA)

Concept node: see the DAG and glossary entry 7.

Your deck has three numpy columns: suits, ranks, locations. Each field lives in its own array, indexed by entity. This layout is called Structure of Arrays — SoA. The opposite layout — a single list[Card] where each element is a dataclass holding all three fields — is called Array of Structs — AoS. They are different choices about where the same data lives.

# SoA: three columns, indexed in lockstep
suits     = np.zeros(52, dtype=np.uint8)
ranks     = np.zeros(52, dtype=np.uint8)
locations = np.zeros(52, dtype=np.uint8)

# AoS: one list of objects
@dataclass
class Card:
    suit: int
    rank: int
    location: int

cards: list[Card] = [...]  # 52 instances

Most Python programmers reach for AoS by default. It is what every introductory tutorial teaches: define a class for the entity, put instances in a list. The trouble is that in a real loop “the entity” is whatever the inner loop reads, not whatever the data model says belongs together. A system that counts cards in player 1’s hand reads only the location column — it does not need suits or ranks at all.

What “reads only one column” actually costs

With SoA, that count is one numpy primitive:

held_by_p1 = int(np.sum(locations == 1))

That call walks N bytes of locations, generates an N-byte boolean mask, and sums it — all inside C, no Python-level iteration. At N = 1,000,000 cards on this machine, the call takes ~0.5 ms.

With AoS, the same count is a Python for loop:

held_by_p1 = sum(1 for c in cards if c.location == 1)

That loop pays for one bytecode dispatch per card, one getattr per card, one comparison per card, and one increment per card. From §1, interpreter dispatch is ~5 ns/element, and getattr adds more. At N = 1,000,000 the same count takes 30-50 ms — two orders of magnitude slower for the identical answer on the identical data.

This is the bandwidth-bound vs interpreter-bound regime distinction from §4. SoA pushes the inner loop into C and walks contiguous bytes; AoS keeps the inner loop in the interpreter. The SoA call can run inside a 30 Hz tick (33 ms budget) at 1 million entities and use under 2% of the budget. The AoS call uses the entire tick budget at 1 million entities, leaving no room for the rest of the simulation.

The Python AoS penalty does not shrink with width

In a Rust AoS layout, the cost grows with the size of the struct: a 19-byte Card fills a cache line with three cards instead of sixty-four bytes of locations. A reader who does not need suits and ranks pays for them anyway because they ride in on the same cache line. Add a 16-byte nickname field and the gap widens.

In Python the story is different. Every field of a dataclass is a PyObject* pointer, so a “wider” Card does not put more bytes in the same cache line — it puts more pointers. The cost of c.location is not “extra cache traffic”; it is the fixed overhead of the Python attribute lookup. Adding fields you do not read makes each Card heavier in absolute terms (more allocation, more refcounts) but does not slow down the per-attribute access. The penalty is fixed by interpreter dispatch and getattr.

This makes the SoA win in Python categorical, not just quantitative. The numpy primitive escapes the interpreter entirely; the AoS loop does not. No amount of @dataclass(slots=True) discipline removes the per-attribute dispatch cost. From §6, slots reduce construction cost and per-instance memory, but every read of c.location still goes through Python’s attribute machinery.

SoA is the default

SoA is therefore the default in this book. AoS is sometimes the right choice — for example when every system reads every field of every entity on every tick (rare), or when N is so small that the loop overhead dominates regardless of layout (think dozens of items, not millions). But this is a tradeoff to earn by measurement, not to assume by habit. Write SoA first; switch to AoS only when a benchmark forces you to.

The §3 exhibit (code/measurement/aos_vs_soa_footprint.py) is the reference measurement for this chapter. Re-read its sum-column-0 row: list-of-tuples (the AoS twin) summed column 0 of one million ten-field rows in 30 ms; numpy SoA did the same in 0.4 ms. 75× faster for the canonical “system reads one column” operation. That is the regime your inner loops will live in for the rest of this book.

Exercises

You will need time.perf_counter() for some of these.

1. Build both layouts. Take your deck.py from §5 and add an AoS twin: a list[Card] of 52 entries, where Card is a @dataclass with three int fields. Build both and verify they encode the same logical content.
2. Count cards in a player’s hand, both ways. Write count_held_soa(locations, player) using np.sum(locations == player) and count_held_aos(cards, player) using a Python generator expression. Confirm they return the same number on the same deck.
3. Time the count at 10,000 entries. Replicate your deck to length 10,000. Time both functions with timeit (e.g., number=1000 for the numpy one, number=100 for the AoS one). Note the ratio in nanoseconds per element.
4. Scale to 1,000,000 entries. Repeat at length 1,000,000. The SoA version reads 1 MB of bytes; the AoS version walks a million pointer-chases through Python’s attribute machinery. Note the ratio. On most machines it is in the 50-200× range.
5. The hot/cold case, Python edition. Extend Card with a nickname: str = "" field and a dealt_at: int = -1 field — five fields total instead of three. Rebuild both. Time the count again. Note that the SoA time is unchanged (the count still walks only locations) and the AoS time is also roughly unchanged (interpreter dispatch dominates either way). Compare to the Rust version of this chapter, where the AoS time grows with row size — Python’s penalty is fixed differently.
6. A case where AoS does not lose. Write a function that updates every field of one specific card. SoA writes to three (or five) different columns; AoS writes to one Python object. For the case “update every field of one card” — single entity, no loop — AoS is competitive or better. Time it. Note that this case has no inner loop, which is why the regime distinction from §4 doesn’t apply.
7. Construct, then read. From §6 you know constructing dataclass instances is slow. Time building a million-entry AoS list once, then summing the location query 1000 times. Compare to building a million-entry SoA once, then summing 1000 times. The construction cost amortises over many reads; for short-lived data, even SoA construction time becomes a factor. (Hint: this is a foreshadowing of §22 — mutations buffer.)
8. (stretch) A from-scratch SoaDeck class. Wrap the columns (suits, ranks, locations, dealt_at) in one Python class that owns them all. Provide reorder(self, order) as the only public mutator. What do you gain in correctness? What do you lose in flexibility? (Hint: you have just rebuilt the contract from §25 — ownership of tables, four chapters ahead of schedule.)

Reference notes in 07_structure_of_arrays_solutions.md.

What’s next

§8 — Where there’s one, there’s many is the universalising principle. The deck taught it implicitly; the next section names it.

Solutions: 7 — Structure of arrays (SoA)

Exercise 1 — Build both layouts

import numpy as np
from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

def make_soa():
    return (np.repeat(np.arange(4, dtype=np.uint8), 13),
            np.tile(np.arange(13, dtype=np.uint8), 4),
            np.zeros(52, dtype=np.uint8))

def make_aos():
    return [Card(i // 13, i % 13, 0) for i in range(52)]

The two layouts encode the same logical content. The SoA version costs 156 bytes of data plus three numpy header objects. The AoS version costs 52 Card instances (each ~88 bytes including header, refcount, and three int fields) plus a list pointing at them — close to 5 KB total. 30× memory difference at N=52, before you’ve added any operations.

Exercise 2 — Count cards in a player’s hand

def count_held_soa(locations, player):
    return int(np.sum(locations == player))

def count_held_aos(cards, player):
    return sum(1 for c in cards if c.location == player)

# both return the same number on the same logical deck

np.sum(locations == player) produces a boolean mask in C, sums its True entries in C, returns an int — no Python iteration. The generator-expression form pays per-element interpreter dispatch plus getattr (.location) on each Card.

Exercise 3 — Time the count at N = 10,000

import timeit
n = 10_000
cards = [Card(i%4, i%13, i%5) for i in range(n)]
suits = np.tile(np.arange(4, dtype=np.uint8),  n//4+1)[:n]
ranks = np.tile(np.arange(13, dtype=np.uint8), n//13+1)[:n]
locations = np.tile(np.arange(5, dtype=np.uint8), n//5+1)[:n]

t_soa = timeit.timeit(lambda: count_held_soa(locations, 1), number=1000) / 1000
t_aos = timeit.timeit(lambda: count_held_aos(cards, 1),     number=100)  / 100
print(f"SoA: {t_soa*1e6:.2f} µs   AoS: {t_aos*1e6:.1f} µs   ratio: {t_aos/t_soa:.0f}×")

SoA:    5.89 µs   AoS:   181.9 µs   ratio:   31×

At N=10,000 the SoA version is 31× faster for the same answer.

Exercise 4 — Scale to 1,000,000 entries

SoA:  226.07 µs   AoS: 12,008.3 µs   ratio:   53×

The ratio widens with N because the SoA call stays bandwidth-bound (a tight C loop reading int8s sequentially) while the AoS call stays interpreter-bound (one Python step per row). Doubling N doubles both costs, but they live in different regimes — at 1M, SoA finishes in 0.2 ms, AoS in 12 ms. AoS uses 36% of a 30 Hz tick budget on a single count-by-attribute query.

Exercise 5 — The hot/cold case, Python edition

Add nickname: str = "" and dealt_at: int = -1 to Card, rebuild, time again:

SoA:  226.07 µs   AoS5: 12,524.5 µs   (vs AoS3: 12,008.3)

The SoA time is unchanged (the count still walks only locations). The AoS time is also roughly unchanged (~4% slower from slightly larger objects, but not the multiplicative blowup the Rust edition’s chapter shows).

This is the Python-specific shape of the SoA win the chapter prose names: in Rust, AoS pays for unread fields by dragging them into the cache line. In Python, AoS pays a fixed-per-attribute interpreter cost regardless of how wide the row is — adding fields you don’t read makes each Card heavier in memory but does not slow the per-attribute access. The penalty is set by getattr and bytecode dispatch, not by cache-line traffic.

The categorical SoA win in Python is the escape from the interpreter. The numpy primitive runs in C; the AoS loop runs in CPython. No slots=True, no __slots__, no @dataclass(frozen=True) removes that gap.

Exercise 6 — A case where AoS does not lose

import time, numpy as np

# AoS: update one card, all five fields
cards = [Card(0, 0, 0) for _ in range(1_000_000)]
target = cards[42]
t0 = time.perf_counter()
target.suit = 1; target.rank = 5; target.location = 2
t1 = time.perf_counter()
print(f"AoS 1-card update: {(t1 - t0) * 1e9:.0f} ns")

# SoA: same update, three columns
suits = np.zeros(1_000_000, dtype=np.uint8)
ranks = np.zeros(1_000_000, dtype=np.uint8)
locations = np.zeros(1_000_000, dtype=np.uint8)
t0 = time.perf_counter()
suits[42] = 1; ranks[42] = 5; locations[42] = 2
t1 = time.perf_counter()
print(f"SoA 1-card update: {(t1 - t0) * 1e9:.0f} ns")

For a single row update, AoS and SoA are within noise of each other — both pay one or three Python attribute accesses, no inner loop, no scaling. The regime distinction from §4 doesn’t apply because there is no loop to be inside or outside of. AoS is competitive whenever your access pattern is “touch one row, read or write all its fields” — for example, a UI inspector showing details of a selected entity.

The book’s argument is not that AoS is always worse. It is that the inner loop of every system in the simulator reads one or two columns across many rows — exactly the case where SoA wins by the order of magnitude. AoS for the bookkeeping (the list of system names, the schedule); SoA for the rows.

Exercise 7 — Construct, then read

import time, numpy as np
from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

n = 1_000_000

# AoS: build once, query 1000 times
t0 = time.perf_counter()
cards = [Card(i%4, i%13, i%5) for i in range(n)]
t1 = time.perf_counter()
build_aos = t1 - t0
t0 = time.perf_counter()
for _ in range(1000):
    sum(1 for c in cards if c.location == 1)
t1 = time.perf_counter()
read_aos = t1 - t0

# SoA: build once, query 1000 times
t0 = time.perf_counter()
suits = np.tile(np.arange(4, dtype=np.uint8), n//4+1)[:n]
ranks = np.tile(np.arange(13, dtype=np.uint8), n//13+1)[:n]
locations = np.tile(np.arange(5, dtype=np.uint8), n//5+1)[:n]
t1 = time.perf_counter()
build_soa = t1 - t0
t0 = time.perf_counter()
for _ in range(1000):
    int(np.sum(locations == 1))
t1 = time.perf_counter()
read_soa = t1 - t0

print(f"AoS: build {build_aos*1000:.1f} ms, read 1000× {read_aos*1000:.1f} ms, total {(build_aos+read_aos)*1000:.1f} ms")
print(f"SoA: build {build_soa*1000:.1f} ms, read 1000× {read_soa*1000:.1f} ms, total {(build_soa+read_soa)*1000:.1f} ms")

Build cost amortises across many reads. For long-lived data (a deck that exists for the duration of a game session), the construction cost is a one-off. For short-lived data (a list of “cards dealt this hand” rebuilt every tick), construction can dominate — and even SoA pays a non-trivial construction time for million-element columns. This foreshadows §22 — mutations buffer: pre-allocate once, mutate in place, never reconstruct in the inner loop.

Exercise 8 — A from-scratch SoaDeck class (stretch)

class SoaDeck:
    """The single owner of the deck columns. The only mutation entry point is `reorder`."""

    def __init__(self):
        self.suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
        self.ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
        self.locations = np.zeros(52, dtype=np.uint8)
        self.dealt_at  = np.full(52, 255, dtype=np.uint8)

    def reorder(self, order: np.ndarray) -> None:
        """Apply `order` to every column in lockstep — the only function permitted to do so."""
        self.suits[:]     = self.suits[order]
        self.ranks[:]     = self.ranks[order]
        self.locations[:] = self.locations[order]
        self.dealt_at[:]  = self.dealt_at[order]

    def shuffle(self, rng: np.random.Generator) -> None:
        self.reorder(rng.permutation(len(self.suits)))

    def sort_by_suit_then_rank(self) -> None:
        self.reorder(np.lexsort((self.ranks, self.suits)))

    def deal(self, indices: list[int], player: int, tick: int) -> None:
        self.locations[indices] = player
        self.dealt_at[indices]  = tick

What you gain: one writer per column. Adding a fifth column means editing one method (reorder) and one constructor; every existing call site keeps working. Forgetting to reorder a column at a call site is impossible — there is only one site.

What you lose: explicit access to the columns from outside. Code that wants to read suits directly has to either reach through deck.suits (still allowed; reads are not the issue) or go through a method. For mostly-read systems this is fine; for diagnostic code that wants to peek at internals, the indirection adds friction.

The pattern is the §25 ownership-of-tables discipline at the smallest scale. The simulator’s actual tables are larger and have more callers, but the contract is identical: one writer, one reorder, columns read freely.

8 — Where there’s one, there’s many

Concept node: see the DAG and glossary entry 8.

Code is written for the array. A function that operates on one entity is just the special case of N = 1; it does not need its own abstraction. A card game with 52 cards is three arrays — suit, rank, location — not 52 objects. A simulation with 100 creatures is six arrays of length 100, not 100 instances of Creature. The plural is the primary unit; the singular is the trivial case.

The pattern is simple. Write the array version first. The singleton drops out as a one-element slice. To shuffle one card you swap two indices in the order array — same as shuffling the whole deck. To find the highest-rank card in player 1’s hand you scan the (small) hand array — same shape as scanning all 52. To deal one card you write one cell in locations — same shape as dealing many cells.

The OOP instinct, named

This stands against an instinct most Python programmers acquire on day one: the urge to write card.shuffle() or creature.update() and then puzzle over how to do it for many. Almost every Python tutorial models behaviour as methods on objects, then introduces lists of objects as the natural way to have many, then introduces for c in creatures: c.update() as the natural way to do something for each. Three steps, each locally sensible, that together build the pattern this chapter is asking you to drop.

The puzzle does not exist when you write for arrays from the start. shuffle(deck) is one function that works for any deck, including a deck of one. update(creatures) — taking the columns as numpy arrays — is one function that works for any population, including a population of one. The method-on-object form is strictly more code than the function-over-slice form: it requires a class, an __init__, a self argument that does nothing useful at the array level, and a calling convention that prevents the inner loop from ever leaving the interpreter.

A useful test: when you find yourself writing a method on a class, ask what does this look like over an array? If the array version is shorter, drop the method. If the array version is the same length, keep it as a free function over numpy arrays — def shuffle(suits, ranks, locations, order), not class Deck: def shuffle(self): .... Either way, the singleton was never the right unit of code.

The performance argument

There is also a performance reason — sharper in Python than in any compiled language. A method that operates on one entity at a time forces the system that uses it to call the method N times. From code/measurement/cache_cliffs.py, Python per-element work cost ~5 ns regardless of the size of the data; numpy bulk work cost ~0.2 ns/element. The ratio is roughly 25× at any size, and that is just the dispatch cost — before you add the cost of getattr(creature, 'energy') once per call, the refcount work on every return, and the lost opportunity for numpy to use SIMD instructions on contiguous bytes.

In a compiled language, an “obvious” inner loop over creatures.iter().for_each(|c| c.update()) is something the optimizer can usually rescue — inline the method, fuse the body into the loop, autovectorize the result. In Python the optimizer is the bytecode dispatcher and it cannot do any of that. The per-method-call form is essentially the worst case the language offers. Writing for arrays first is a request the interpreter can fulfil — it can hand the work to numpy and step out of the loop entirely. Writing for singletons-and-iterate is a request that pins the work inside the interpreter for every element.

“Where there’s one, there’s many” is therefore not an architectural slogan but a daily practice. It costs nothing the first time. It costs everything the first time you forget.

Exercises

These extend deck.py once more. The aim is to feel the array-first pattern in your fingertips before Part 3 turns into the rest of the book.

1. The function over a slice. Write def highest_rank_in_hand(hand, ranks) where hand is a numpy array of card indices and ranks is the deck’s rank column. Body should be one line: int(ranks[hand].max()). Use it on a 5-card hand. Then use it on a 1-card hand. Then use it on an empty hand. Same function, three N values.
2. Reverse the urge. Given an OOP-style def is_face_card(self) -> bool that lives on a hypothetical Card class, rewrite it as def face_cards(ranks) returning a numpy boolean mask of shape (N,). Apply it to all 52 cards in one call: mask = face_cards(ranks); face_count = int(mask.sum()).
3. The N = 0 case. What does highest_rank_in_hand do when hand is empty? arr.max() on an empty array raises. Pick a behaviour — return None, return a sentinel, raise — and justify the choice. (Hint: most uses can short-circuit with if hand.size == 0: return None.)
4. Predicate over a single value. Suppose you want is_red(suit) for a single card (suits 0 and 1 are hearts/diamonds). Write the array version def red_mask(suits) first — one line: (suits < 2). Then convince yourself the singleton case is red_mask(np.array([suit]))[0] — the array version covers it.
5. Count overhead. Time sum(is_face_card_per_row(suits[i], ranks[i]) for i in range(52)) against int(face_cards(ranks).sum()). The array version should be measurably faster at 52, much faster at 100,000. Document the ratio. (Repeat at N = 100,000 by replicating the deck.)
6. The dataclass twin, revisited. Take your list[Card] from §7 exercise 1. Write face_count_aos(cards) as a generator-expression sum and face_count_soa(ranks) as the numpy version. Time both at 1,000,000 entities. The ratio you measure here is the same ratio §7 measured for count_held — it is not specific to one query, it is the per-element dispatch cost of any inner loop you write in pure Python.
7. (stretch) From a tutorial. Find any Python tutorial that uses a class Card with methods (__init__, is_face, __repr__, etc.). Rewrite their full card game as three (or four) numpy arrays plus free functions. Compare line counts. Compare clarity. Compare what happens when you want to query “all face cards across the table” — one numpy call versus a loop over per-card method calls.

Reference notes in 08_where_theres_one_theres_many_solutions.md.

What’s next

You have closed Identity & structure. Cards behave; rows align; layouts are SoA; the singleton drops out. The next phase is Time & passes, starting with §11 — The tick. The ecosystem simulator from code/sim/SPEC.md is about to start running.

Solutions: 8 — Where there’s one, there’s many

These exercises ask you to write the array version first and let the singleton fall out as the trivial case.

Exercise 1 — The function over a slice

def highest_rank_in_hand(hand, ranks):
    return int(ranks[hand].max())

ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
print(highest_rank_in_hand(np.array([0, 13, 26, 39, 12]), ranks))   # 12 (K)
print(highest_rank_in_hand(np.array([12]), ranks))                  # 12 (K)
print(highest_rank_in_hand(np.array([], dtype=np.int64), ranks))    # raises — see ex 3

One function, three N values. The function does not branch on N; numpy’s indexing primitive handles all three identically (modulo the empty case).

Exercise 2 — Reverse the urge

def face_cards(ranks):
    return ranks >= 10                  # J=10, Q=11, K=12

mask = face_cards(ranks)
print(int(mask.sum()))                  # 12 — three face cards × four suits

The OOP-shaped def is_face_card(self) -> bool would force every caller to write for c in cards: if c.is_face_card(): ... — back to the interpreter-bound regime. The array version face_cards(ranks) is one numpy primitive that returns a mask, costs ~25 µs at N=100K, and also answers the singleton case via face_cards(np.array([rank]))[0].

Exercise 3 — The N = 0 case

def highest_rank_in_hand(hand, ranks):
    if hand.size == 0:
        return None                    # explicit "no answer" — caller decides
    return int(ranks[hand].max())

arr.max() on an empty array raises ValueError: zero-size array ... has no identity. Three reasonable resolutions:

• Return None. Forces the caller to handle the empty case explicitly. Best when “no cards” is a normal state.
• Return a sentinel (e.g., -1 for ranks). Cheap; risks confusing data with metadata. Avoid unless the type already has a natural sentinel.
• Raise. Right when “empty hand” is a programming error in this code path (e.g., a function that should only be called when at least one card is held).

The book leans toward returning None for “no answer” cases because the type signature Optional[int] documents the possibility at the call site. The Rust edition has Option<u8> for the same reason.

Exercise 4 — Predicate over a single value

def red_mask(suits):
    return suits < 2                   # 0=♠, 1=♥, 2=♦, 3=♣ — wait, suits 0 and 1 are spades and hearts here

# the chapter assumes suit indexing where 0,1 are red. Use the project's convention.
# If suits 1 (♥) and 2 (♦) are red:
def red_mask(suits):
    return (suits == 1) | (suits == 2)

# singleton case naturally:
suit = 1
is_red = red_mask(np.array([suit]))[0]

The array version covers the singleton; the singleton wraps the array version’s input in a one-element array. There is no separate code path. (The exact suit indexing — which numbers are red — is a convention to pick once and write down; the book’s elsewhere-conventions can drift between editions.)

Exercise 5 — Count overhead

import timeit, numpy as np

# at N = 52
t_arr = timeit.timeit(lambda: int(face_cards(ranks).sum()), number=10_000) / 10_000
t_loop = timeit.timeit(lambda: sum((int(ranks[i]) >= 10) for i in range(52)), number=1_000) / 1_000
print(f"N=52:        array={t_arr*1e6:.2f} µs   loop={t_loop*1e6:.1f} µs   ratio={t_loop/t_arr:.0f}×")

N=52:        array=  2.29 µs   loop=    5.8 µs   ratio= 3×
N=100,000:   array= 25.00 µs   loop= 1199.0 µs   ratio=48×
N=1,000,000: array=228.70 µs   loop=12525.0 µs   ratio=55×

At N=52 the array version is only 3× faster — numpy’s per-call overhead matters at small N. At N=100K the ratio settles at ~50× and stays there as N grows. The interpreter-vs-bandwidth gap from §1 is this ratio.

The lesson: even at N=52, where the array version’s overhead is dominant, it is still faster. Where there’s one, there’s many; the array version is never slower beyond a couple dozen elements, and is wildly faster past a few hundred.

Exercise 6 — The dataclass twin, revisited

from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int

def face_count_aos(cards):
    return sum(1 for c in cards if c.rank >= 10)

def face_count_soa(ranks):
    return int((ranks >= 10).sum())

n = 1_000_000
ranks_col = np.tile(np.arange(13, dtype=np.uint8), n // 13 + 1)[:n]
cards = [Card(0, int(ranks_col[i])) for i in range(n)]

t_aos = timeit.timeit(lambda: face_count_aos(cards), number=5) / 5
t_soa = timeit.timeit(lambda: face_count_soa(ranks_col), number=100) / 100
print(f"AoS face count: {t_aos*1e3:.1f} ms   SoA face count: {t_soa*1e3:.2f} ms   ratio: {t_aos/t_soa:.0f}×")

AoS face count: 12.5 ms   SoA face count: 0.23 ms   ratio: 55×

Same 55× ratio as §7’s count_held. The cost gap is not query-specific; it is a property of any per-element work done in pure Python over getattr-accessed fields. Every loop you write in CPython that walks for entity in entities: ... entity.field ... lives in this cost regime. SoA + numpy primitives moves the loop into C and out of the regime.

Exercise 7 — From a tutorial (stretch)

Pick almost any “Object-oriented programming in Python” tutorial that builds a card game (Real Python, Programiz, GeeksforGeeks, the Python docs themselves all have versions). The canonical shape is:

class Card:
    SUITS = ['♠', '♥', '♦', '♣']
    RANKS = ['A', '2', ..., 'K']
    def __init__(self, suit, rank):
        self.suit = suit
        self.rank = rank
    def __repr__(self):
        return f"{self.RANKS[self.rank]}{self.SUITS[self.suit]}"
    def is_face(self):
        return self.rank >= 10

class Deck:
    def __init__(self):
        self.cards = [Card(s, r) for s in range(4) for r in range(13)]
    def shuffle(self):
        random.shuffle(self.cards)
    def deal(self, n):
        return [self.cards.pop() for _ in range(n)]

The numpy rewrite is approximately:

import numpy as np

class Deck:
    def __init__(self):
        self.suits = np.repeat(np.arange(4, dtype=np.uint8), 13)
        self.ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
        self.locations = np.zeros(52, dtype=np.uint8)
        self.dealt_at = np.full(52, 255, dtype=np.uint8)
    def shuffle(self, rng):
        order = rng.permutation(52)
        self.suits[:] = self.suits[order]
        self.ranks[:] = self.ranks[order]
        self.locations[:] = self.locations[order]
        self.dealt_at[:] = self.dealt_at[order]

Line counts: the OOP version is typically 30-50 lines for Card + Deck. The numpy version is ~15 lines. And “all face cards across the table” is one numpy call (np.where(self.ranks >= 10)[0]) instead of a loop over per-card method invocations.

Beyond line count: the numpy version is the precondition for everything in Phase 3+. Persistence is np.savez(self.suits, self.ranks, self.locations, self.dealt_at) — three or four arrays out, the same arrays in. Replay is “store the seed, replay the operations.” Parallel partitioning is “split the index range.” None of these work cleanly when the data lives behind self.cards = list[Card]. The savings show up not in this chapter but in the rest of the book.

9 — Sort breaks indices

Concept node: see the DAG and glossary entry 9.

In §5 — Identity is an integer, exercise 10 left you with a bug. Player 1 was holding the index list [3, 17, 21, 28, 41]. The dealer sorted the deck columns by suit. Player 1’s hand was now wrong — the same indices, the same slots, but different cards.

That bug is the structural fact this section names. Sorting did not damage anything; the player’s reference was never robust to begin with. An index points at a slot, not at a thing. When the slot’s contents change, the index quietly changes meaning.

It is not only sorting. Any rearrangement does it: swap_remove (a O(1) deletion that moves the last row into the freed slot, coming in §21), reshuffling for locality (§28), compacting after a batch of deletions. The same index, the same array, the same line of code, now means a different card.

“But Python objects are stable references — can’t I just go back to that?”

This is the moment many readers feel the urge to retreat. The Python reflex from §6 — class Card with attributes — gave you object identity for free. A Card instance you held a reference to last week is still the same Card object today, regardless of what happened to the list it was in. id(card) does not change. The pointer through the Python interpreter to the heap-allocated Card is stable for the lifetime of the object.

So the temptation is real: keep the index-aligned numpy columns and a parallel list[Card] of object references, and use the objects when you need stability. Or just go back to list[Card] entirely — at least the references work.

This trade does not survive contact with the §3 footprint table or the §7 access-cost table. The numpy-SoA layout is 5× smaller and 75× faster at single-column queries than list[Card]; carrying a parallel object list to “rescue” reference stability gives back most of the footprint win and adds the synchronisation problem of keeping the column data in step with the object data. You have not solved the problem; you have hidden it inside an additional invariant.

The structural fix is the one §10 builds: an id column that travels with the row across rearrangements, plus (for variable-quantity tables) a generation counter on top. The card itself is a slot; the card’s name is an integer that we choose to be stable. The cost is one extra np.uint32 column. The benefit is that every rearrangement we will need from now on — sort, swap_remove, locality-driven reordering, compaction — works without breaking outside references.

This section’s only job is to make the slot vs name distinction concrete enough that §10’s solution feels inevitable rather than ceremonial.

Note — Why feel the pain first? Because the fix in §10 is small — one extra column — and small fixes only stick if the student knows what they fix. Reading “always store an id” without first feeling the bug produces students who add ids cargo-culted, then drop them when the codebase looks too cluttered. Reading it after watching player 1 lose their hand produces students who never drop them.

Exercises

You should still have your deck.py from §5. These exercises extend it.

1. Reproduce the bug. With player 1 holding [3, 17, 21, 28, 41], sort the deck columns themselves (suits, ranks, and locations in lockstep) by suit. The pattern is order = np.argsort(suits, kind="stable"); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]. Print player 1’s hand using card_to_string. Confirm the cards have changed.
2. A second rearrangement. Instead of sorting, swap two cards’ positions:

   suits[[3, 17]] = suits[[17, 3]]
   ranks[[3, 17]] = ranks[[17, 3]]
   locations[[3, 17]] = locations[[17, 3]]
   

   Print player 1’s hand again. Same bug shape, different cause.
3. A third rearrangement. Remove the card at slot 7 with the swap_remove pattern (move the last row into slot 7, then drop the last row): suits[7] = suits[-1]; suits = suits[:-1] and likewise for the other columns. Print player 1’s hand. Note that the cards at slots [17, 21, 28, 41] are unchanged but slot 3 may now hold what was previously the last card; meanwhile slot 51 has silently been deleted.
4. Quantify the breakage. Write a function that takes the original [3, 17, 21, 28, 41] plus a freshly built deck, applies a Fisher-Yates shuffle to the deck columns themselves (order = rng.permutation(52) and reorder all three columns), and counts how many of the five references still point at the same (suit, rank) value. Run it 100 times. Roughly what fraction of references survive a random shuffle of the deck? (Spoiler: very small. With probability 1/52 per slot, the expected number that survive by accident is 5/52 ≈ 0.1.)
5. A reference that can survive. Without writing any new code — on paper — describe what kind of reference would survive a shuffle. (Hint: you already know. The card’s (suit, rank) is unique to that card. The reference that survives is the one that does not depend on the slot.)
6. The “object reference” non-fix. Build a parallel list[Card] (use a @dataclass if you wish) alongside the numpy columns. Fill them so that cards[i] mirrors (suits[i], ranks[i], locations[i]). Now sort the numpy columns by suit without updating the object list. What does player 1 see if they read from the object list? What if they read from the numpy columns? Note that you have introduced a new bug — desynchronised state — without fixing the old one.
7. (stretch) The cost of never rearranging. Suppose you decide to never sort, swap, or remove from the deck columns, to avoid this bug forever. How would shuffling work? How would discarding a card work? Why does this not scale to ten thousand creatures?

Reference notes for these exercises in 09_sort_breaks_indices_solutions.md.

What’s next

Exercise 5 points at the answer; exercise 7 makes the never-rearrange option look bad. The real fix is to store identity separately from position — an id column that travels with the row across rearrangements, with a generation counter on top for variable-quantity tables. §10 — Stable IDs and generations builds it.

Solutions: 9 — Sort breaks indices

These exercises produce the bug, vary it, and quantify it. The structural fix is in §10.

Exercise 1 — Reproduce the bug

import numpy as np
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)

# Shuffle once so positions are non-trivial
rng = np.random.default_rng(42)
order = rng.permutation(52)
suits[:] = suits[order]; ranks[:] = ranks[order]

# Player 1 records indices [3, 17, 21, 28, 41]
held = [3, 17, 21, 28, 41]
locations[held] = 1
print("before sort:",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# ['K♦', '3♣', 'A♦', '4♥', 'A♠']

# Sort the columns themselves by suit
order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

print("after sort:",
      [f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in held])
# ['10♠', 'Q♥', '3♥', '9♦', 'J♣']

Slots [3, 17, 21, 28, 41] now hold completely different cards. Player 1’s reference list has not changed; the slot contents have. The same line of code (ranks[3]) returns a different value before and after the sort. The bug is not in the sort; the bug is that the index was never a name for the card.

Exercise 2 — A second rearrangement

# fresh deck (rebuild from ex 1)
suits[[3, 17]] = suits[[17, 3]]
ranks[[3, 17]] = ranks[[17, 3]]
locations[[3, 17]] = locations[[17, 3]]

print([f"{RANK[ranks[i]]}{SUIT[suits[i]]}" for i in [3, 17, 21, 28, 41]])

Two cards swap. Player 1’s references at indices 3 and 17 now point at each other’s old contents. References at 21, 28, 41 are unchanged. Same shape of bug — index is a slot, not a name — different cause.

Exercise 3 — A third rearrangement

# swap_remove slot 7
suits[7] = suits[-1]
ranks[7] = ranks[-1]
locations[7] = locations[-1]
suits     = suits[:-1]
ranks     = ranks[:-1]
locations = locations[:-1]

Slot 7 now holds what was the last card (slot 51). Slot 51 no longer exists — the array is length 51. Player 1’s references at indices 17, 21, 28, 41 still see the original cards (those slots untouched). Reference at 3 is unchanged because slot 3 was untouched too — but the card formerly at slot 51 has been silently removed from the universe of “cards.”

This is the §21 swap_remove pattern: O(1) deletion at the cost of moving one row’s worth of data, plus changing the index of one other row. Cheap, fast, and devastating to external references.

Exercise 4 — Quantify the breakage

def survival(rng):
    suits = np.repeat(np.arange(4, dtype=np.uint8), 13)
    ranks = np.tile(np.arange(13, dtype=np.uint8), 4)
    # shuffle once so the deck is non-trivial
    o = np.random.default_rng(42).permutation(52)
    suits[:] = suits[o]; ranks[:] = ranks[o]

    held = [3, 17, 21, 28, 41]
    pairs = [(suits[i], ranks[i]) for i in held]

    # rearrange and count survivors
    o = rng.permutation(52)
    suits[:] = suits[o]; ranks[:] = ranks[o]
    return sum(1 for (s, r), i in zip(pairs, held) if (suits[i], ranks[i]) == (s, r))

rng = np.random.default_rng(0)
print(f"survived: {sum(survival(rng) for _ in range(100))} / 500")

survived: 13 / 500

Expected value: each reference has probability 1/52 ≈ 1.9% of pointing at its original card after a uniform shuffle. Five references × 100 trials × 1/52 ≈ 9.6 expected survivors. Empirically: 13 — within Poisson noise of the prediction.

98% of references are wrong after one shuffle. Not “occasionally broken in edge cases” — catastrophically broken in the common case.

Exercise 5 — A reference that can survive

The reference that survives a shuffle is the one that does not depend on the slot. The natural-key reference (suit, rank) survives any rearrangement because (suits[i], ranks[i]) is a property of the card, not of the slot. The dealer rearranges slots; the cards themselves are not changed.

But natural keys break in two cases:

• Duplicates. Variable-quantity tables (creatures, items, projectiles) routinely have rows with identical field values; “the creature with energy=10 at position (5,5)” can have many matches. A natural key needs to be a guaranteed-unique property of the row.
• Re-issues. A row removed and a new row added with the same values is indistinguishable by natural key. For variable-quantity tables this is a bug waiting to happen.

The structural fix in §10 is to invent a name and write it down: an id column whose values are guaranteed unique within the table, plus a generation counter to handle re-issues.

Exercise 6 — The “object reference” non-fix

from dataclasses import dataclass

@dataclass
class Card:
    suit: int
    rank: int
    location: int

# parallel lists
suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
locations = np.zeros(52, dtype=np.uint8)
cards = [Card(int(suits[i]), int(ranks[i]), int(locations[i])) for i in range(52)]

# sort the numpy columns, NOT the object list
order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]

# now: numpy columns sorted, cards list still in original order
print(f"numpy slot 3:   {RANK[ranks[3]]}{SUIT[suits[3]]}")
print(f"object cards[3]: rank={cards[3].rank} suit={cards[3].suit}")
# they disagree

The object list and the numpy columns now describe two different decks. Player 1 reading from cards[3] sees one card; reading (suits[3], ranks[3]) sees another. You have not fixed the index-into-slot problem — you have added a synchronisation problem on top of it.

This is why §9 explicitly rejects the parallel-object-list approach: it preserves stable references at the cost of doubling memory and inventing an alignment invariant the original problem didn’t have. The cure is worse than the disease.

Exercise 7 — The cost of never rearranging (stretch)

If the deck columns are never sorted, swapped, or compacted:

• Shuffling: must produce an order array each time and read through it indirectly. for i in range(52): print(card_at(order[i])). Every read pays one extra indirection. Workable for 52 cards.
• Discarding a card: cannot remove it from the columns; must mark it dead via a status column (e.g., locations[i] = 255). The columns grow forever. For 52 cards over a single game session, fine.
• Adding a card: np.concatenate to grow each column. O(N) per addition.

Why this doesn’t scale to 10,000 creatures (let alone the simulator’s 100M):

1. Forever-growing tables. A simulator that runs for an hour and births 10K creatures per second has 36M dead rows by the end. Reading through them costs proportionally; bandwidth is the budget; you’ve spent it on tombstones.
2. No compaction means no locality. Live and dead rows are interleaved. Cache lines hold half-tombstones. The §28 sort for locality pattern is impossible.
3. Parallel partition is impossible. §31-§32 split the table by index range; if the live data is sparse and randomly distributed across a forever-growing array, you can’t carve clean ranges.

The never-rearrange policy works for constant-quantity tables (52 cards, fixed grid sizes). It fails for everything that breathes — births, deaths, additions, removals. The book’s simulator is variable-quantity, so the next chapter builds the fix.

10 — Stable IDs and generations

Concept node: see the DAG and glossary entry 10.

In §9 you watched a player’s reference go stale because they were holding slots, not names. The fix is to give each row a name — a stable identifier — that travels with the row when it moves.

A stable id is one extra column. For the deck:

ids = np.arange(52, dtype=np.uint32)

Now every card has both a slot (its current index in the columns) and an id (its name). When you sort the columns, you reorder ids in lockstep with everything else:

order = np.argsort(suits, kind="stable")
suits[:]     = suits[order]
ranks[:]     = ranks[order]
locations[:] = locations[order]
ids[:]       = ids[order]

The card with id == 17 is still the same card — its suit, rank, and location are unchanged. It is just at a different slot.

To find a card by id, scan the ids column:

def slot_of(ids: np.ndarray, target: int) -> int | None:
    matches = np.where(ids == target)[0]
    return int(matches[0]) if matches.size else None

That is O(N), which is fine for a 52-card deck and slow for a million creatures. The fix — an id_to_slot map maintained on every rearrangement — is §23 — Index maps. For now the linear scan is honest pedagogy.

Generations: when slots are reused

The deck is constant-quantity. Always 52 cards, never more, never less. The simple ids column is enough.

For variable-quantity tables — creatures that are born and die, packets that arrive and are processed, sessions that come and go — slots get reused. A new creature is born in the slot that just held a dead one. The ids column for such a table behaves like an auto-incrementing primary key in a database: every new row gets a fresh, never-reused integer; old rows keep their original ids forever. The simulator differs from a database in one structural way — it recycles slots to keep memory bounded, while a database table just grows. That recycling is what generations exist for. Imagine code that held a reference to the dead creature: their reference points at a slot that may now hold a different creature with possibly the same id (if id reuse happens) or — worse — a valid-looking row that is no longer the row they cared about.

One more column fixes it: a gens (generation) counter that increments every time a slot is recycled. A reference is now a pair (id, gen). To dereference it, you check that the row’s stored gen still matches the reference’s gen. If it does, the reference is live. If it does not, the slot has been recycled since the reference was taken, and the dereference returns None.

from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

def get_slot(creatures, ref: CreatureRef) -> int | None:
    slot = creatures.id_to_slot.get(ref.id)
    if slot is None:
        return None
    if int(creatures.gens[slot]) != ref.gen:
        return None
    return slot

(This is one of the few places in the book where a NamedTuple earns its weight: a CreatureRef is a value passed through external code, and giving it field names makes the API readable. Per §6, the cost is real — a NamedTuple allocation per reference — but references are rare, not per-tick. Where the same lesson runs through hot data, the answer is still numpy columns.)

This is the pattern called a generational arena. It is the single mechanism behind every “handle” type in every ECS engine: Bevy’s Entity, Rust’s slotmap::SlotMap, C++’s entt::registry, and the indirect-handle pattern in databases. They differ in details — width of the id, packing into a u64, generation overflow handling — but the structural idea is the same: one column for identity, one for generation, a checked dereference.

That is enough machinery for the rest of the book to lean on. Sorting now works because the id column travels with the row. Deletion now works because the generation counter rejects stale references. Append-only and recycling tables (§24) are two policies on the same machinery.

Note — The strong form of §5 still applies. If your row has a natural key — (suit, rank), (date, ticker), (species, position) — you do not need a surrogate id. The card-game deck can be played without ids; the reference that survives is the (suit, rank) pair, because the data is unique by construction. Surrogate ids and generations earn their keep when the data has no natural unique tuple — which is most of the time once you start producing rows at runtime.

Exercises

These extend the §5 deck once more, then take a step toward the simulator’s variable-quantity case.

1. Add the id column. Add ids = np.arange(52, dtype=np.uint32) to your deck. Modify your sort so it reorders ids along with the other columns. Verify the original ids are still there, just in a new order.
2. Find a card by id. Implement slot_of(ids, target) as in the prose. Use it to look up the card with id == 17 after a sort.
3. Resolve the §9 bug. With player 1 holding ids [3, 17, 21, 28, 41] (not slots), sort the deck. Use slot_of to translate ids to slots and print the hand. Confirm the cards are unchanged.
4. Permutation-friendly hand query. Rewrite cards_held_by(locations, ids, player) -> np.ndarray to return ids, not slots. The player now holds names. Test by sorting the deck after a deal and confirming cards_held_by still returns the same five cards.
5. A first generation counter. Add gens = np.zeros(52, dtype=np.uint32). The 52-card deck does not actually recycle, but extend a small swap_remove-like operation: pop the last card from the deck (location 0), insert a “fresh” card at the freed slot, and bump that slot’s gens by one. Take a CreatureRef-style (id, gen) reference before the operation. After the operation, look up the slot by id; check gens[slot] against the reference’s gen. Confirm the dereference correctly reports stale.
6. (stretch) A tiny generational arena. Outside the deck, build a Creatures class with pos: np.ndarray (float32), gens: np.ndarray (uint32), plus free: list[int] of slots awaiting reuse. Implement insert(pos) -> CreatureRef, remove(ref), and get(ref) -> float | None. Convince yourself by example that stale references cannot read a fresh creature’s data.
7. (stretch) The shape of id_to_slot. Right now slot_of is O(N). Sketch (do not implement) the id_to_slot array — np.full(N_ids, MAX, dtype=np.uint32) — that lets you do the lookup in O(1). Note what has to happen on every reorder: when slot i is the new home of id k, id_to_slot[k] = i. This is a foreshadow of §23 — Index maps. The lookup speedup costs you another column to keep aligned.
8. (stretch) Compare with a real ECS handle. Read the Entity documentation for bevy_ecs (Rust) or look at the EntityHandle docs of any Python ECS library. Identify which of your fields and operations correspond. What does the production library add that you didn’t need for the simulator? Decide consciously whether to adopt it. (This is the from-scratch-then-price-the-crate move from §41 — Compression-oriented programming and §42 — You can only fix what you wrote.)

Reference solutions for the deck exercises (1-5) in 10_stable_ids_and_generations_solutions.md. The arena and library exercises follow the same shape and are worth working without reference.

What’s next

You now have stable references. The next thing the simulator will need is to look up a row by id in O(1) rather than O(N) — an id_to_slot map maintained on every reordering. That is §23 — Index maps. It is one extra np.ndarray, updated whenever the columns move.

Part 2 is closed. Identity is an integer; rows align in lockstep; SoA is the default; the singleton drops out; sort breaks indices and ids fix it. The next phase is Time & passes, starting with §11 — The tick. The ecosystem simulator from code/sim/SPEC.md is about to start running.

Solutions: 10 — Stable IDs and generations

The fix from §9: one extra column.

Exercise 1 — Add the id column

import numpy as np

def new_deck():
    suits     = np.repeat(np.arange(4, dtype=np.uint8), 13)
    ranks     = np.tile(np.arange(13, dtype=np.uint8), 4)
    locations = np.zeros(52, dtype=np.uint8)
    ids       = np.arange(52, dtype=np.uint32)        # the new column
    return suits, ranks, locations, ids

def reorder(suits, ranks, locations, ids, order):
    suits[:]     = suits[order]
    ranks[:]     = ranks[order]
    locations[:] = locations[order]
    ids[:]       = ids[order]                          # one extra line

# verify ids permute, not regenerate
suits, ranks, locations, ids = new_deck()
order = np.argsort(suits, kind="stable")
reorder(suits, ranks, locations, ids, order)
assert sorted(ids.tolist()) == list(range(52))         # same set, just permuted

The id column is just a numpy column. The reorder function gains one line.

Exercise 2 — Find a card by id

def slot_of(ids: np.ndarray, target: int) -> int | None:
    matches = np.where(ids == target)[0]
    return int(matches[0]) if matches.size else None

# after a sort, find the card with id = 17
slot = slot_of(ids, 17)
print(f"id 17 is now at slot {slot}: "
      f"{RANK[ranks[slot]]}{SUIT[suits[slot]]}")

O(N) on each lookup. Fine for 52 cards. For million-row tables, §23 caches the inverse map; that’s an optimisation, not a correction.

Exercise 3 — Resolve the §9 bug

# fresh deck, pre-shuffled so positions are non-trivial
suits, ranks, locations, ids = new_deck()
rng = np.random.default_rng(42)
reorder(suits, ranks, locations, ids, rng.permutation(52))

# Player 1 records IDs [3, 17, 21, 28, 41] — names, not slots
held_ids = [3, 17, 21, 28, 41]
slots = [slot_of(ids, k) for k in held_ids]
locations[slots] = 1
print("before sort:",
      [f"{RANK[ranks[s]]}{SUIT[suits[s]]}" for s in slots])
# ['4♠', '5♥', '9♥', '3♦', '3♣']

# Sort the columns by suit (in lockstep with ids)
reorder(suits, ranks, locations, ids, np.argsort(suits, kind="stable"))

# Look up the same ids — get the new slots — read the cards
slots2 = [slot_of(ids, k) for k in held_ids]
print("after sort: ",
      [f"{RANK[ranks[s]]}{SUIT[suits[s]]}" for s in slots2])
# ['4♠', '5♥', '9♥', '3♦', '3♣']  — same cards!

The slots changed; the cards did not. Player 1’s reference list is in the id domain — names, not addresses — and survives any rearrangement of the columns.

Exercise 4 — Permutation-friendly hand query

def cards_held_by(locations: np.ndarray, ids: np.ndarray, player: int) -> np.ndarray:
    return ids[locations == player]                    # return ids, not slots

# deal, then sort, then re-query — should return the same set
suits, ranks, locations, ids = new_deck()
locations[[0, 1, 2, 3, 4]] = 1
held_before = set(cards_held_by(locations, ids, 1).tolist())

reorder(suits, ranks, locations, ids, np.argsort(suits, kind="stable"))
held_after = set(cards_held_by(locations, ids, 1).tolist())

assert held_before == held_after                       # same five ids, regardless of sort

locations == player is a boolean mask of slots in the player’s hand. Indexing the ids column with that mask returns the names of those cards. The set of names is invariant under reordering of the columns; the set of slots is not.

Exercise 5 — A first generation counter

from typing import NamedTuple

class CardRef(NamedTuple):
    id:  int
    gen: int

suits, ranks, locations, ids = new_deck()
gens = np.zeros(52, dtype=np.uint32)

# Take a reference to the card with id=17 BEFORE we recycle anything
slot = slot_of(ids, 17)
ref  = CardRef(id=17, gen=int(gens[slot]))             # gen=0

# A swap_remove-like operation: pop the card from slot 51, fill slot 17 with a "fresh" card
# (not realistic for a 52-card deck, but mimics the simulator pattern)
suits[17]     = suits[51]                              # recycle: move the last card here
ranks[17]     = ranks[51]
locations[17] = locations[51]
ids[17]       = 52                                     # fresh id (would be next sequence number)
gens[17]     += 1                                      # bump the generation: slot was reused

def deref(ids, gens, ref: CardRef) -> int | None:
    slot = slot_of(ids, ref.id)
    if slot is None:                                   # id no longer in the table
        return None
    if int(gens[slot]) != ref.gen:                     # slot recycled since ref taken
        return None
    return slot

print(deref(ids, gens, ref))                           # None — correctly stale

The (id, gen) pair is the read receipt. After the recycle, slot_of(ids, 17) returns None (id 17 was overwritten with id 52). Even if id 17 had been re-issued — e.g., into slot 17 — the generation bump (0 → 1) would have caught it: the reference’s gen=0 would not match the slot’s gens[slot]=1, and deref would correctly report stale.

This is the generational arena pattern in 30 lines. The same shape carries the simulator’s variable-quantity tables in the rest of the book.

Exercise 6 — A tiny generational arena (stretch)

import numpy as np
from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

class Creatures:
    def __init__(self, capacity: int = 1024):
        self.cap   = capacity
        self.pos   = np.zeros((capacity, 2), dtype=np.float32)
        self.ids   = np.full(capacity, np.iinfo(np.uint32).max, dtype=np.uint32)  # MAX = empty
        self.gens  = np.zeros(capacity, dtype=np.uint32)
        self.free: list[int] = list(range(capacity - 1, -1, -1))                  # stack of free slots
        self.next_id = 0

    def insert(self, x: float, y: float) -> CreatureRef:
        if not self.free:
            raise MemoryError("Creatures table full")
        slot = self.free.pop()
        self.pos[slot, 0] = x
        self.pos[slot, 1] = y
        new_id = self.next_id
        self.next_id += 1
        self.ids[slot] = new_id
        return CreatureRef(id=new_id, gen=int(self.gens[slot]))

    def _slot_of(self, target_id: int) -> int | None:
        m = np.where(self.ids == target_id)[0]
        return int(m[0]) if m.size else None

    def remove(self, ref: CreatureRef) -> bool:
        slot = self._slot_of(ref.id)
        if slot is None or int(self.gens[slot]) != ref.gen:
            return False
        self.ids[slot] = np.iinfo(np.uint32).max        # mark empty
        self.gens[slot] += 1                            # bump generation
        self.free.append(slot)
        return True

    def get(self, ref: CreatureRef) -> tuple[float, float] | None:
        slot = self._slot_of(ref.id)
        if slot is None or int(self.gens[slot]) != ref.gen:
            return None
        return float(self.pos[slot, 0]), float(self.pos[slot, 1])

# Stale-reference test
c = Creatures(capacity=4)
ref_a = c.insert(1.0, 2.0)             # id=0, gen=0
c.remove(ref_a)
ref_b = c.insert(99.0, 99.0)           # id=1, possibly in the same slot, gen=1 there

assert c.get(ref_a) is None             # stale ref correctly rejected
assert c.get(ref_b) == (99.0, 99.0)

A 70-line generational arena. The contract: a CreatureRef is the only valid handle into the table; the table guarantees that a get or remove against a stale ref returns None/False rather than reading or writing the wrong row.

Exercise 7 — The shape of id_to_slot (stretch)

# capacity-bounded inverse map: id → slot, kept in step with the columns
MAX_IDS = 1_000_000
id_to_slot = np.full(MAX_IDS, np.iinfo(np.uint32).max, dtype=np.uint32)

def slot_of_o1(id_to_slot: np.ndarray, target_id: int) -> int | None:
    s = int(id_to_slot[target_id])
    return None if s == np.iinfo(np.uint32).max else s

What the inverse map costs:

• Memory: MAX_IDS × 4 bytes — 4 MB at 1M ids. Constant per id, not per row in the table.
• Update on every reorder: when order is applied to the columns, also rebuild id_to_slot so id_to_slot[ids[i]] = i for every new slot. That’s another loop of length N (one numpy primitive: id_to_slot[ids] = np.arange(N)).

What it buys: O(1) lookups at every dereference. For a simulator that does 100K+ dereferences per tick, this is the difference between a feasible inner loop and a broken one. §23 builds it properly with the lifecycle (ids issued, freed, recycled) handled.

Exercise 8 — Compare with a real ECS handle (stretch)

bevy_ecs::entity::Entity is conceptually two values packed into a u64:

• index: 32-bit slot in the entity table (≈ this chapter’s id field)
• generation: 32-bit reuse counter (≈ this chapter’s gen field)

Mapping:

What bevy adds that you don’t strictly need: packed handle (one u64 vs two integers), explicit Entity::PLACEHOLDER constant, deserialisation tagging, integration with bevy’s reflection/inspector. None of these are required for a working ECS — they’re ergonomics for a public API used by hundreds of downstream crates.

This is the §41 / §42 move. Build the small version yourself first; you now know what Entity does. When you later read bevy’s source you can see what it adds and price each addition against your needs. Most simulators don’t need a packed u64 handle; some do. The cost-benefit is yours, with the from-scratch version in hand.

11 — The tick

Concept node: see the DAG and glossary entry 11.

A program’s life has a shape:

• Start-up — initialisation. Tables are allocated, inputs are opened, the RNG is seeded, the world reaches a known state.
• Steps — ticks of the clock in a simulation, turns in a card game, requests in a server. The repeating unit of forward motion.
• Save and load — the in-memory state is preserved to disk so a future run can resume from where this one left off. Optional, but if you want it, it lives here.
• Exit — resources are returned to the kernel. Memory, file handles, sockets, lockfiles. Failure to do this cleanly is called a memory leak (or a stale lock, or a broken socket).

This section is about the step. The step is where the time budget binds, where the system DAG runs, where determinism either holds or breaks. The other phases are real — the book returns to save and load when persistence is named at §36, and exit is mostly the operating system’s job — but the inner step is what makes or breaks every other property the book builds on.

Each step is a tick. State at the start of a tick is read; state at the end is written; nothing is half-updated mid-tick. Even an interactive program — a card game waiting for the next move, a text editor waiting for a keystroke — is a tick loop, just with an external trigger driving it. A program that does a single pass over a file and exits is a degenerate tick loop with N=1.

Two shapes of tick

A time-driven tick fires at a fixed rate. The simulator from code/sim/SPEC.md runs at 30 Hz: one tick every 33 ms. The loop wakes up, advances every system by one step, sleeps until the next tick. Most simulations, games, control loops, audio engines, and animation systems are time-driven. The rate is a contract with the rest of the world: at this rate, output appears.

A turn-based tick fires when an event arrives. A card game ticks when a player makes a move. A chess engine ticks when its opponent moves. A discrete-event simulator ticks at the timestamp of the next pending event, however far in the future that is. The clock advances with the events, not under them. Turn-based ticks have no fixed rate; their pace is set by the input stream.

Both are ticks. The difference is what triggers the next pass:

# time-driven
import time

TICK_S = 1.0 / 30.0  # 33.3 ms

while running:
    start = time.perf_counter()
    run_all_systems(world)
    elapsed = time.perf_counter() - start
    if elapsed < TICK_S:
        time.sleep(TICK_S - elapsed)

# turn-based
while running:
    event = wait_for_next_event()
    apply_event(world, event)

The §0 simulator runs time-driven. The card game from §5 ran turn-based — every card you dealt was one tick. Both are valid; both fit the same framework.

Not asyncio. Not threads.

Two reflexes the modern Python reader will reach for, and neither is the right tool here.

The asyncio reflex says “control loops are async.” asyncio is a scheduler for I/O-bound work — code that spends most of its time waiting for sockets, files, or sleeps. A simulation tick is CPU-bound: every tick, you have computation to do, and the goal is to do it as fast as possible and then sleep precisely until the next deadline. The asyncio event loop adds dispatch overhead (awaitable wrapping, task stepping, the event loop’s own bookkeeping) without giving you anything in return — you are not waiting on external I/O. A synchronous while True: loop with time.sleep is the correct shape, and it is shorter.

The threading reflex says “use a Timer thread to fire ticks.” This is worse. CPython’s GIL means the timer thread and the main thread cannot run Python code simultaneously; the timer thread firing the tick at 33 ms intervals contends for the same lock the simulation needs. You add scheduler nondeterminism (the OS picks who gets the GIL after each tick interval), you add the GIL-acquisition cost on every wakeup, and you gain nothing — you could have called time.sleep from the main thread directly.

A simulation tick wants three things: precision (sleep until exactly the next deadline), determinism (the same input produces the same output), and simplicity (one place to read to understand the loop). A synchronous loop with time.perf_counter and time.sleep provides all three. The two reflexes above provide none of them. Reach for the simplest tool that gives you the property you actually need.

What fits in a tick

The budget binds the design. From code/measurement/tick_budget.py, one motion system (pos += vel * dt) measured on this machine:

Read the rows. At 100,000 entities, both layouts fit comfortably at 30 Hz, but the dataclass loop already uses 125× more of the budget than the numpy version. At 1,000,000 entities, the dataclass version eats 84% of the 30 Hz budget on one system — the rest of the simulator has 5 ms left for everything else. It does not fit at 60 Hz at all. The numpy version still has 98% of the budget free. At 10,000,000 entities, even the numpy version is at 87% of the 30 Hz budget; the simulation has hit a scale limit on this hardware, and the next move is either reducing the work per element, partitioning the work across processes (§31), or accepting a slower tick rate.

The dataclass version at 10,000,000 was skipped because it would extrapolate to ~280 ms per tick — eight ticks of 30 Hz budget — for one system, before any other work. The right reading of that gap is not “numpy is fast” but “an interpreter-bound inner loop puts a hard ceiling on the population your tick can sustain, and the ceiling is much lower than most readers expect.”

The budget is also where mixing turn-based and time-driven thinking in the same loop produces drift: the turn-based subsystem’s pace bleeds into the time-driven subsystem’s budget. The fix is to keep the two cleanly separated — typically one outer loop and the other as an event source feeding it.

A tick is the unit of forward motion in any program that has forward motion. The next sections name what fits in one tick, in what order, and what does not.

Exercises

You will need a fresh project for these. mkdir tick_lab && cd tick_lab && uv init is enough.

1. A 30 Hz time-driven loop. Write a main() that loops at 30 Hz. Each iteration, print the elapsed time since program start. Sleep between ticks to maintain the rate. Run it for 10 seconds. Did you actually get 300 iterations? Use time.perf_counter() — time.time() can go backwards on clock corrections.
2. The naive sleep mistake. Replace your sleep logic with time.sleep(1/30) (no measurement of work time). Run for 30 seconds. Does the program drift over time? Why? (Hint: each iteration’s work + sleep is now 33 ms + work_ms, not 33 ms total.)
3. Dropped frames. Inside the loop, sleep for 50 ms — longer than the budget. The loop is now running at 20 Hz; it has missed frames. Print a warning when this happens. The right way to detect: if elapsed > TICK_S: print(f"missed deadline by {elapsed - TICK_S:.3f} s").
4. A turn-based loop. Write a tiny REPL: print > , read a line with input(), print you said: <line>. Each line is one tick. Run it. Note that the loop has no fixed rate — its pace is your typing.
5. Run the tick-budget exhibit. uv run code/measurement/tick_budget.py. Note the row where the dataclass version stops fitting at 60 Hz. Note the row where it stops fitting at 30 Hz. Note that the numpy version is still fine at both N values. The book is asking you to keep the numpy line running for the next thirty chapters.
6. The asyncio comparison. Rewrite exercise 1 using asyncio.run and await asyncio.sleep. Measure: does it tick at the same rate? Does the program use more memory? More wall time per tick? Compare your two implementations side by side. Most readers will find the asyncio version harder to read and not measurably faster — exactly the calibration the prose above predicts.
7. (stretch) A discrete-event tick loop. Maintain a list of (timestamp, message) events sorted by timestamp. Pop the smallest-timestamp event, advance a “simulation clock” to that timestamp, print the message, repeat until the queue is empty. This is the structure of a discrete-event simulator and a preview of §12. Use heapq for the priority queue.

Reference notes in 11_the_tick_solutions.md.

What’s next

Exercise 7 hints at the next section. The clock can live on the events themselves, independent of how often the loop fires. §12 — Event time vs tick time names that separation.

Solutions: 11 — The tick

Exercise 1 — A 30 Hz time-driven loop

import time

TICK_S = 1.0 / 30.0
start = time.perf_counter()
end   = start + 10.0
ticks = 0
while time.perf_counter() < end:
    t0 = time.perf_counter()
    ticks += 1
    print(f"t={t0 - start:6.3f}s  tick={ticks}")
    elapsed = time.perf_counter() - t0
    if elapsed < TICK_S:
        time.sleep(TICK_S - elapsed)

print(f"{ticks} ticks in {time.perf_counter()-start:.2f}s")

Expected: 300 ticks ± 1 in 10 seconds. The loop sleeps for TICK_S - work_done, so each iteration ends exactly TICK_S after it began (modulo OS scheduling). time.perf_counter() is monotonic; time.time() can step backwards on NTP corrections and is the wrong tool here.

Exercise 2 — The naive sleep mistake

while True:
    do_some_work()
    time.sleep(1/30)            # always 33 ms, regardless of work time

Each iteration takes work_ms + 33 ms, not 33 ms total. If the work consistently takes 5 ms, the loop ticks at 1 / (0.005 + 0.033) ≈ 26.3 Hz, not 30. Over a minute that is 1,580 ticks instead of 1,800 — a 12% deficit, and the program reports “running at 30 Hz” because that’s what it asked for.

The drift is silent: nothing in the program complains. Only an external observer (the wall clock, an event log, an animation that runs slow) notices. The fix is to measure work time and subtract, as in exercise 1.

Exercise 3 — Dropped frames

while running:
    t0 = time.perf_counter()
    do_some_work()                    # may take longer than TICK_S
    elapsed = time.perf_counter() - t0
    if elapsed > TICK_S:
        print(f"missed deadline by {(elapsed - TICK_S)*1000:.1f} ms")
    else:
        time.sleep(TICK_S - elapsed)

If do_some_work() sleeps 50 ms (longer than the 33 ms budget), the loop runs at 20 Hz and prints missed deadline by 16.7 ms every iteration. Detecting missed deadlines is half the battle; responding to them is the rest. The simplest response is “log it and continue”; smarter responses (skip a frame’s interpolation, drop secondary work, lower the visible LOD) live at the application layer.

A simulator that has missed its tick budget is a simulator running on the wrong hardware or with the wrong N. Naming the deadline-miss is how you know.

Exercise 4 — A turn-based loop

while running:
    line = input("> ")
    if line.strip() in {"quit", "exit"}: break
    print(f"you said: {line}")

Each input() blocks until a line arrives. The loop has no fixed rate — its pace is whatever the typist provides. The same shape carries a chess engine (one tick per move), a card game (one tick per play), a discrete-event simulator (one tick per event timestamp). The trigger is “a thing happened”, not “33 ms passed”.

Exercise 5 — Run the tick-budget exhibit

uv run code/measurement/tick_budget.py

Source: code/measurement/tick_budget.py.

          N  layout                  tick (ms)   30 Hz            60 Hz
--------------------------------------------------------------------------------
     10,000  numpy SoA                   0.005   fit  (  0.0%)    fit  (  0.0%)
     10,000  Python dataclass list       0.272   fit  (  0.8%)    fit  (  1.6%)
    100,000  numpy SoA                   0.019   fit  (  0.1%)    fit  (  0.1%)
    100,000  Python dataclass list       2.750   fit  (  8.3%)    fit  ( 16.5%)
  1,000,000  numpy SoA                   0.278   fit  (  0.8%)    fit  (  1.7%)
  1,000,000  Python dataclass list      27.525   fit  ( 82.6%)    OVER (  165%)
 10,000,000  numpy SoA                  16.609   fit  ( 49.8%)    fit  ( 99.7%)
 10,000,000  Python dataclass list   (skipped)   extrapolates over   extrapolates over

The 60 Hz line on 1M dataclass: 165% over budget. The 30 Hz line on 1M dataclass: 82.6% used by one motion system, leaving 5.7 ms for everything else the simulator needs to do per tick. The book is asking you to keep the numpy line because that is the population at which Python becomes feasible. Below 100K entities the layout choice doesn’t matter much; above 100K it determines whether the simulator runs at all.

Exercise 6 — The asyncio comparison

import asyncio, time

TICK_S = 1.0 / 30.0

async def loop():
    start = time.perf_counter()
    end   = start + 10.0
    while time.perf_counter() < end:
        t0 = time.perf_counter()
        # do_work()
        elapsed = time.perf_counter() - t0
        if elapsed < TICK_S:
            await asyncio.sleep(TICK_S - elapsed)

asyncio.run(loop())

Tick rate: same as the synchronous version (~30 Hz). Memory: ~1-2 MB more, for the event loop, the task object, and the awaitable infrastructure. Wall time per tick: 5-20 µs higher because each iteration goes through the event loop’s task-stepping machinery to schedule the next wakeup.

What you got for the cost: nothing. The work is CPU-bound; there are no other awaitables to interleave; the event loop has no useful work to do during the sleep. await asyncio.sleep becomes a slightly more expensive time.sleep. The asyncio scheduler is the right shape for I/O-bound programs (web servers, network clients) where a single thread juggles many waiting connections; it is the wrong shape for a CPU-bound tick loop.

The lesson is the chapter’s: reach for the simplest tool that gives you the property you actually need. Asyncio is correct for many programs. This is not one of them.

Exercise 7 — A discrete-event tick loop (stretch)

import heapq

events: list[tuple[float, str]] = []
heapq.heappush(events, (1.0, "creature_birth"))
heapq.heappush(events, (0.5, "food_spawn"))
heapq.heappush(events, (2.5, "starvation_check"))
heapq.heappush(events, (1.5, "creature_birth"))

clock = 0.0
while events:
    t, msg = heapq.heappop(events)
    clock = t                                   # advance to the event's timestamp
    print(f"t={clock:.2f}  event: {msg}")

t=0.50  event: food_spawn
t=1.00  event: creature_birth
t=1.50  event: creature_birth
t=2.50  event: starvation_check

Two properties to notice:

• The clock advances with the events, not in fixed steps. There is no “tick” between t=0.5 and t=1.0; the simulation simply jumps. Long quiet periods cost nothing.
• No external time reference is needed. Everything is internal — events have timestamps, the clock follows them. This is the discrete-event-simulation (DES) shape that production tools (SimPy, NS-3, OMNeT++, MATLAB Simulink) build on.

§12 — Event time vs tick time names this distinction: the clock the simulation uses doesn’t have to be the clock the loop uses. A 30 Hz time-driven loop with a discrete-event subsystem inside it is a common shape — the outer loop advances the world by 33 ms; the inner DES processes all events with timestamps in the next 33 ms.

12 — Event time is separate from tick time

Concept node: see the DAG and glossary entry 12.

Most beginners assume the loop’s frequency sets the model’s time resolution. If the loop runs at 30 Hz, surely the model can only resolve events at 1/30 s = 33 ms? This is wrong, and the confusion costs many simulations their precision.

The tick rate is how often the loop runs. It says nothing about what the loop does inside one tick. Inside one tick, the loop can process events at arbitrary timestamps — microsecond, picosecond, whatever the data carries. The clock lives on the events, not on the loop.

Concretely: a 30 Hz loop receiving 1,000 events per tick, each with microsecond-precision timestamps, processes them in timestamp order — applying each event’s effect with the precision the timestamp implies. Output to the rest of the world (rendering, logging, network) happens at 30 Hz, but the physics inside runs at microsecond resolution. The tick is a sampling rate; the events are the actual phenomena.

This is the model used by:

• Discrete-event simulators (queueing networks, traffic, supply chains): events fired at exact times.
• Game replay systems (rollback netcode, multiplayer): events arrive late but with their original timestamps.
• Trade execution engines: orders carry nanosecond timestamps; the loop processes them in order.
• Logic simulators in chip design: gate transitions at picosecond resolution; the simulator advances one transition at a time.

In each case, the tick rate of the host loop is irrelevant to the simulation’s resolution. The data carries the time.

How time wants to be stored

The Python reflex when a chapter mentions “timestamps” is to reach for datetime. It is the obvious choice — the standard library provides it, every tutorial uses it, comparisons work with < and >, subtractions return a readable timedelta. It is also one of the most expensive ways to store time at scale.

From code/measurement/event_time_storage.py, one million events covering an hour at microsecond resolution, on this machine:

The headline numbers, both ways:

• 7× smaller footprint moving from datetime list to either typed numpy column. Each datetime instance is ~56 bytes (header, refcount, eight integer fields, pointer); each numpy element is 8 bytes (an int64 micro-since-epoch under datetime64[us], or a float64 second-from-base for the f8 representation).
• 17× faster count of “how many events happened before time T?” — the per-tick query that decides what gets processed this tick. The numpy versions evaluate the comparison as one bandwidth-bound bulk op; the datetime version pays per-element interpreter dispatch and a < method call.
• Sort time is mixed and dtype-sensitive — measure your specific case. On this run numpy’s float64 sort was slower than its datetime64 sort, which was slightly faster than Python’s Timsort on the already-sorted datetime list. Sort cost matters for ingestion; count cost matters per tick. The tick is the binding budget.

The simlog reference implementation (vendored at .archive/simlog/logger.py) stores time as f8 — float64 seconds. That is the disciplined choice for an event log: small, sortable, amenable to bulk numpy ops, and the same width as everything else in the column store. datetime64[us] is a reasonable alternative when you need to read the timestamps as wall-clock dates without conversion. Use datetime objects only at the boundary — formatting a string for a log line, comparing against a user-supplied timestamp from a request — never as your in-memory storage at simulation scale.

The decoupling, in code

The pitfall is hard-coding the tick interval as the simulation’s clock granularity. Code that says

# anti-pattern: bad!
creature.energy -= 1.0 / 30.0  # "one tick worth of fuel"

is conflating the two clocks. The right shape is

energy[mask] -= elapsed_event_seconds * burn_rate[mask]

using the actual elapsed event-time, not the tick interval. The numpy form is also column-shaped — mask is a boolean filter selecting the affected creatures, burn_rate is per-creature. The same computation works for one event affecting one creature and a thousand events affecting a thousand creatures, because event time and tick time are decoupled. The same model can be sampled at any tick rate the application needs — visualisation at 30 Hz, recording at 60 Hz, fast-forward replay at 1 kHz — without changing what the model means.

This separation is what makes the simulator’s pending_event table possible. Each tick, the loop builds a list of events that should fire — collisions, eats, reproductions — each tagged with its predicted timestamp as an f8. The events fire in timestamp order regardless of which tick they were predicted in. A creature that “would have eaten 2 µs into the tick” has its eat applied at that exact moment, not at the start or end of the tick.

Exercises

These extend the discrete-event loop from §11 exercise 7.

1. A tiny event queue. Use numpy arrays: times = np.array([...], dtype=np.float64) of timestamps and messages = np.array([...], dtype=object) of strings. Push 10 events with random timestamps in [0, 10] seconds. Pop them in time order using order = np.argsort(times). Print each as [t=<sec>] <message>. Verify the output is timestamp-sorted.
2. The wrong way: tick-rate clock. Run a 30 Hz loop. In each tick, advance a counter by 1.0 / 30.0. Use this counter as your “simulation time”. Try to fire an event at t = 0.005 s (5 ms). What happens? When does the event fire? (Hint: 5 ms < 33 ms; the event waits for the next tick boundary, losing 28 ms of resolution.)
3. The right way: timestamp on events. Run the same 30 Hz loop, but each tick pop all events with timestamp ≤ current real time, applied in timestamp order. Fire an event at t = 0.005 s. Show that the event applies at exactly that time, not at the next tick boundary.
4. Sampling at different rates. Run the same model under a 30 Hz loop, then a 60 Hz loop, then a 1 Hz loop. The events should fire at the same simulation times in all three runs (down to whatever precision the loop allows).
5. Float and time. What is the smallest time step np.float32 can represent for events at t ≈ 1 hour? At t ≈ 1 day? At t ≈ 1 year? When do you need np.float64? (See §2. Hint: np.spacing(np.float32(3600)) is a fast way to find the answer for one hour.)
6. Run the storage exhibit. uv run code/measurement/event_time_storage.py. Note the count-time row — that is the per-tick query cost in three layouts. Note where the datetime list lands and where the numpy columns land.
7. (stretch) A budget-aware loop. Modify your 30 Hz loop: at the start of each tick, pop events until either (a) the queue is empty or (b) you have used 25 ms of the 33 ms budget. Defer remaining events to the next tick. This is the soft-real-time pattern used in interactive simulators.

Reference notes in 12_event_time_vs_tick_time_solutions.md.

What’s next

§13 — A system is a function over tables introduces the building block of every tick: the system. Read-set in, write-set out, no hidden state, no surprises.

Solutions: 12 — Event time is separate from tick time

Exercise 1 — A tiny event queue

import numpy as np

rng = np.random.default_rng(0)
times    = rng.uniform(0.0, 10.0, size=10).astype(np.float64)
messages = np.array([f"event_{i}" for i in range(10)], dtype=object)

order = np.argsort(times)
for t, m in zip(times[order], messages[order]):
    print(f"[t={t:.3f}] {m}")

The order array is the only thing that’s “sorted.” times and messages are unchanged. The decoupling pattern from §5: data lives in columns; iteration goes through an index array.

Exercise 2 — The wrong way: tick-rate clock

import time
TICK_S = 1.0 / 30.0

# anti-pattern: bad! "simulation time" advances in tick-sized steps
sim_time = 0.0
target   = 0.005       # 5 ms event
fired    = False

while sim_time < 1.0 and not fired:
    if sim_time >= target:
        print(f"event fired at sim_time={sim_time:.4f}")
        fired = True
    sim_time += TICK_S                  # 33 ms granularity
    time.sleep(TICK_S)

Output: event fired at sim_time=0.0333 — 28 ms late. The event’s “true time” was 5 ms; the sim_time clock cannot resolve below 33 ms because that is the step size. Every event between two tick boundaries gets snapped to the next boundary, losing precision proportional to the tick rate.

This is the conflation the chapter warns against. The 30 Hz tick rate is how often the loop wakes up; it is not the resolution of the model. Hard-coding 1.0/30.0 as the simulation’s time delta makes them the same thing — and pins the simulation’s accuracy to the loop’s wake-up rate.

Exercise 3 — The right way: timestamp on events

import time, heapq, numpy as np

events: list[tuple[float, str]] = []
heapq.heappush(events, (0.005, "early_event"))
heapq.heappush(events, (0.040, "second_event"))
heapq.heappush(events, (0.080, "third_event"))

start = time.perf_counter()
TICK_S = 1.0 / 30.0

while events:
    now = time.perf_counter() - start
    while events and events[0][0] <= now:
        t, msg = heapq.heappop(events)
        print(f"[real={now:.4f}, sim={t:.4f}] {msg}")
    time.sleep(TICK_S)

Each tick processes all events whose timestamp has passed. The 5 ms event fires inside the first tick (the loop has been running for >5 ms by the time the tick finishes). The event’s t is preserved — print(...) shows the original 0.005, not the snapped tick boundary.

The simulator processes the event with its own time, not the loop’s. Same model, sub-tick precision, no overhead beyond a heap pop per event.

Exercise 4 — Sampling at different rates

def run(tick_hz, events_in):
    import time, heapq
    events = list(events_in)
    heapq.heapify(events)
    start = time.perf_counter()
    tick_s = 1.0 / tick_hz
    fired_times = []
    while events:
        now = time.perf_counter() - start
        while events and events[0][0] <= now:
            t, msg = heapq.heappop(events)
            fired_times.append(t)
        time.sleep(tick_s)
    return fired_times

events_in = [(0.005, "a"), (0.040, "b"), (0.080, "c"), (0.150, "d")]
for hz in (30, 60, 1):
    print(f"{hz:>3} Hz fires at: {run(hz, events_in)}")

The list of fired event times is the same in all three runs (modulo floating-point comparison): [0.005, 0.040, 0.080, 0.150]. The 30 Hz, 60 Hz, and 1 Hz runs differ only in how often the loop checked — they all see and apply the same set of events at the same simulation timestamps. The model is sample-rate-independent.

Exercise 5 — Float and time

import numpy as np
print(np.spacing(np.float32(3600)))          # ~2.4e-04 = 244 µs
print(np.spacing(np.float32(86400)))         # ~7.8e-03 = 7.8 ms
print(np.spacing(np.float32(31_536_000)))    # 2.0 s
print(np.spacing(np.float64(31_536_000)))    # ~3.7e-09 = 3.7 ns

float32 runs out of precision fast once the absolute time grows. A simulation that runs for more than a day at sub-millisecond resolution needs float64. This is the §2 catastrophic-cancellation lesson re-applied: precision is a function of the magnitude of the values you’re representing, not just the size of the differences you care about.

Exercise 6 — Run the storage exhibit

uv run code/measurement/event_time_storage.py

Source: code/measurement/event_time_storage.py.

layout                                          data (MB)   build (ms)   sort (ms)   count <T (ms)
---------------------------------------------------------------------------------------------------
list of datetime objects                          53.62       387.6        5.71       19.980
numpy datetime64[us]                               7.63        92.3        6.12        1.198
numpy float64 (seconds-from-base)                  7.63        44.6       42.53        0.894

vs 'list of datetime objects':
  numpy datetime64[us]                       7.0× smaller    0.9× faster sort    16.7× faster count
  numpy float64 (seconds-from-base)          7.0× smaller    0.1× faster sort    22.3× faster count

The per-tick query is count <T: 22× faster on float64 vs the datetime list. That is the column the simulator hits every tick to decide what events fire. Sort cost is one-off (ingestion); count cost compounds across millions of ticks. The tick is the binding budget, so the count column is the one to optimise.

Exercise 7 — A budget-aware loop (stretch)

import time, heapq

TICK_S       = 1.0 / 30.0
SOFT_BUDGET  = 0.025                              # 25 ms of the 33 ms tick

events: list[tuple[float, str]] = [...]           # populated from outside
heapq.heapify(events)

while True:
    tick_start = time.perf_counter()
    deadline   = tick_start + SOFT_BUDGET

    processed = 0
    while events and events[0][0] <= time.perf_counter() - tick_start:
        if time.perf_counter() > deadline:
            break                                 # over budget — defer the rest
        t, msg = heapq.heappop(events)
        apply_event(msg)
        processed += 1

    elapsed = time.perf_counter() - tick_start
    if events and elapsed > SOFT_BUDGET:
        print(f"deferred {len(events)} events; tick used {elapsed*1000:.1f}ms")

    sleep_for = TICK_S - elapsed
    if sleep_for > 0:
        time.sleep(sleep_for)

This is the soft real-time pattern: the loop prefers to process every due event each tick, but guarantees it will return within budget. Surplus events spill into the next tick.

This shape is what runs game engines, animation systems, and interactive simulators. It is also what the simulator’s §35 — boundary is the queue builds on — events at the edge of the tick belong to the next tick’s queue, not this one’s stretch goal.

The pattern fails gracefully when overloaded: latency degrades but the loop continues. The alternative — process every event whatever it costs — fails catastrophically when overloaded: the loop blows its tick budget, drops the next deadline, and either loses real-time properties or piles up an ever-growing deficit.

13 — A system is a function over tables

Concept node: see the DAG and glossary entry 13.

A system is a function that reads from one or more tables and writes to one or more tables. It declares its inputs (the read-set) and its outputs (the write-set). It has no hidden state, no global side effects, no interaction with the outside world during a tick. The signature is the contract.

def motion(pos_x: np.ndarray, pos_y: np.ndarray,
           vel_x: np.ndarray, vel_y: np.ndarray,
           dt: float) -> None:
    pos_x += vel_x * dt
    pos_y += vel_y * dt

Read-set: vel_x, vel_y, dt. Write-set: pos_x, pos_y. That is the entire contract. This system can run any time those four columns and dt are available and nothing else is writing pos_x or pos_y. It runs once per tick over the whole population — there is no per-creature loop in the body. The for-loop disappeared into numpy.

Three shapes

Every system takes one of three shapes.

An operation is 1→1: every input row produces exactly one output row. motion is an operation — each creature’s position is updated to its new position. Most update functions are operations.

A filter is 1→{0, 1}: every input row produces zero or one output rows. apply_starve (from code/sim/SPEC.md) is a filter — each creature with energy ≤ 0 produces an entry in to_remove; creatures with energy > 0 produce nothing. The numpy form is one line:

def starving(energy: np.ndarray) -> np.ndarray:
    return np.where(energy <= 0)[0]  # returns the indices to remove

An emission is 1→N: every input row produces zero or more output rows. apply_reproduce is an emission — a parent above the energy threshold produces two offspring (a 1→2 emission).

These three shapes are the same shapes a database query takes. SELECT * FROM t WHERE p is a filter, SELECT a + b FROM t is an operation, SELECT explode(arr) FROM t is an emission. A system is a database operation written in Python against numpy columns instead of SQL against tables. If you have ever written SQL, you already know the vocabulary; the work is recognising your simulation in those terms.

Return type is half the contract

The three shapes also fix the return type. An operation mutates its write-set in place and returns None — the work has already happened by the time the call returns. A filter returns a new array of indices. An emission returns one or more new arrays. The pattern is: mutators return None; producers return the thing they produced.

The reason for the asymmetry is that the alternative — having a mutator return its own write-set — is a silent aliasing bug. world = step(world) reads like it produces a new world, but if step mutates and returns the same object, both names point at the same state and the caller cannot tell from the call site which is true. Python’s standard library encodes the rule exactly: list.sort() returns None so that xs = xs.sort() fails loudly; sorted() returns a new list. The system convention is the same rule applied to columns.

There is one named exception: a function that builds a world from nothing — from a seed, a file, or a log — returns the new world. Its signature gives it away: it does not take an existing world to mutate. build_world(seed), load(path), replay(initial, log) are constructors, not systems. The return value is the only place the new state can go.

The OOP method is the anti-shape

This is the moment to name what most Python tutorials teach instead. The method-on-object shape — class Creature: def tick(self, dt): self.pos += self.vel * dt — is the same lesson rotated through self, and the rotation costs you everything important. The signature def tick(self, dt) does not tell you what the method reads or writes. The body does, but only after you read it. The contract is no longer expressible at the call site; it is implicit in the body of the method, which means you cannot reason about composition without inlining every method.

It also costs you the loop. The natural caller for Creature.tick is for c in creatures: c.tick(dt) — a Python-level loop, one method dispatch per element, interpreter-bound at the floor of ~5 ns per element from §1, plus another ~50-100 ns of getattr and method-call overhead per attribute. From code/measurement/tick_budget.py the cost is 27.9 ms per tick at 1,000,000 creatures for one motion system, against 0.6 ms for the function-over-columns form. The system shape is not just clearer — it is the only one that fits inside a 30 Hz budget at scale.

The wider rule: a function that takes self does not have a declared read-set or write-set. A function that takes columns does. This is one of the two or three places where “OOP versus data-oriented” is not a stylistic choice — it is whether your system has a contract you can read.

Logging is a separate system

The other reflex Python encourages is to write to stdout from inside the loop. print(f"creature {i} starved"), logger.info(...), traceback.print_exc() — all of these are side effects that violate the system’s no-hidden-output contract. The fix is the same shape as everything else in this book: there is a log_events table, a logging system writes to it, and a separate flush system writes the table to disk or stdout.

The book builds this discipline at §37 — The log is the world. For now, the rule is: if a system needs to communicate with the outside, it does so through a column declared in its write-set. There are no surprise prints.

Observability and tests are systems too

A debug inspector is a system whose read-set is “all relevant columns” and whose write-set is “nothing observable” — it gathers data for inspection and produces no side effects on the world. In production it is absent, not gated by a flag — the program simply does not contain it.

A test is also a system. assert pos.shape == vel.shape and not np.any(np.isnan(pos)) is a system whose read-set is pos and vel, write-set is nothing, and whose effect is to fail loudly if the contract of the previous system was violated. Tests-as-systems is the §43 topic, but you have been writing them since §5 exercise 1.

A system declares its inputs, declares its outputs, and does no more. That is the shape that lets every other discipline in the book work.

A few patterns to watch for

A function that reads a column, writes to it, and reads it again in the same call is not a system — it has implicit ordering inside the body. Either split it into two systems with explicit ordering, or buffer the writes until the function exits. A function that takes a world object and mutates whatever it likes is not a system — it has no declared write-set, and you cannot reason about it from its signature.

The contract that the system has no hidden state is what makes systems compose. Two systems with disjoint write-sets can run in parallel without coordination (§31). Two systems whose read-set and write-set form a chain must run in order (§14). The contract is the basis for all of this.

Exercises

Use the deck from §5, your tick_lab from §11, or the §0 simulator skeleton; any of them provides enough tables.

1. Identify the shape. Classify each as operation, filter, or emission:
   • Squaring every entry in a np.ndarray of float32.
   • Filtering even integers from a np.ndarray of int32.
   • Splitting each string in a list[str] into words, returning all words.
   • Computing the sum of a np.ndarray of int32.
2. Write motion as a system. With pos_x, pos_y, vel_x, vel_y as numpy float32 columns of length 100, write motion(pos_x, pos_y, vel_x, vel_y, dt) as defined in the prose. Apply it to 100 creatures with random initial positions and velocities. Print the position of one creature across 10 ticks. The body is two lines.
3. Declare the contract. Add a docstring to motion listing its read-set and write-set explicitly. The signature plus the docstring is the system’s contract.
4. Write a filter. With energy: np.ndarray, write starving(energy) returning a numpy array of indices where energy[i] <= 0. This is the read-only first half of apply_starve.
5. Write an emission. With parent_energy: np.ndarray, threshold threshold: float, write reproduce(parent_energy, threshold) returning two parallel arrays — parent_indices and offspring_energies — for each parent above threshold, with two entries each. This is a 1→2 emission. (Hint: mask = parent_energy > threshold; idx = np.where(mask)[0]; np.repeat(idx, 2).)
6. Observe non-systems. Find a function in your previous work (or any Python tutorial) that takes self and mutates whatever it likes, or writes to a global, or calls print from inside the body. Note what makes it not a system. Try to express its read-set and write-set from the signature alone — confirm you cannot.
7. The OOP cost in your fingers. Run uv run code/measurement/tick_budget.py. Read the table. Note that you have just seen, at 1,000,000 creatures, what happens when the loop is in the body of a method instead of in numpy. The 30 Hz row is over for the Python dataclass version. The system-shaped version uses 1.8% of the budget.
8. (stretch) A test as a system. Write def no_creature_moved_too_far(prev_pos_x, prev_pos_y, cur_pos_x, cur_pos_y, max_step) returning indices where any creature moved further than max_step between two ticks. The “test” is just an inspection system reading the world. Hint: dx = cur_pos_x - prev_pos_x; dy = cur_pos_y - prev_pos_y; np.where(dx*dx + dy*dy > max_step*max_step)[0].

Reference notes in 13_system_as_function_solutions.md.

What’s next

§14 — Systems compose into a DAG takes the next step: when many systems run together, how do they fit?

Solutions: 13 — A system is a function over tables

Exercise 1 — Identify the shape

Reductions deserve a footnote: they collapse a column into a scalar. They are systems too — read-set is the column, write-set is one scalar. The book mostly uses reductions inline (sum, max, count_nonzero) rather than as named systems, but the contract still applies.

Exercise 2 — Write motion as a system

import numpy as np

def motion(pos_x, pos_y, vel_x, vel_y, dt):
    pos_x += vel_x * dt
    pos_y += vel_y * dt

rng = np.random.default_rng(0)
n = 100
pos_x = rng.uniform(0, 10, n).astype(np.float32)
pos_y = rng.uniform(0, 10, n).astype(np.float32)
vel_x = rng.uniform(-1, 1, n).astype(np.float32)
vel_y = rng.uniform(-1, 1, n).astype(np.float32)

dt = 1.0 / 30.0
for t in range(10):
    print(f"t={t}: creature 17 at ({pos_x[17]:.3f}, {pos_y[17]:.3f})")
    motion(pos_x, pos_y, vel_x, vel_y, dt)

Two lines in the body. No per-creature loop, no method dispatch, no self. The += operator on a numpy column is a single C-level pass.

Exercise 3 — Declare the contract

def motion(pos_x, pos_y, vel_x, vel_y, dt):
    """Advance every creature's position by one tick of motion.

    Read-set:  vel_x, vel_y, dt
    Write-set: pos_x, pos_y     (in-place)
    Contract:  pos_*.shape == vel_*.shape; arrays are float32 columns.
    """
    pos_x += vel_x * dt
    pos_y += vel_y * dt

The signature plus the docstring is the entire contract. A reader of motion does not need to inline the body to know it does not touch energy or birth_t; the docstring says so. The §14 DAG construction reads exactly this declaration to schedule the system.

A test that the contract is honest (a §43 test-as-system) compares the declared write-set to the columns the function actually mutated. If motion ever silently writes to energy, the test catches it.

Exercise 4 — Write a filter

def starving(energy: np.ndarray) -> np.ndarray:
    """Return indices of creatures with energy <= 0.

    Read-set: energy
    Write-set: nothing  (returns indices for a separate apply step)
    """
    return np.where(energy <= 0)[0]

energy = np.array([3.0, -1.0, 5.0, 0.0, 7.0], dtype=np.float32)
print(starving(energy))     # [1 3]

The filter is read-only. It returns the indices that satisfy the predicate; a separate “apply” system writes them into to_remove. This separation is the §22 mutations buffer discipline applied at the smallest scale: filter and apply are separate systems with different read-sets and write-sets.

Exercise 5 — Write an emission

def reproduce(parent_energy: np.ndarray, threshold: float):
    """For each parent above threshold, produce two offspring (1→2 emission).

    Read-set: parent_energy, threshold
    Write-set: nothing  (returns parallel arrays for the apply step)
    Returns:
        parent_indices: which parent each offspring came from (length 2*K)
        offspring_energies: starting energy for each offspring (length 2*K)
    """
    mask = parent_energy > threshold
    idx  = np.where(mask)[0]
    parent_indices    = np.repeat(idx, 2)                    # parent appears twice
    offspring_energies = np.repeat(parent_energy[idx] / 2, 2) # half-energy each
    return parent_indices, offspring_energies

energies = np.array([3.0, 7.0, 1.0, 9.0, 5.0, 11.0], dtype=np.float32)
p, o = reproduce(energies, 5.0)
print(f"parents:    {p}")              # [1 1 3 3 5 5]
print(f"offspring:  {o}")              # [3.5 3.5 4.5 4.5 5.5 5.5]

np.repeat(arr, 2) is the emission primitive: each input row produces two output rows in column form. For a 1→N emission with variable N per row, np.repeat(arr, counts) takes a per-row count array. The shape is “filter, then expand”; the apply system later inserts the rows into the table.

Exercise 6 — Observe non-systems

A canonical non-system from the wild:

class GameObject:
    def update(self):
        self.pos += self.vel * GLOBAL_DT
        if self.energy <= 0:
            print(f"{self.name} died")          # side effect — not in any signature
            self.dead = True
            World.remove(self)                  # mutates global state
        for nearby in World.find_nearby(self):  # reads global state
            self.energy += nearby.value

What the signature def update(self) declares: nothing. What the body actually does:

• Reads self.pos, self.vel, self.energy, self.name, World.objects (implicit, through find_nearby)
• Reads global GLOBAL_DT
• Writes self.pos, self.dead, self.energy, World.objects (implicit, through World.remove)
• Writes stdout

You cannot tell any of this from the signature. To compose update with another system, you’d have to inline the body and trace every method call. Two update calls cannot run in parallel because both write World.objects. Tests cannot mock the read-set without mocking the world. The function has no contract anyone can read; it has behaviour, which is not the same thing.

Exercise 7 — The OOP cost in your fingers

uv run code/measurement/tick_budget.py

The 1M-creatures row:

The dataclass form has one motion system eating 82.6% of the 30 Hz budget; the simulator has 5.7 ms left for everything else (collision, energy, reproduction, rendering). At 60 Hz the loop has already missed its deadline. The system-as-function-over-numpy form runs the same logic in 0.278 ms and leaves 32.7 ms (98%) of the 30 Hz budget for the rest of the simulator.

The 100× cost gap is the cost of putting the per-creature loop inside the interpreter instead of inside numpy. There is no syntactic refactor of the OOP version that closes this gap — the cost is structural.

Exercise 8 — A test as a system (stretch)

def no_creature_moved_too_far(prev_pos_x, prev_pos_y,
                              cur_pos_x, cur_pos_y, max_step) -> np.ndarray:
    """Return indices of creatures that moved further than max_step between ticks.

    Read-set:  prev_pos_*, cur_pos_*, max_step
    Write-set: nothing  (the caller decides whether to assert, log, or correct)
    """
    dx = cur_pos_x - prev_pos_x
    dy = cur_pos_y - prev_pos_y
    return np.where(dx * dx + dy * dy > max_step * max_step)[0]


# Used as an assertion in a tick:
violators = no_creature_moved_too_far(prev_x, prev_y, x, y, 1.0)
assert violators.size == 0, f"creatures {violators} teleported"

This is a system by the chapter’s definition: declared read-set, no write-set, no hidden state. Its presence in the program is what an invariant looks like — the rest of the program is required to keep no_creature_moved_too_far returning an empty array. Failing this test is the simulator telling you the motion system has a bug.

The Rust edition would write this as a fn taking slices; the only difference is that Python tests run inside the same loop while Rust tests usually run as #[cfg(test)] builds. The discipline is identical — the test is a system over the same tables, with the same contract shape, that happens to report rather than transform. §43 — tests are systems generalises this.

14 — Systems compose into a DAG

Concept node: see the DAG and glossary entry 14.

A program with one system is uninteresting; a program with many systems must say what runs in what order. The order is given by data dependencies: a system that reads a table must run after every system that writes that table within the same tick. No ordering is fixed by intuition; everything is given by the read-sets and write-sets §13 just made you declare.

Draw the dependency graph. Each system is a node. For every system that reads table T and every system that writes T, draw an edge writer → reader. The result is a directed acyclic graph — the DAG. A topological sort gives a valid execution order: any sort that respects the edges is correct. The program executes one such sort.

The simulator’s tick from code/sim/SPEC.md:

flowchart TB
    food_spawn --> motion
    motion --> next_event
    next_event --> apply_eat
    next_event --> apply_reproduce
    next_event --> apply_starve
    apply_eat --> cleanup
    apply_reproduce --> cleanup
    apply_starve --> cleanup
    cleanup --> inspect

food_spawn runs first because its output is food, which motion and next_event read. next_event produces pending_event, which the three appliers consume in parallel (their write-sets are disjoint). cleanup runs after all of them because its read-set includes their writes. inspect runs last because it reads everything and writes nothing.

This is the same shape as a query plan in a database. The query optimiser takes a SQL statement, builds a graph of relational operations (each one a system!), and topo-sorts them into an execution plan. A simulator is a query plan running every tick. Students who follow this thread end up writing their own minimal query engine without realising it.

Not callbacks. Not signals. Not pub/sub.

This is where the Pythonic “loose coupling” idioms come asking, and the right answer is to refuse them. Three patterns to name and exclude:

Observers / event buses. A system subscribes to an event (“a creature was born”) and runs some handler. The order in which handlers fire is whoever subscribed first, or whatever the framework picks, or — most commonly — unspecified by design. This is the opposite of what this chapter is asking for. The DAG fixes order; an event bus deliberately does not.

Django/Flask-style signals. Frameworks teach signal.connect(handler) so that any module can wire itself into any lifecycle point. The result is a tick whose execution order depends on which modules were imported, in what order, and which connect calls ran. The DAG depends on declared data dependencies; signals depend on import order.

Callbacks. A system “calls back” to user code at some point in its body. Now the user code is part of the tick, but it has no declared read-set, no declared write-set, and runs at a moment determined by the implementation of the calling system. The contract from §13 is gone.

In all three cases the problem is the same: order is not declared; it is emergent from runtime accidents. A reader that runs before its writer reads stale data — yesterday’s snapshot of a table that was supposed to have been updated. A reader that runs after its consumer reads garbage — a half-written table mid-update. The DAG is the contract that prevents both. Each of the three patterns above replaces the contract with a hope.

A simulator’s tick is a topologically-sorted call list:

def tick(world: World, dt: float) -> None:
    food_spawn(world.food, dt)
    motion(world.pos_x, world.pos_y, world.vel_x, world.vel_y, dt)
    next_event(world.pending_event, world.pos_x, world.pos_y, world.food, ...)
    apply_eat(world.energy, world.food, world.pending_event)
    apply_reproduce(world.to_insert, world.energy, world.pending_event)
    apply_starve(world.to_remove, world.energy, world.pending_event)
    cleanup(world.to_remove, world.to_insert, ...)
    inspect(world)  # read-only, write-set empty

Eight function calls, in topological order. Adding a system means adding a line and re-deriving the order from the new system’s read-set and write-set. There is no register(), no subscribe(), no signal.connect(). The sequence is the program; the program is the sequence.

Why acyclic

A cycle is a contradiction. Suppose system A writes table T, system B reads T and writes U, system A reads U. Now A both produces T (which B reads) and consumes U (which B writes). A and B cannot both run before each other in the same tick.

A cycle in the system graph is a design bug; it must be broken — usually by buffering one system’s write so it is consumed next tick instead of this tick. That buffering is exactly what §15 — State changes between ticks names. Cycles do not disappear when you write a simulation; they get a name and a discipline.

Parallelism for free

Once the DAG is explicit, parallelism becomes trivial. Any two systems on the same DAG level — neither one a transitive dependency of the other — can run on different processes. In the simulator above, apply_eat, apply_reproduce, and apply_starve all consume pending_event and produce disjoint output tables (energy / food, to_insert, to_remove); they can run in parallel without coordination. The schedule is implied by the graph. §31 picks this up under the GIL.

The observer-pattern alternative cannot offer this. Without an explicit DAG, the framework cannot tell which handlers are independent and which are not — so it either runs everything serially or relies on the user to add manual synchronisation. The DAG-first design gets parallelism for free the moment the read-sets and write-sets are accurate; the observer-first design has to invent it back.

Exercises

1. Draw the DAG. Take the eight simulator systems (motion, food_spawn, next_event, apply_eat, apply_reproduce, apply_starve, cleanup, inspect) and draw the dependency graph yourself, deriving the edges from each system’s read-set and write-set in code/sim/SPEC.md. Compare with the diagram above.

2. Spot the cycle. Suppose apply_starve writes to food (returning fuel to the world when a creature dies). Now apply_starve writes food, which food_spawn reads. food_spawn writes food, which next_event reads. next_event writes pending_event, which apply_starve reads. Where’s the cycle? How would you break it? (Hint: §15.)

3. Topological sort by hand. Given:

   • A writes X
   • B reads X, writes Y
   • C reads X, writes Z
   • D reads Y and Z, writes W

   Which systems can run in parallel? What’s a valid execution order? Are there multiple valid orders?

4. Topological sort in Python. Implement def topo_sort(systems: list[tuple[str, set[str], set[str]]]) -> list[str] taking (name, read_set, write_set) triples and returning a valid execution order. Use Kahn’s algorithm. Apply it to your answer to exercise 1 — it should produce the same ordering (or one of the valid alternatives).

5. Compose two systems. Write motion (operation, writes pos_x, pos_y) and next_event (operation, writes pending_event). Wire them into a tick(world, dt) function that calls them in order. Inspect pending_event after the tick.

6. Add cleanup. Add a cleanup system that processes to_remove and to_insert (both initially empty arrays). Wire it after next_event. Confirm the call list reads top-to-bottom in dependency order.

7. The wrong way: an observer. Implement the same three-system tick using an event-bus pattern: bus.subscribe("tick", motion); bus.subscribe("tick", next_event); bus.subscribe("tick", cleanup); bus.fire("tick", world). Run it. Note that the order is now implicit in registration order, and any new subscriber inserted at runtime can change the order silently. Compare reading the resulting code to reading the function-call form. Which one tells you what runs when?

8. (stretch) A query planner. Take five hand-written SQL queries (each one a system shape) and draw the relational-algebra plan for each. Compare with how motion → next_event → apply_* decomposes the simulator. The shape is the same.

Reference notes in 14_systems_compose_into_a_dag_solutions.md.

What’s next

§15 — State changes between ticks is the rule that makes the DAG actually work: mutations buffer; the world transitions atomically.

Solutions: 14 — Systems compose into a DAG

Exercise 1 — Draw the DAG

Reading each system’s read-set and write-set:

food_spawn       reads {}                                         writes {food}
motion           reads {vel_x, vel_y, food}                       writes {pos_x, pos_y}
next_event       reads {pos_x, pos_y, food}                       writes {pending_event}
apply_eat        reads {pending_event}                            writes {energy_delta}
apply_reproduce  reads {pending_event}                            writes {to_insert}
apply_starve     reads {pending_event}                            writes {to_remove}
cleanup          reads {to_remove, to_insert, energy_delta}       writes {next_state}
inspect          reads {pos_x, pos_y, energy, food, ids}          writes {}

Edges (writer → reader):

food_spawn → motion             (food)
food_spawn → next_event         (food)
food_spawn → inspect            (food)
motion     → next_event         (pos_x, pos_y)
motion     → inspect            (pos_x, pos_y)
next_event → apply_eat          (pending_event)
next_event → apply_reproduce    (pending_event)
next_event → apply_starve       (pending_event)
apply_eat       → cleanup       (energy_delta)
apply_reproduce → cleanup       (to_insert)
apply_starve    → cleanup       (to_remove)

This matches the chapter’s diagram. The three appliers form a “fan-out”; cleanup is the “fan-in” that consumes their outputs.

Exercise 2 — Spot the cycle

If apply_starve writes food (returning fuel when a creature dies), the chain becomes:

food_spawn → next_event → apply_starve → food_spawn? (already ran this tick!)

food_spawn writes food; apply_starve reads pending_event (from next_event, which reads food from food_spawn); apply_starve writes food — but food_spawn already wrote food earlier this tick. The cycle is:

food_spawn → next_event → apply_starve → food_spawn   (back-edge: both write `food`)

A cycle of writers to the same column is the same-tick contradiction the chapter warns against. Break it by buffering: apply_starve writes to a food_returns buffer; food_spawn next tick reads food_returns and incorporates it into the new food table. The cycle becomes a tick boundary — the §15 mutations buffer discipline.

Exercise 3 — Topological sort by hand

A writes X
B reads X, writes Y
C reads X, writes Z
D reads Y and Z, writes W

Dependencies:

• B depends on A (X)
• C depends on A (X)
• D depends on B (Y) and C (Z)

Parallelism: B and C have the same predecessor (A) and disjoint write-sets (Y vs Z). They can run in parallel.

Valid execution orders:

• A, B, C, D
• A, C, B, D
• A, {B || C}, D (B and C concurrent)

All three are correct; the schedule chooses one. Multiple valid sorts is the norm — any sort respecting the edges is correct, and the DAG itself does not pick.

Exercise 4 — Topological sort in Python (Kahn’s algorithm)

def topo_sort(systems: list[tuple[str, set[str], set[str]]]) -> list[str]:
    """Kahn's algorithm. systems = [(name, read_set, write_set), ...]"""
    writers: dict[str, set[str]] = {}
    for name, _, ws in systems:
        for t in ws:
            writers.setdefault(t, set()).add(name)

    edges:  dict[str, set[str]] = {name: set() for name, _, _ in systems}
    in_deg: dict[str, int]      = {name: 0    for name, _, _ in systems}

    for name, rs, _ in systems:
        for t in rs:
            for w in writers.get(t, ()):
                if w != name and name not in edges[w]:
                    edges[w].add(name)
                    in_deg[name] += 1

    queue = sorted(n for n, d in in_deg.items() if d == 0)
    order: list[str] = []
    while queue:
        queue.sort()                                  # deterministic across runs
        n = queue.pop(0)
        order.append(n)
        for m in sorted(edges[n]):
            in_deg[m] -= 1
            if in_deg[m] == 0:
                queue.append(m)

    if len(order) != len(systems):
        raise ValueError("cycle in DAG")
    return order


# Apply to the sim DAG (with cleanup writing to a buffer to break the cycle from §2)
sim = [
    ("food_spawn",     set(),                                     {"food"}),
    ("motion",         {"vel_x","vel_y","food"},                  {"pos_x","pos_y"}),
    ("next_event",     {"pos_x","pos_y","food"},                  {"pending_event"}),
    ("apply_eat",      {"pending_event"},                         {"energy_delta"}),
    ("apply_reproduce",{"pending_event"},                         {"to_insert"}),
    ("apply_starve",   {"pending_event"},                         {"to_remove"}),
    ("cleanup",        {"to_remove","to_insert","energy_delta"},  {"next_state"}),
    ("inspect",        {"pos_x","pos_y","energy","ids","food"},   set()),
]

print(topo_sort(sim))
# ['food_spawn', 'motion', 'inspect', 'next_event', 'apply_eat',
#  'apply_reproduce', 'apply_starve', 'cleanup']

A valid order. inspect lands earlier than the chapter diagram suggests because it has no consumers — Kahn’s algorithm pulls it as soon as its read-set is satisfied. Both placements (right after motion or right at the end) are correct topological sorts.

For exercise 3:

sys2 = [("A", set(), {"X"}),
        ("B", {"X"}, {"Y"}),
        ("C", {"X"}, {"Z"}),
        ("D", {"Y","Z"}, {"W"})]
print(topo_sort(sys2))     # ['A', 'B', 'C', 'D']

Exercise 5 — Compose two systems

import numpy as np

class World:
    def __init__(self, n):
        rng = np.random.default_rng(0)
        self.pos_x = rng.uniform(0, 10, n).astype(np.float32)
        self.pos_y = rng.uniform(0, 10, n).astype(np.float32)
        self.vel_x = rng.uniform(-1, 1, n).astype(np.float32)
        self.vel_y = rng.uniform(-1, 1, n).astype(np.float32)
        self.pending_event = []          # list of (timestamp, kind, idx)


def motion(w: World, dt: float) -> None:
    w.pos_x += w.vel_x * dt
    w.pos_y += w.vel_y * dt


def next_event(w: World) -> None:
    w.pending_event.clear()
    # toy: an event for whichever creature is closest to (0, 0)
    d2 = w.pos_x ** 2 + w.pos_y ** 2
    i  = int(np.argmin(d2))
    w.pending_event.append((float(d2[i]), "closest", i))


def tick(w: World, dt: float) -> None:
    motion(w, dt)
    next_event(w)


w = World(100)
tick(w, 1.0 / 30.0)
print(w.pending_event)        # one event per tick

The tick is two function calls in topological order. The DAG is two nodes, one edge (motion → next_event via pos_x/pos_y).

Exercise 6 — Add cleanup

def cleanup(w: World) -> None:
    # toy: drop the closest-to-origin creature (an "eaten" event)
    if w.pending_event:
        _, _, i = w.pending_event[0]
        keep = np.ones(len(w.pos_x), dtype=bool)
        keep[i] = False
        w.pos_x = w.pos_x[keep]
        w.pos_y = w.pos_y[keep]
        w.vel_x = w.vel_x[keep]
        w.vel_y = w.vel_y[keep]


def tick(w: World, dt: float) -> None:
    motion(w, dt)
    next_event(w)
    cleanup(w)


w = World(100)
for _ in range(10):
    tick(w, 1.0 / 30.0)
print(f"after 10 ticks: {len(w.pos_x)} creatures left")    # 90

Three function calls, top to bottom in dependency order. Adding a fourth system means writing one line and re-running topo_sort if the order is non-trivial. There is no register(), no subscribe(). The sequence is the program; the program is the sequence.

Exercise 7 — The wrong way: an observer

class EventBus:
    def __init__(self):
        self.subs: dict[str, list] = {}
    def subscribe(self, event, handler):
        self.subs.setdefault(event, []).append(handler)
    def fire(self, event, *args, **kwargs):
        for h in self.subs.get(event, []):
            h(*args, **kwargs)

bus = EventBus()
bus.subscribe("tick", motion)
bus.subscribe("tick", next_event)
bus.subscribe("tick", cleanup)

w = World(100)
bus.fire("tick", w, 1.0 / 30.0)        # works — but only because we registered in order

Three subtle problems with this version:

1. Order is implicit in registration order. Swap the two subscribe lines for next_event and motion — the program runs without error, with stale data. There is no signal that the order is wrong.
2. A new subscriber inserted at runtime can change the order silently. Some plugin loads at startup, calls bus.subscribe("tick", validate_invariants), and inserts itself in the middle. The loop now runs in a different order; whether that’s correct depends entirely on the plugin’s read/write set, which the bus doesn’t know.
3. Reading the program is harder. To know what bus.fire("tick", ...) does, you must find every bus.subscribe("tick", ...) call across the entire codebase, in import order. Compare to def tick(w, dt): motion(w, dt); next_event(w); cleanup(w) — three lines, locally readable, ordering visible.

The function-call form tells you what runs when. The bus form tells you what can run when. The DAG-explicit version is the one that can be reasoned about, parallelised, tested, and trusted.

Exercise 8 — A query planner (stretch)

Take five SQL queries and decompose into relational-algebra operators:

-- Query 1: "active users by country, top 10"
SELECT country, COUNT(*) AS n FROM users WHERE active = TRUE
GROUP BY country ORDER BY n DESC LIMIT 10;

Plan:

LIMIT(10,
  SORT(n DESC,
    AGGREGATE(GROUP BY country, COUNT(*),
      FILTER(active = TRUE, SCAN(users)))))

Each level is a system in the chapter’s sense:

• SCAN reads the underlying table, writes a stream of rows
• FILTER reads the stream + predicate, writes a filtered stream
• AGGREGATE reads the stream, writes grouped rows
• SORT reads grouped rows, writes ordered rows
• LIMIT reads ordered rows, writes prefix

Each operator declares its read-set (the input stream) and write-set (the output stream); the plan is a topo-sorted DAG. Database optimisers explore alternative plans (a different join order, an index scan instead of a full scan), pick the cheapest, and execute.

A simulator does the same thing every tick, but with the plan fixed at design time rather than chosen by an optimiser. Students who write five small plans by hand notice that a tick-loop and a query plan are the same shape: a DAG of small operators consuming and producing tables.

15 — State changes between ticks

Concept node: see the DAG and glossary entry 15.

Inside a tick, the world is frozen. Systems read consistent snapshots of their inputs; mutations are queued, not applied; only at the tick boundary does the world step forward in one atomic transition.

This is the rule that makes the DAG from §14 actually work. If motion could mutate pos while next_event is reading pos, the data is inconsistent: half the creatures have moved, half have not. Even if the schedule is “correct” by topological order, what each system reads is no longer well-defined. By forbidding mutations to apply in-tick, the world becomes a clean function world_{t+1} = step(world_t, inputs_t). Every system reads world_t; every system writes into a buffer that becomes world_{t+1} only at the tick boundary.

Concretely: apply_starve does not call np.delete(creatures, slot) or pop from a Python list. It writes the doomed slot into to_remove. The creatures columns are unchanged for the rest of the tick. After every system has run, cleanup consumes to_remove and to_insert together, applying every queued change in one sweep. Now the next tick begins with a consistent new world state.

This pattern is called double buffering: there is the world the systems read (world_t), and the buffer of changes that becomes the world the next tick reads (world_{t+1}). The pattern shows up everywhere — graphics frame buffers, database transactions, event-sourced systems. The rule is always the same: writes accumulate, then commit.

The Python footguns this rule prevents

Python has two famous in-place-mutation footguns the discipline above eliminates.

The list-during-iteration bug. Removing from a list while iterating it silently skips elements. The iterator advances by index; list.remove shifts everything down by one; the next element is now at the index the iterator already passed:

# anti-pattern: bad!
creatures = [c1, c2, c3, c4, c5]   # all five starving
for c in creatures:
    if c.energy <= 0:
        creatures.remove(c)        # skips c2 and c4 — they survive
# Surviving creatures: 2 out of 5. The starvation system is broken
# and the simulation will run forever.

The dict-during-iteration bug. Removing from a dict while iterating raises:

# anti-pattern: bad!
for cid, c in creatures.items():
    if c.energy <= 0:
        del creatures[cid]
# RuntimeError: dictionary changed size during iteration

The list version is the dangerous one — it fails silently and hands you a wrong-but-finite simulation. The dict version is dangerous in a different way: the RuntimeError trains the reader to fix it locally (for cid in list(creatures.keys()):) without ever recognising the structural problem. Both are the same lesson: mutating a container while another piece of code is reading it is the bug, regardless of whether the language catches it.

The disciplined Python equivalent in numpy is one boolean mask per buffer:

def apply_starve(energy: np.ndarray, to_remove: list[int]) -> None:
    starvers = np.where(energy <= 0)[0]      # read-only scan
    to_remove.extend(starvers.tolist())       # buffered write

def cleanup(world: World, to_remove: list[int], to_insert: list[CreatureRow]) -> None:
    # apply removals first (swap_remove pattern, §21), then inserts
    ...

The starvation system only writes to to_remove. It never touches creatures. The creatures columns are unchanged when apply_starve returns — they are unchanged when apply_eat and apply_reproduce return. They are mutated exactly once per tick, by cleanup, after every other system is done. There is no window in which a system could see an inconsistent world.

The simlog is what this looks like in production

The reference implementation at .archive/simlog/logger.py is a 700-line columnar logger built on exactly this pattern. It maintains two Containers — pre-allocated numpy arrays plus a write pointer. The simulation writes into one container; when that container fills, the simlog atomically swaps containers and a background thread dumps the full one to disk. The simulation never observes a half-flushed buffer; the disk-flushing thread never observes a half-written row. Read it when this chapter clicks; it is the same idea this chapter teaches, sized up for production.

Costs and trade

Two costs to absorb. First, every mutation is one extra entry pushed to a to_remove or to_insert buffer. Second, the cleanup pass is now its own system in the DAG. The benefit dwarfs the costs: every other system in the book composes cleanly, and parallelism becomes easy. With in-tick mutation, every parallel scheduling decision becomes a race condition. With buffered mutation, races are structurally impossible — disjoint write-sets are disjoint by construction.

A subtle case is insertions. A creature born during a tick (via apply_reproduce) does not appear in any system’s read-set during that tick — it is in to_insert, not in creatures. The newborn lives its first life on the next tick. This is the right behaviour for almost every simulation: it gives every creature an equal first tick of life. The alternative — applying inserts mid-tick — is a closed-loop bug factory.

Within one system, the writes can be in-tick: a system that updates pos_x[:] = pos_x + vel_x * dt for every creature in one numpy call applies all writes “at once” inside that system, because the rest of the system is the only reader and the only writer. The buffering rule is between systems, not between iterations within one system. Inside a system, the writes are sequential (or vectorised); between systems, the writes are batched.

The shape that emerges is: read everything into local arrays at system entry; do work; write outputs to buffers at system exit; commit at tick boundary. It is the same shape as the audio engine’s frame buffer, the database’s transaction commit, and the version-controlled file system’s commit-and-merge. They all solve the same problem: how do you read consistent state while the world is changing?

Exercises

These build on the simulator skeleton. Your to_remove: list[int] and to_insert: list[CreatureRow] should already exist.

1. The list bug. Build a list of 100 creatures where 30 have energy <= 0. Iterate the list, calling creatures.remove(c) whenever c.energy <= 0. Count how many starvers survive. Why did the bug only affect some of them? (Hint: every removal shifts the iterator past one extra element.)
2. The dict bug. Build a dict[int, Creature] of 100 with the same 30 starvers. Iterate creatures.items(), calling del creatures[cid] whenever c.energy <= 0. Note the RuntimeError. Now “fix” it locally with for cid in list(creatures.keys()): — does the simulation now produce the right answer? Yes, but only because the local fix accidentally makes a complete copy first; you have papered over the structural problem at the cost of an O(N) allocation per tick.
3. The buffered fix. Rewrite the function to compute starvers = np.where(energy <= 0)[0] (read-only scan) and append the result to to_remove. After the loop completes, apply all removals in one pass using the swap_remove pattern (preview of §21). Verify all 30 starvers die.
4. The cleanup pass. Write def cleanup(world, to_remove, to_insert). Apply removals first (using swap_remove on each affected column), then insertions. Why this order, and not the other? (Hint: insertions may reuse slots freed by removals — see §24.)
5. Show two ticks. Run the loop for two ticks. After tick 1, log the population. After tick 2, log it again. Confirm that creatures killed in tick 1’s apply_starve do not appear in tick 2’s input — they were removed at the tick boundary, between the two ticks.
6. Insertions are tick-delayed. A creature reproduces in tick 5: parent in creatures, two offspring in to_insert. After cleanup, the offspring are in creatures. In tick 6 the offspring receive their first system pass. Confirm by adding an age_in_ticks column and watching offspring start at 0 in tick 6, not in tick 5.
7. (stretch) A bad design that almost works. Try to apply mutations in-tick carefully — collect dead creatures first, then process them in reverse-index order to avoid the iterator-skip bug. Show one specific case where this still corrupts state. (Hint: a reproduction produces an offspring whose new index conflicts with an in-progress death.)
8. (stretch) Read the simlog. Open .archive/simlog/logger.py. Find the two Container instances. Find the line where they swap. Find the function the background thread runs. Note that the logger never holds both containers locked simultaneously — the swap is atomic, the dump is on the inactive container. This is the production version of what exercise 3 teaches.

Reference notes in 15_state_changes_between_ticks_solutions.md.

What’s next

§16 — Determinism by order is the property the buffering rule guarantees: same inputs, same system order, same outputs. Reproducibility is structural.

Solutions: 15 — State changes between ticks

Exercise 1 — The list bug

class Creature:
    def __init__(self, energy): self.energy = energy

import random
random.seed(0)
cs = [Creature(-1)] * 30 + [Creature(10)] * 70
random.shuffle(cs)

for c in cs:
    if c.energy <= 0:
        cs.remove(c)

remaining_starvers = sum(1 for c in cs if c.energy <= 0)
print(f"30 starvers initially → {remaining_starvers} survived after remove-during-iter")

30 starvers initially → 8 survived

The iterator advances by index. When cs.remove(c) shifts every later element down one slot, the index the iterator advances to skips the next element. So roughly every other starver is missed. The exact count depends on the shuffle (8 here, anywhere from 5-15 typical).

The bug is silent. The simulation runs, the program terminates, the answer is wrong. Nothing complains. The fact that you have to count the survivors to detect it is precisely why the discipline of buffered mutation exists.

Exercise 2 — The dict bug

cs = {i: Creature(-1 if i < 30 else 10) for i in range(100)}

# Naive remove-during-iter
try:
    for cid, c in cs.items():
        if c.energy <= 0:
            del cs[cid]
except RuntimeError as e:
    print(f"got expected: {e}")
# RuntimeError: dictionary changed size during iteration

# Local "fix": iterate a snapshot of the keys
cs = {i: Creature(-1 if i < 30 else 10) for i in range(100)}
for cid in list(cs.keys()):                    # snapshot — O(N) allocation
    if cs[cid].energy <= 0:
        del cs[cid]
print(f"survivors: {len(cs)}")                  # 70 — correct!

The dict version crashes loudly, which is better than silently wrong, but its lesson trains the reader to apply a local fix (list(cs.keys())) without recognising the structural problem. The local fix:

• Costs an O(N) allocation per tick (the snapshot).
• Hides the mutation pattern; the next reviewer assumes the iteration is safe.
• Doesn’t fix the underlying issue: a system reading cs while another piece of code (you, in this case) writes to it.

Both bugs are the same lesson: mutating a container while another piece of code is reading it is the bug, regardless of whether the language catches it.

Exercise 3 — The buffered fix

import numpy as np

energy = np.array([-1.0]*30 + [10.0]*70, dtype=np.float32)
np.random.default_rng(0).shuffle(energy)
ids    = np.arange(100, dtype=np.uint32)

to_remove: list[int] = []

def apply_starve(energy: np.ndarray, to_remove: list[int]) -> None:
    starvers = np.where(energy <= 0)[0]
    to_remove.extend(starvers.tolist())

def cleanup(energy, ids, to_remove):
    """Apply queued removals via swap_remove (preview of §21)."""
    if not to_remove: return energy, ids
    keep = np.ones(len(energy), dtype=bool)
    keep[to_remove] = False
    return energy[keep], ids[keep]

apply_starve(energy, to_remove)
energy, ids = cleanup(energy, ids, to_remove)
print(f"30 starvers → {(energy <= 0).sum()} remain after one tick")    # 0

The starvation system is read-only: it scans energy and writes only to to_remove. The energy array does not change during apply_starve — it changes exactly once per tick, in cleanup. There is no window in which two systems could see different states.

Exercise 4 — The cleanup pass

def cleanup(world, to_remove: list[int], to_insert: list[dict]) -> None:
    """Apply removals first, then insertions.

    Removals first because:
    - swap_remove (§21) frees specific slots by moving the last row in.
    - Inserts can target those freed slots (§24 recycling).
    - Doing inserts first would force them to allocate fresh slots even
      when freed slots are about to become available.
    """
    # Removals via swap_remove
    for slot in sorted(to_remove, reverse=True):     # high-to-low avoids index shifting
        last = len(world.energy) - 1
        if slot != last:
            for col in (world.pos_x, world.pos_y, world.energy, world.ids, world.gens):
                col[slot] = col[last]
        # truncate (in real code: world.live_count -= 1)
        ...
    to_remove.clear()

    # Insertions into freed (or newly-allocated) slots
    for row in to_insert:
        slot = world.allocate_slot()                 # reuses recycled slots first
        for col_name, value in row.items():
            getattr(world, col_name)[slot] = value
    to_insert.clear()

Removals first means freed slots immediately host inserts on the same tick; the table doesn’t grow when births and deaths balance. Inserts first would force every newborn to push the table to a fresh slot before the dying creatures return their slots — a one-tick high-water mark proportional to the death rate. §24 makes this explicit.

Exercise 5 — Show two ticks

energy = np.array([-1.0, -1.0, 5.0, 8.0, -1.0], dtype=np.float32)
ids    = np.array([10, 11, 12, 13, 14], dtype=np.uint32)
to_remove: list[int] = []

# Tick 1
apply_starve(energy, to_remove)
energy, ids = cleanup(energy, ids, to_remove)
to_remove.clear()
print(f"after tick 1: ids={ids.tolist()}, energy={energy.tolist()}")
# ids=[12, 13], energy=[5.0, 8.0]

# Tick 2 — only the survivors run
apply_starve(energy, to_remove)         # nothing dies
print(f"tick 2 input: ids={ids.tolist()} (the dead ids 10,11,14 are not in this list)")

The dead creatures from tick 1 are gone between ticks. Tick 2’s apply_starve sees only the survivors. The systems don’t have to know about the death — the cleanup pass handled the bookkeeping at the tick boundary.

Exercise 6 — Insertions are tick-delayed

ages = np.zeros(N, dtype=np.uint16)

def age_creatures(ages):
    ages += 1                                   # one increment per tick

# Tick 5: a parent reproduces; offspring go into to_insert with age=0
to_insert.append({"pos_x": 1.0, "pos_y": 2.0, "energy": 5.0, "ages": 0})

cleanup(world, to_remove, to_insert)            # offspring now in `creatures`

# Tick 6: age_creatures runs over all live creatures, including the new ones
age_creatures(world.ages)                       # offspring goes 0 → 1

The newborn does not appear in any system’s read-set during tick 5 — it is in to_insert, not in creatures. Its first tick of life is tick 6, where it is incremented from 0 to 1 by age_creatures. Every creature gets a full tick of life on its first tick, regardless of when in the previous tick it was born.

The alternative (in-tick insertion) would mean a creature born at the start of tick 5 ages from 0 → 1 in the same tick, while one born at the end of tick 5 ages 0 → 0. That arbitrariness is what the rule prevents.

Exercise 7 — A bad design that almost works (stretch)

The “fix” of processing dead creatures in reverse-index order:

# anti-pattern: bad!
def apply_starve_inplace(creatures):
    dead = [i for i, c in enumerate(creatures) if c.energy <= 0]
    for i in sorted(dead, reverse=True):
        del creatures[i]                        # high-to-low avoids index-shift

The case where it still corrupts state: insertions during the same tick. Suppose apply_reproduce runs after apply_starve and pushes new creatures onto the same creatures list:

# tick body
apply_starve(creatures)                          # deletes some, frees indices
apply_reproduce(creatures)                       # appends new ones
inspect(creatures)                               # sees a mixed-state world

What inspect sees: a mix of creatures who were alive at the start of the tick (still here, at possibly-different indices), creatures born this tick (already alive in creatures), and the gaps if the death pattern wasn’t a clean suffix. Other systems that captured indices at the start of the tick (e.g., a pending_event from next_event) now point at wrong rows.

The buffered approach prevents all of this by definition: nothing changes in creatures until cleanup, and cleanup applies removals + insertions in one consistent sweep.

Exercise 8 — Read the simlog (stretch)

The vendored copy at .archive/simlog/logger.py is the production-grade version of this chapter’s pattern. Things to find:

• Two Container instances. The logger maintains two pre-allocated numpy column buffers (active and inactive). The simulation writes only to active.
• The atomic swap. When active fills, the logger atomically swaps the two references (active, inactive = inactive, active). The simulation’s next write goes to the now-empty buffer; the previously-active buffer is now inactive and ready for flushing.
• The background flush thread. A worker thread sleeps until inactive is non-empty, then writes its contents to disk (.npz chunks) and clears it.

The simulator never holds both containers at once. The flush thread never sees a write in progress. The whole apparatus is the chapter’s “writes accumulate, then commit” rule — at production scale, with a background flush, ~1 µs per logged event, and zero coordination cost on the hot path. Worth reading once you have written the toy version yourself.

16 — Determinism by order

Concept node: see the DAG and glossary entry 16.

A program is deterministic if the same inputs and the same execution produce the same outputs, every time. Sounds obvious. It is not — most modern Python programs are not deterministic by default. Threads run in OS-scheduled order. Sets iterate in randomised order across processes. The system clock differs by run. random.random() reads from a global instance whose state depends on import order and prior calls.

In an ECS architecture, determinism is structural. Same world state at tick start + same system order + same inputs (events, RNG seed) = same world state at tick end. Bit-identical. Every time.

This is not a quality goal; it is a precondition for almost everything the book builds on:

• Replay. The world is the log decoded (§37). Replay reconstructs world state by re-running the inputs through the same system sequence. Without determinism, replay is impossible.
• Testing. A property test fixes an RNG seed and asserts the simulator behaves identically across runs. Without determinism, every test is flaky.
• Distributed simulation. Multiple machines run identical copies of the world. Without determinism, they drift apart by tick 1.
• Debugging. A bug at tick 4783 should appear at tick 4783 every run. Without determinism, debugging real-time bugs becomes guesswork.

The recipe, Python edition

The recipe for determinism is to forbid every source of non-determinism in the inner systems. In Python the sources have specific names.

No raw set iteration. From code/measurement/set_iteration_order.py, three fresh subprocesses iterating the same six-element set produced three different orders:

run 1: delta,foxtrot,echo,bravo,charlie,alpha
run 2: bravo,foxtrot,delta,echo,alpha,charlie
run 3: echo,delta,foxtrot,charlie,bravo,alpha

CPython hashes strings using a per-process random seed (PYTHONHASHSEED), and set iteration order is a function of the hash table’s bucket layout. Across processes, the layout differs; the iteration order differs. This is by design — it protects servers from hash-flooding attacks — but it is also a source of non-determinism that the simulator forbids. Never iterate a set inside a system. If you need an iteration order, use a sorted list, a numpy array, or a dict (which is insertion-ordered since CPython 3.7 and survives the same test):

run 1: alpha,bravo,charlie,delta,echo,foxtrot
run 2: alpha,bravo,charlie,delta,echo,foxtrot
run 3: alpha,bravo,charlie,delta,echo,foxtrot

No system clock inside a system. Get time from input events, not from time.time() or time.perf_counter(). Time is a value passed into the system, not read from the OS. The tick loop’s outer scaffolding may read the wall clock; the systems inside the tick may not.

One RNG, seeded. A single np.random.default_rng(seed) per simulator instance, used in a defined order. Each system that needs randomness reads from it in DAG order. Never random.random() (reads global state), never np.random.random() without the rng object (uses a global). Pass the rng as a parameter — it has a declared read-set just like any other input.

No threads inside a system. A system runs single-threaded internally. The GIL does not save you from non-determinism here; it serialises Python bytecode but not the order in which threads acquire it. Parallelism happens between systems with disjoint write-sets (§31) using multiprocessing, not inside one system using threading.

Buffered mutations. §15’s rule: mutations apply at tick boundaries, not mid-tick.

One Python-specific footnote: hash() itself. Hash randomisation has been on by default since CPython 3.3 for str and bytes (and the containers that derive from them, including frozenset). If a system computes hash(some_string) and uses that value as part of its output, the output is non-deterministic across processes. Use hashlib.blake2b(s.encode()).digest() — or any deterministic hash — when you need a stable hash inside a system.

These rules are restrictive. They are also the price of every benefit listed above. Most modern Python programs decline to pay this price and accept the costs — flaky tests, unreproducible bugs, divergent distributed simulation. The book pays the price.

The cost is at the boundary, not in the body

The cost of determinism is not absolute. Within a system, the implementation is free to use whatever it likes — vectorised numpy, low-level optimisations, even occasional non-deterministic libraries — as long as the inputs and outputs are bit-identical to what the abstract specification demands. The discipline is at the system boundary: between systems, everything must be reproducible.

Inside motion, you can use pos_x += vel_x * dt (numpy bulk op, deterministic) or np.einsum or write your own Cython kernel. As long as the output pos_x for given inputs is bit-identical across runs, the system is deterministic regardless of how its internals work. The contract is at the function boundary; the freedom is inside.

Testing for determinism

A test for determinism is concrete. Run the simulator twice with the same seed, the same input event log, the same system order. After 1,000 ticks, hash the entire world state — feed every numpy column through hashlib.blake2b(arr.tobytes()).hexdigest() and combine. If the hashes match, you are deterministic. If they do not, find the system whose output first differs, and trace the source of variability. Often: a set iterated, a time.time() call, a random.random() reading global state.

A simulator that is deterministic is also a simulator that can be tested. Once that property holds, every other quality goal — performance, parallelism, distribution — becomes safe to optimise toward. Without determinism, every optimisation is a coin flip.

The full payoff of determinism arrives at the save and load phase named in §11. The simulator can be paused, its tables serialised to disk, reloaded later, and resumed — and the result must be indistinguishable from a run that never paused. The mechanics arrive in §36 — Persistence is table serialization: a snapshot is the world’s columns written as .npz files — the same bytes they have in memory. Combined with the input event log, replay is structural — read the snapshot, replay events through the same DAG with the same seed, you reconstruct the world at any later tick exactly. Determinism (this section), serialization (§36), and log-as-world (§37) are the three legs of replay.

Exercises

1. Run the iteration-order exhibit. uv run code/measurement/set_iteration_order.py. Observe the set rows differ; the dict rows do not. Note that the dict survival is not a guarantee against frozenset keys, dict.values() derived from a set, or any operation that goes through hash bucket order — only the surface-level “I added these in order” pattern survives.
2. Hash the world. Write def hash_world(world) -> str that produces a hex digest by feeding every column through hashlib.blake2b(arr.tobytes()).update(...). Use this to compare world states across runs.
3. Two identical runs. Run the simulator twice with the same RNG seed (np.random.default_rng(42)) and the same input events. Hash the world at tick 100. Confirm they are equal.
4. Introduce non-determinism deliberately. Replace your seeded default_rng(42) with np.random.default_rng() (no seed — uses entropy). Run twice. Show the hashes differ.
5. Find the culprit. Suppose your hashes differ. Hash the world after each system in the DAG. Identify which system’s output first differs, and what source of non-determinism it pulls from. Common offenders: for k in some_set:, time.time(), random.random(), hash(some_string).
6. Time as input. Find a system that uses time.perf_counter() and refactor it to instead take current_time: float as a parameter. The system is now deterministic; the source of current_time is the only place non-determinism can enter.
7. The set trap up close. Build a set of 1,000 random integers (use a default_rng(42) so the set contents are deterministic). Iterate it three times in the same process. Are the orders the same? Now run the program twice in two fresh shells. Are the orders the same across runs? (Hint: the answers are yes and no, in that order. The trap is that a single test run will not catch the bug; two test runs in two CI workers will.)
8. (stretch) A property test. Hand-roll a simple property test: generate 100 random seeds. For each, run the simulator for 100 ticks. Hash the resulting world. Verify that the same seed always produces the same hash, and that different seeds usually produce different hashes.

Reference notes in 16_determinism_by_order_solutions.md.

What’s next

You have closed Time & passes. Determinism is structural; replay is architectural; the next phase is Existence-based processing, starting with §17 — Presence replaces flags. The simulator’s hunger and starvation systems are about to lose their booleans.

Solutions: 16 — Determinism by order

Exercise 1 — Run the iteration-order exhibit

uv run code/measurement/set_iteration_order.py

Source: code/measurement/set_iteration_order.py.

Set iteration order across runs:
  run 1: delta,bravo,foxtrot,echo,alpha,charlie
  run 2: alpha,foxtrot,delta,charlie,echo,bravo
  run 3: bravo,echo,alpha,foxtrot,charlie,delta
  → 3 distinct orders — sets are non-deterministic.

Dict iteration order across runs:
  run 1: alpha,bravo,charlie,delta,echo,foxtrot
  run 2: alpha,bravo,charlie,delta,echo,foxtrot
  run 3: alpha,bravo,charlie,delta,echo,foxtrot
  → orders match — dicts are insertion-ordered since CPython 3.7.

The dict survival is the insertion order of the keys, which is a property of how you populated the dict. It’s not a guarantee against:

• dict(some_set) — values come from set iteration; first survival breaks.
• frozenset keys — same hash-bucket randomness applies.
• dict.fromkeys(some_set) — same.

Survive only what you can prove. When in doubt: sorted(...).

Exercise 2 — Hash the world

import hashlib
import numpy as np

def hash_world(world) -> str:
    h = hashlib.blake2b(digest_size=16)
    for col in (world.pos_x, world.pos_y, world.vel_x, world.vel_y,
                world.energy, world.ids, world.gens):
        h.update(col.tobytes())
    h.update(np.array([len(world.pos_x)], dtype=np.int64).tobytes())  # length too
    return h.hexdigest()

arr.tobytes() returns the contiguous in-memory bytes. blake2b is fast and 16 bytes is plenty for run-to-run comparison. Including the length prevents two worlds with different sizes but identical-prefix data from hashing the same.

Exercise 3 — Two identical runs

def run(seed: int, ticks: int) -> str:
    rng = np.random.default_rng(seed)
    world = build_world(rng, n=100)
    for _ in range(ticks):
        tick(world, rng, dt=1.0/30.0)
    return hash_world(world)

assert run(42, 100) == run(42, 100)             # same seed → same hash
print(run(42, 100))                             # any deterministic 32-char hex

Bit-identical. If this fails, your simulator has a non-determinism source somewhere — exercise 5 is the diagnostic.

Exercise 4 — Introduce non-determinism deliberately

def run_unseeded(ticks: int) -> str:
    rng = np.random.default_rng()               # no seed — entropy from OS
    world = build_world(rng, n=100)
    for _ in range(ticks):
        tick(world, rng, dt=1.0/30.0)
    return hash_world(world)

print(run_unseeded(100))   # something
print(run_unseeded(100))   # something else

Without a seed, default_rng() reads os.urandom. Two consecutive runs draw from different entropy and produce different results. The hashes differ. The simulator is now untestable.

Exercise 5 — Find the culprit

def hash_world_per_system(world) -> dict:
    """Run each system and hash the world after each."""
    snapshots = {}
    snapshots["start"] = hash_world(world)
    food_spawn(world)
    snapshots["after food_spawn"] = hash_world(world)
    motion(world)
    snapshots["after motion"] = hash_world(world)
    next_event(world)
    snapshots["after next_event"] = hash_world(world)
    apply_eat(world)
    snapshots["after apply_eat"] = hash_world(world)
    # ... and so on
    return snapshots

Run twice with the same seed; compare the two snapshots dicts. The first key whose hash differs identifies the offending system. Look inside its body for:

• for x in some_set:
• time.time(), time.perf_counter(), datetime.now()
• random.random(), np.random.random() without an rng instance
• hash(some_string) used in any output computation
• os.environ, os.getpid()
• Iteration over a frozenset, dict_keys, or dict.fromkeys(set)

Most simulator non-determinism in the wild is one of these patterns hiding inside a “harmless” helper.

Exercise 6 — Time as input

# Before — system reads the OS clock:
def schedule_event_bad(events):
    now = time.perf_counter()                   # non-deterministic
    events.append((now + 0.5, "fire"))

# After — system takes time as a parameter:
def schedule_event(events, current_time: float):
    events.append((current_time + 0.5, "fire"))

# The tick loop scaffolding reads the wall clock — once, at the boundary:
def tick(world, current_time: float, dt: float):
    schedule_event(world.events, current_time)
    motion(world, dt)
    ...

current_time = 0.0
for _ in range(100):
    tick(world, current_time, dt)
    current_time += dt

The systems are pure functions of their inputs. The tick loop chooses what current_time is. For a real-time simulator, current_time = time.perf_counter() - start. For a deterministic replay, current_time comes from the event log. Same systems, two execution modes, the difference at the boundary, not in the body.

Exercise 7 — The set trap up close

import numpy as np
rng = np.random.default_rng(42)
s   = set(rng.integers(0, 1_000_000, size=1000).tolist())   # contents deterministic

# Three iterations IN THE SAME PROCESS
o1 = list(s); o2 = list(s); o3 = list(s)
print(o1 == o2 == o3)                # True — same process, same hash table layout

Within one process, set iteration order is stable (the hash table layout doesn’t change between iterations). Run the same program in two fresh shells and you get two different orders, because PYTHONHASHSEED is randomised per process.

The trap: a single test run does not catch the bug. The CI worker that runs the same test in five parallel processes catches it. The user who reports “works on my machine, fails in CI” has hit it.

The fix: never iterate a set in a system. Always:

for x in sorted(s):                  # deterministic across runs
    ...

Or store the data in something that’s already ordered (a list, a numpy array). Or use a dict with insertion order if order matters and uniqueness matters.

Exercise 8 — A property test (stretch)

import numpy as np

def property_test_determinism(n_seeds: int = 100, ticks: int = 100):
    seeds = list(range(n_seeds))
    for seed in seeds:
        h1 = run(seed, ticks)
        h2 = run(seed, ticks)
        assert h1 == h2, f"non-deterministic at seed {seed}: {h1} vs {h2}"
    distinct = len({run(s, ticks) for s in seeds})
    assert distinct > n_seeds * 0.95, "different seeds collapse to same world — bug"
    print(f"OK: {n_seeds} seeds, all reproducible, {distinct} distinct worlds")

property_test_determinism()

Two assertions:

1. Same seed → same world. Catches non-determinism (a set iteration, a time.time() call).
2. Different seeds → different worlds (mostly). Catches accidental seed-loss (a global random.seed() overriding the per-run rng) — without it, every run produces the same hash regardless of the seed parameter.

This is the entire core of property-based simulation testing. Hypothesis (the Python library) builds elaborate generators and shrinkers around it; the underlying assertion is the same.

17 — Presence replaces flags

Concept node: see the DAG and glossary entry 17.

A creature can be hungry. Three ways to model it.

The instinct most Python programmers arrive with is a boolean field on the object: is_hungry: bool on every Creature, set to True when energy drops below a threshold, set to False when energy is restored. Every system that cares about hunger checks the flag: for c in creatures: if c.is_hungry: .... This is everywhere; it is the natural choice; it is what most Python tutorials reach for. It is also the worst of the three options — it is both AoS (per-creature object) and flag-shaped (one bit per creature regardless of state), and it forces every consumer to scan all N creatures to find the K hungry ones.

The middle option — better than per-instance booleans, still not the disciplined choice — is a boolean column. is_hungry = np.zeros(N, dtype=bool), indexed in lockstep with the rest of the creature table. This is what most readers will reach for after Part 2. It pays one byte per creature (numpy’s bool is one byte, not one bit), but the bytes are contiguous; numpy vectorises the scan; SIMD reads forty creatures per cache line. Compared to the per-object form, it is one-to-two orders of magnitude faster. Compared to the disciplined form below, it still costs N bytes regardless of how few creatures are hungry.

The data-oriented alternative is membership. There is a hungry table — a np.ndarray of creature ids, of length K (the number of currently-hungry creatures), no longer than it has to be. A creature is hungry if and only if its id is in hungry. The flag does not exist as a field; it exists as a fact about which table the creature appears in.

# Three representations of "is this creature hungry?"
is_hungry_attr     = creatures[i].is_hungry          # AoS bool field
is_hungry_mask     = bool(is_hungry[i])              # SoA bool column, O(N) bytes
is_hungry_presence = np.isin(creature_ids[i], hungry) # presence table, O(K) bytes

The substitution looks small: a bool field becomes a row in another table. The implications are not.

Four shifts that follow

Dispatch changes shape. The flag version is a per-creature filter inside every consuming system — walk all creatures, check the flag, do work if true. The membership version skips the filter — walk hungry, do work for every entry. At 1,000,000 creatures with 100,000 hungry, the flag version processes 1,000,000 rows; the membership version processes 100,000 — a 10× difference in work, and a 10× difference in memory bandwidth. §19 names this.

Storage changes shape. A np.bool_ column stores one byte per creature whether the flag is set or not. A creature with eight possible states needs eight bool columns = 8 bytes per creature; a million creatures store 8 MB of flags, most of which are False. Eight presence tables store only the entries that are set — if 10% of creatures are hungry, the hungry table is 10% the size of the flag column.

Persistence changes shape. Serialising a flag column writes the flag for every creature, including the ones where it is False. Serialising a presence table writes only the entries that exist. The latter is also closer to the natural shape of an event log (§37): a hungry_added event per entry, and that is the whole story.

Concurrency changes shape. Two bool columns on the same creature table sit adjacent in memory; concurrent writers to either column fight over the same cache lines (§33 — false sharing). Two presence tables are physically separate numpy arrays; concurrent writers to disjoint tables never collide (§31).

The reversal

The clean way to phrase the move: instead of asking each entity about its state, ask the state-table which entities have that state. The query is reversed; the lookup is reversed; the work shrinks. Most programs spend their lives doing the wrong direction; the data-oriented mindset is to reverse it.

A production example: in a real ECS daemon, an admission decision is is_admitted = peer_id in established_contacts. There is no is_admitted: bool on a peer; there is only the question “is this peer’s id in the table?”. With an id_to_slot index map (§23) this is O(1), no I/O, no enum.

When flags are right

Presence is not the only valid representation. A bool column is sometimes right — when nearly every entity has the state set (a near-universal flag wastes nothing as a column and saves on the membership scan); when the predicate is so cheap to compute on the fly that materialising it is silly (is_positive_x = pos_x > 0); when the data is short-lived and persistence does not matter; when the lookup pattern is “give me this creature’s hunger state” (per-creature query, where a column lookup is O(1) but a presence-table membership scan is O(K) without an index).

In this book, presence is the default; flags are a tradeoff to earn.

Exercises

These extend the §0 simulator skeleton.

1. Add a hungry table. Add hungry = np.empty(0, dtype=np.uint32) to your world. It is empty at start.
2. Populate it. Write a system def classify_hunger(energy, ids) -> np.ndarray that returns the ids of all creatures with energy[i] < HUNGER_THRESHOLD. The body is one numpy line: ids[energy < HUNGER_THRESHOLD]. Replace the world’s hungry with the result each tick.
3. Build the flag version. Add a parallel is_hungry = np.zeros(N, dtype=bool) indexed by creature slot. Write the equivalent classification system that sets the bool column.
4. Build the AoS version. Build a list[Creature] where Creature is a @dataclass(slots=True) with an is_hungry: bool field. Write the equivalent classification — a Python for loop. (Foreshadow: this is the version most tutorials teach.)
5. Time all three at 1M creatures, 10% hungry. Time classify_hunger (presence), the bool-column version (flag), and the AoS version. Note the ordering and the magnitudes. Presence and flag should be within ~2-5× of each other (both numpy); the AoS version should be one to two orders of magnitude slower than either (interpreter-bound, per §1).
6. The membership query. Write def is_hungry_p(hungry, id) -> bool (presence — bool(np.any(hungry == id))) and def is_hungry_f(is_hungry_col, slot) -> bool (flag — bool(is_hungry_col[slot])). Time both at 1M creatures. Note: presence is O(K) without an index map; the flag is O(1). §23 — Index maps is the fix that makes presence O(1) too.
7. “How many are hungry?” Write it three ways. Presence: len(hungry). Flag column: int(is_hungry.sum()). AoS: sum(1 for c in creatures if c.is_hungry). Compare wall times at 1M creatures with 10% hungry. The presence version is constant-time; the flag-column version walks all 1M as a single numpy reduction; the AoS version walks all 1M with interpreter dispatch on every step.
8. (stretch) Persist both. Serialise the flag-column version with np.save("is_hungry.npy", is_hungry) and the presence version with np.save("hungry.npy", hungry). Note the disk size for 1M creatures with 10% hungry. The presence file is ~400 KB; the flag-column file is ~1 MB even though 90% of the bits are 0. (Compression closes some of this gap, but not all of it — np.savez_compressed will help on the flag column more than on the presence array, because the flag column has a long run of zeros to compress and the presence array is already small.)

Reference notes in 17_presence_replaces_flags_solutions.md.

What’s next

§18 — Add/remove = insert/delete names what changes between the two representations: in the presence world, state transitions are structural moves between tables, not flag flips.

Solutions: 17 — Presence replaces flags

Exercise 1 — Add a hungry table

import numpy as np

class World:
    def __init__(self, n):
        self.energy = ...                                 # existing
        self.ids    = np.arange(n, dtype=np.uint32)
        self.hungry = np.empty(0, dtype=np.uint32)        # the new presence table

Empty array. No bool column, no flag, no boolean attribute. The world starts with zero hungry creatures.

Exercise 2 — Populate it

HUNGER_THRESHOLD = 10.0

def classify_hunger(energy: np.ndarray, ids: np.ndarray) -> np.ndarray:
    return ids[energy < HUNGER_THRESHOLD]

One numpy line. The system’s read-set is energy and ids; the write-set is whatever the caller assigns the result to. Each tick:

world.hungry = classify_hunger(world.energy, world.ids)

Exercise 3 — Build the flag version

is_hungry = np.zeros(N, dtype=bool)              # one byte per creature

def classify_flag(energy, is_hungry):
    is_hungry[:] = energy < HUNGER_THRESHOLD     # in-place, broadcasts

Same data, parallel column shape. Length N regardless of how many are actually hungry; one byte per creature wasted on the false ones.

Exercise 4 — Build the AoS version

from dataclasses import dataclass

@dataclass(slots=True)
class Creature:
    energy: float
    is_hungry: bool = False

creatures = [Creature(float(e), False) for e in energy]

def classify_aos(creatures):
    for c in creatures:
        c.is_hungry = c.energy < HUNGER_THRESHOLD

The Python tutorial canonical version. Every consumer of “is this creature hungry” reads c.is_hungry and pays for getattr on every access.

Exercise 5 — Time all three at 1M creatures

classify presence:  1.41 ms  (100K hungry of 1M)
classify flag:      0.05 ms
classify AoS:      13.1  ms

Two surprises:

• Flag is faster than presence at the classification step. Building the boolean mask alone is cheap; building the list of ids that pass the mask needs an extra pass to materialise the index array. For one-shot classification, the flag column wins.
• AoS is ~10× slower than the worst numpy version. That’s the cost of the per-element interpreter loop, exactly as §13 promised.

The presence advantage shows up downstream — at the consumer step, not the classifier. Next exercise.

Exercise 6 — The membership query

def is_hungry_p(hungry: np.ndarray, target_id: int) -> bool:
    return bool(np.any(hungry == target_id))         # O(K)

def is_hungry_f(is_hungry: np.ndarray, slot: int) -> bool:
    return bool(is_hungry[slot])                     # O(1)

flag:     ~50 ns
presence: O(K) ms — proportional to len(hungry)

The flag wins for single-creature lookup — direct array indexing is faster than scanning. Presence wins for whole-table operations (count, iterate the hungry set) because there is no scanning of the false rows. The right answer depends on the query pattern; the wrong reflex is to assume one is always faster than the other.

§23 — index maps is the fix that makes presence O(1) for membership too: an id_to_slot array lets you check membership in one read. With the index map, presence beats flag on every operation that matters in the simulator.

Exercise 7 — “How many are hungry?”

count presence:  30 ns         (len(hungry))
count flag:     204 µs         (int(is_hungry.sum()))
count AoS:        10 ms         (sum(1 for c in creatures if c.is_hungry))

Presence is 6800× faster than flag here. Why? len(hungry) is a single Python attribute read on a numpy array — it does not iterate. The flag version has to iterate (sum a million booleans). The AoS version pays for it 50,000× over.

This is where presence pays back. The classification cost is paid once per tick; the count is read by every system that needs to know “how many are hungry?” If even one consumer asks for the count per tick, the presence form pays back its classification overhead instantly. Most simulators have several such consumers (UI display, log entry, decision in the food-spawn policy, etc.).

Exercise 8 — Persist both (stretch)

np.save("is_hungry.npy", is_hungry)               # 1 MB (1 byte × 1M)
np.save("hungry.npy",    hungry)                  # ~400 KB (4 bytes × 100K)

np.savez_compressed("is_hungry.npz", is_hungry)   # ~120 KB (compresses runs of zeros)
np.savez_compressed("hungry.npz",    hungry)      # ~395 KB (already dense)

Uncompressed: presence is 2.5× smaller. Compressed: flag becomes smaller because 90% of its bytes are zeros that compress almost to nothing; presence is essentially incompressible random integers.

This reverses the conclusion at storage time but not at I/O time: writing the flag column requires reading 1 MB of bytes from RAM to compress, while writing the presence array reads 400 KB. In RAM, presence wins; on disk after compression, flag wins (sometimes); at write time, presence wins. Pick the layout that matches your dominant access pattern; persistence is one consideration among several.

For the simulator’s case — frequent in-memory operations, infrequent persistence — presence is the right default. For an archive that’s mostly written once and read rarely, the trade is closer.

18 — Add/remove = insert/delete

Concept node: see the DAG and glossary entry 18.

In the flag world, a state transition is a write. To make a creature hungry, set is_hungry = True. To stop it being hungry, set is_hungry = False. The flag was always there; only its value changed.

In the presence world, a state transition is a move between tables. To make a creature hungry, insert a row into hungry. To stop it being hungry, remove the row. The state has no field to flip; it has only the question of which table the creature is currently a row of.

# flag (canonical Python tutorial)
def become_hungry_flag(is_hungry: np.ndarray, slot: int) -> None:
    is_hungry[slot] = True

# presence
def become_hungry_presence(hungry: list[int], creature_id: int) -> None:
    hungry.append(creature_id)

def stop_being_hungry_presence(hungry: np.ndarray, creature_id: int) -> np.ndarray:
    pos = np.where(hungry == creature_id)[0]
    if pos.size:
        # swap_remove: move last entry into the freed slot, drop last
        hungry[pos[0]] = hungry[-1]
        return hungry[:-1]
    return hungry

“But I just set the bool, what’s the problem?”

The Python idiom that this chapter is asking you to abandon is older and more universal than is_hungry. It is creature.alive = False — the soft delete. Every Python tutorial that introduces classes teaches it: when a thing should stop being processed, set a bool, and check that bool before processing it. Tens of thousands of production codebases run on exactly this pattern.

The cost is real. From code/measurement/alive_fraction.py, one motion update over 1,000,000 creatures at varying alive-fraction:

Read the rows. At 1% alive — the typical case for a transient state like “hungry,” “dying,” or “just-spawned” — presence is 10× faster than the bool-mask version, and 150× faster than the AoS version. As alive-fraction climbs, the gap closes; around 80-90% alive the bool mask starts winning, and at 100% alive it is faster (numpy spots the all-True mask and uses a contiguous slice path instead of fancy indexing).

The AoS column is flat at 25-35 ms regardless of alive-fraction. The interpreter is iterating all one million creatures and paying the getattr(c, "alive") cost on every one, even when 99% of them are skipped a moment later. The “soft delete” pattern saves the actual work but never escapes the per-element dispatch tax.

The honest reading of the table: presence is the right default for transient state (low alive-fraction, the common case for hungry/dying/sleeping-and-soon-to-wake); bool masks are the right default for near-universal state (alive ≥ 90%); AoS is wrong at every alive-fraction. There are no scale ranges where the interpreter loop wins.

Note — “Alive” generalises further than this chapter uses it. In an MMORPG, the relevant set of creatures is the ones inside the player’s render radius — and the radius itself can shrink dynamically when CPU is tight, trading visible-creature count against the tick-budget headroom from §4. The presence table is a query, not a metaphysical state; its entries change when the system asks a different question. “Alive,” “hungry,” “in-scope,” “subscribed,” “active-this-frame” — same shape, different question. The crossover numbers above apply to whichever question your simulation is asking, with whichever fraction the answer happens to have.

Two consequences worth naming

The transition is structural. When a creature crosses the hunger threshold, a row in hungry actually appears or disappears. There is no in-place mutation; the table grows by one or shrinks by one. This is why §22 (mutations buffer; cleanup is batched) exists — adds and removes during a tick must be queued, then applied at the boundary, so that the iteration in progress does not see half the change. The deferred-cleanup pattern is born in this section.

The vocabulary disappears. There is no set_hungry(True), no set_hungry(False), no is_hungry() accessor pair. There is become_hungry (insert) and stop_being_hungry (remove), and even those are usually inlined into the system that detects the transition. The data-oriented program does not have getters and setters; it has systems that move rows between tables. No @property. No __setattr__ hooks. No “validation lives on the model” decorators. The system that detects the threshold is the validation, is the transition, is the audit trail.

A useful test: can you describe the transition without naming a bool? “This creature became hungry” — well, did anything change? Yes: the hungry table grew by one entry. “This creature stopped being hungry” — the table shrank by one entry. Every state change in the system has a structural counterpart, and the structural counterpart is the canonical description.

Multi-table transitions

The same pattern handles richer transitions. Imagine a creature that can be hungry, sleepy, or dead. Three tables: hungry, sleepy, dead. A creature transitions by moving between them. Becoming sleepy while hungry adds a row to sleepy (it can be in both). Dying removes the creature from hungry and sleepy (cleanup affects all relevant presence tables) and adds to dead. The transition is a multi-table operation, but each table is still just a numpy array of ids.

This shape — state changes as inserts and removes — is the precondition for everything else EBP gives you. The dispatch in §19 iterates over the table directly, so the table’s contents being the canonical state of the world is structurally necessary. There is no flag to consult; there is only what is in the table right now.

Exercises

1. Hunger transitions. Use your hungry table from §17. Each tick: read energy; for any creature that crossed below the threshold, append to a hungry_to_add buffer; for any that crossed back above, append to a hungry_to_remove buffer; apply both at the tick boundary. Run for 100 ticks with energy varying randomly; verify hungry always contains exactly the creatures whose current energy is below threshold.
2. Run the alive-fraction exhibit. uv run code/measurement/alive_fraction.py. Note the crossover row — the alive-fraction at which the bool mask starts beating presence. Note that the AoS column does not have a crossover; it loses at every fraction.
3. No bool, no setter. Search your code for any boolean field on a creature. Replace it with a presence table. The setter and getter both disappear. Search for any @property decorator that wraps a state field; same fate.
4. A second presence state. Add a sleepy table. A creature is sleepy if its energy is high enough that it does not need to eat right now. A creature can be in both sleepy and hungry? No — by definition the conditions are mutually exclusive. (Or: design them so they are.) Verify the invariant by checking after each tick that np.intersect1d(hungry, sleepy).size == 0.
5. Death. Add a dead table. When a creature’s energy drops below zero, append to dead and remove from hungry (and from sleepy if present). The cleanup logic is now multi-table; introduce a small transition_to_dead(ids, hungry, sleepy, dead) helper that handles all the affected presence tables.
6. The transition log. Add events: list[tuple[int, int, str]] (tick number, creature id, event name). Every insert/remove emits a row. After 100 ticks, the events log is the canonical history — every state change recorded. This is a preview of §37 — The log is the world.
7. (stretch) Reconstruct from the log. Given only the events log and the initial creature ids, reconstruct the final hungry, sleepy, and dead tables. The reconstruction is a one-shot replay; if it produces the same tables as the live simulation, your transitions are correctly captured.
8. (stretch) The crossover, on your machine. Re-run the exhibit varying alive-fraction more finely between 70% and 95% — say at 70, 75, 80, 85, 90, 95%. Find the alive-fraction at which mask and presence cross over on your hardware. The exact crossover depends on cache size, branch predictor, and the specific numpy build.

Reference notes in 18_add_remove_insert_delete_solutions.md.

What’s next

§19 — EBP dispatch names the dispatch shape that the table-membership representation makes free.

Solutions: 18 — Add/remove = insert/delete

Exercise 1 — Hunger transitions

import numpy as np

THRESHOLD = 10.0

def classify_transitions(prev_hungry, energy, ids):
    """Return (to_add, to_remove) for the hungry presence table."""
    is_hungry_now  = energy < THRESHOLD
    was_hungry     = np.zeros(len(energy), dtype=bool)
    if prev_hungry.size:
        # mark slots that were in prev_hungry — assumes ids[i] == i (dense table)
        was_hungry[prev_hungry] = True
    just_became   = ids[ is_hungry_now & ~was_hungry]
    just_recovered = ids[~is_hungry_now &  was_hungry]
    return just_became, just_recovered

def apply_hunger_changes(hungry: np.ndarray,
                          to_add: np.ndarray,
                          to_remove: np.ndarray) -> np.ndarray:
    if to_remove.size:
        hungry = hungry[~np.isin(hungry, to_remove)]
    if to_add.size:
        hungry = np.concatenate([hungry, to_add])
    return hungry

# Per-tick
to_add, to_remove = classify_transitions(world.hungry, world.energy, world.ids)
# (events are batched; apply at tick boundary, §22)
world.hungry = apply_hunger_changes(world.hungry, to_add, to_remove)

# Invariant after the tick:
assert set(world.hungry.tolist()) == set(world.ids[world.energy < THRESHOLD].tolist())

The invariant check verifies the table’s contents match the predicate at the end of every tick. A simulator that respects this invariant has correctly implemented the structural transition.

Exercise 2 — Run the alive-fraction exhibit

uv run code/measurement/alive_fraction.py

 alive %    AoS (ms)   mask (ms)   presence (ms)    mask/presence
-----------------------------------------------------------------
    1.0%        8.85       0.696           0.070             9.9×
   10.0%       17.48       3.908           0.607             6.4×
   50.0%       23.13       9.718           2.438             4.0×
   90.0%       31.19       3.512           4.559             0.8×
  100.0%       32.80       1.518           4.928             0.3×

The crossover is somewhere between 50% and 90% alive — at 50% presence is 4× faster, at 90% mask is 1.3× faster. The exact crossover depends on hardware (next exercise).

The AoS column has no crossover. At every alive-fraction it loses to both numpy versions by 5-50×. The interpreter loop is paying the per-creature dispatch tax regardless of how few creatures actually need work.

Exercise 3 — No bool, no setter

A typical search shows fields like:

class Creature:
    is_hungry: bool = False
    is_alive:  bool = True
    is_visible: bool = True
    is_in_combat: bool = False

After the refactor:

class World:
    hungry:    np.ndarray = np.empty(0, dtype=np.uint32)
    visible:   np.ndarray = np.empty(0, dtype=np.uint32)
    in_combat: np.ndarray = np.empty(0, dtype=np.uint32)
    # `alive` becomes `live_count` + the implicit "all rows up to live_count are alive"

@property decorators that wrap state fields disappear too. The “validation” they encoded becomes part of the system that detects the transition — the system that causes a row to enter in_combat is also the only place where the validity of “this entity entered combat” gets checked. There’s no separate setter to wrap.

The vocabulary shrinks. creature.set_hungry(True) is replaced by whatever system produced the threshold crossing appending to hungry_to_add. There is no setter; there is a transition.

Exercise 4 — A second presence state

SLEEP_THRESHOLD = 80.0   # high energy → sleepy (won't need to eat)
HUNGER_THRESHOLD = 10.0

def classify_states(energy, ids):
    hungry = ids[energy < HUNGER_THRESHOLD]
    sleepy = ids[energy >= SLEEP_THRESHOLD]
    return hungry, sleepy

# Invariant: a creature cannot be in both
hungry, sleepy = classify_states(energy, ids)
assert np.intersect1d(hungry, sleepy).size == 0

The mutual exclusion is structural (the predicate ranges don’t overlap) — energy < 10 and energy >= 80 cannot both hold. If the predicates could overlap (e.g., is_hungry and is_running), one option is to enforce mutual exclusion in the apply step; another is to allow a creature to appear in both tables and let the dispatch code in §19 handle the overlap explicitly.

Exercise 5 — Death

def transition_to_dead(dying_ids: np.ndarray,
                      hungry: np.ndarray,
                      sleepy: np.ndarray,
                      dead:   np.ndarray):
    """A multi-table transition. Removes from all 'live state' tables, adds to dead."""
    new_hungry = hungry[~np.isin(hungry, dying_ids)]
    new_sleepy = sleepy[~np.isin(sleepy, dying_ids)]
    new_dead   = np.concatenate([dead, dying_ids])
    return new_hungry, new_sleepy, new_dead

dying = world.ids[world.energy < 0]
world.hungry, world.sleepy, world.dead = transition_to_dead(
    dying, world.hungry, world.sleepy, world.dead
)

A multi-table transition is one helper, not three independent updates. The helper is the audit trail: any change to the affected tables goes through it. If you later add a frozen table, you add it to the helper signature in one place. No place outside the helper writes to these tables — the §25 ownership-of-tables discipline at the multi-table scale.

Exercise 6 — The transition log

events: list[tuple[int, int, str]] = []          # (tick, creature_id, event_name)

def log_transitions(events, tick, to_add_hungry, to_remove_hungry):
    for cid in to_add_hungry.tolist():
        events.append((tick, cid, "became_hungry"))
    for cid in to_remove_hungry.tolist():
        events.append((tick, cid, "stopped_being_hungry"))

# After 100 ticks, the events list is the canonical history of state transitions.
print(f"events captured: {len(events)}")
print(f"first 5: {events[:5]}")

Every state transition is now a row in the events log. The current state of hungry is equivalent to the sequence of became_hungry and stopped_being_hungry events applied in order. This equivalence is the §37 log-is-world claim — once you have it, replay, audit, and rollback all become projections of the log.

For a real simulator the events would be stored in numpy columns (timestamp, creature_id_int, event_kind_int) — see the simlog reference. For the exercise, a Python list is fine; converting at end-of-run is cheap.

Exercise 7 — Reconstruct from the log (stretch)

def replay(events: list[tuple[int, int, str]]) -> dict[str, set[int]]:
    """Reconstruct the live tables from an event log."""
    tables = {"hungry": set(), "sleepy": set(), "dead": set()}
    for _, cid, name in events:
        if name == "became_hungry":          tables["hungry"].add(cid)
        elif name == "stopped_being_hungry":  tables["hungry"].discard(cid)
        elif name == "became_sleepy":         tables["sleepy"].add(cid)
        elif name == "stopped_being_sleepy":  tables["sleepy"].discard(cid)
        elif name == "died":
            tables["hungry"].discard(cid)
            tables["sleepy"].discard(cid)
            tables["dead"].add(cid)
    return tables

# Compare with the live simulator
live_tables = {
    "hungry": set(world.hungry.tolist()),
    "sleepy": set(world.sleepy.tolist()),
    "dead":   set(world.dead.tolist()),
}
assert replay(events) == live_tables

If the assertion holds, the events captured every transition. The event log is now the canonical state; the in-memory tables are the projection. Snapshots of the tables become a performance optimisation (reading the snapshot is faster than replaying from t=0); the truth is the log.

This is exactly the architecture of every event-sourced system, every database WAL, every blockchain.

Exercise 8 — The crossover, on your machine (stretch)

Finer sweep between 70% and 95%:

# add to the SHARES list in alive_fraction.py:
SHARES = [0.01, 0.10, 0.50, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00]

Expected pattern: presence and mask cross somewhere between 75% and 90% on most modern machines. The exact crossover varies by:

• Cache size — bigger L2/L3 means the bool mask stays warm at higher fractions.
• Memory bandwidth — more bandwidth helps the mask version (which reads more bytes).
• Branch predictor quality — modern predictors handle the regular branches in the bool sum well; older CPUs were worse at it.

The point of the exercise is not to memorise a number. The point is that the right layout depends on your alive-fraction and your hardware. Measure, then choose. Defaulting to presence (the chapter’s stance) is right for transient state; defaulting to bool masks is right for near-universal state. Both happen to be correct on a wide range of hardware, just at different fractions.

19 — EBP dispatch

Concept node: see the DAG and glossary entry 19.

A system that needs to act on hungry creatures has two ways to find them.

Filtered iteration. Walk all creatures; for each, ask “is it hungry?”; do work if yes:

for slot in range(len(creatures)):
    if is_hungry[slot]:
        drive_hunger_behaviour(slot)

Existence-based dispatch. Walk the hungry table directly; do work for every entry:

for creature_id in hungry:
    drive_hunger_behaviour(creature_id)

In numpy, both shapes lift to one bulk operation:

# filtered (mask-based)
energy[is_hungry] -= HUNGER_BURN_RATE * dt

# EBP (presence-based)
energy[hungry] -= HUNGER_BURN_RATE * dt

The two produce the same result. The two have very different costs.

The filtered version evaluates is_hungry for every creature — a 1,000,000-byte scan to find the 100,000 hungry ones. The EBP version reads the 100,000 entries of hungry and indexes directly. From code/measurement/alive_fraction.py (the §18 exhibit), at 10% sparsity the presence version was 5× faster than the bool mask version, and at 1% it was 10× faster. Most simulator states are sparse — a small fraction of creatures are eating at any given tick, a small fraction are reproducing, a small fraction are dying — so EBP’s compounding advantage shows up everywhere.

A useful intuition: it is the difference between a wandering shopper trying to remember what they need and a shopper with a list. The list version is shorter, faster, and correct by construction. You do not consult the list to ask “is this aisle on my list?” — you walk down the list and visit each aisle once.

Three Python anti-shapes that collapse to “filtered iteration”

Python tutorials teach several patterns that all amount to filtered iteration. Each looks different on the page; they all consult a per-entity predicate instead of walking a presence table.

1. isinstance chains. When entities are modelled as a class hierarchy — Hungry(Creature), Sleepy(Creature), Dead(Creature) — dispatch usually walks one big list:

# anti-pattern: bad!
for entity in entities:
    if isinstance(entity, Hungry):
        drive_hunger(entity)
    elif isinstance(entity, Sleepy):
        drive_sleep(entity)
    elif isinstance(entity, Dead):
        # nothing to do
        pass

The list contains every entity; the body asks the type-tag predicate per entity. The presence-table version splits this into three independent systems, each iterating its own table.

2. Polymorphic method dispatch. The “more Pythonic” version uses dynamic dispatch:

# anti-pattern: bad!
for entity in entities:
    entity.update(dt)

Where Creature.update is overridden in Hungry, Sleepy, Dead. The if/elif is gone from the source code; it has been hidden inside Python’s method resolution order. Every iteration still pays an attribute lookup, an MRO walk, and a function-call setup. The predicate is now invisible but it is still being consulted per entity, and the cache penalty for jumping into a different method body for each subclass type is real. EBP replaces this with three explicit functions, each over its own table.

3. List-comprehension filters. The Pythonic functional-flavoured version:

# anti-pattern: bad!
hungry_creatures = [c for c in creatures if c.is_hungry]
for c in hungry_creatures:
    drive_hunger(c)

This looks like EBP — there is a list of just the hungry ones — but the list was built by scanning all N creatures and allocating a fresh Python list with K pointers. The filter pass is the same cost as the filtered-iteration version, plus a list allocation. EBP avoids the scan because the presence table was kept up to date as state transitions happened (§18); reads do not have to recompute it.

All three anti-shapes consult the predicate at iteration time. EBP arranges the world so the predicate has already been answered before the system runs — the table itself is the answer.

What EBP looks like as a system

A system that uses EBP looks like:

def drive_hunger(hungry: np.ndarray,
                 energy: np.ndarray,
                 id_to_slot: np.ndarray,
                 dt: float) -> None:
    """Read-set: hungry, id_to_slot.
       Write-set: energy (only at slots indexed by hungry)."""
    slots = id_to_slot[hungry]
    energy[slots] -= HUNGER_BURN_RATE * dt

Read-set declared. Write-set declared. No per-row branch; the table is the dispatcher. The signature is the contract — exactly the system shape from §13. EBP is not a separate idea; it is the natural shape that a system takes when its inputs are presence tables.

EBP also composes cleanly with parallelism. A million creatures with 100,000 hungry can be split across multiple processes — each takes a slice of hungry and does its work. The processes never need to consult creatures that are not hungry; their reads do not interfere. §31 develops this under multiprocessing + shared_memory.

The takeaway: EBP is the dispatch that falls out of §17’s presence-replaces-flags substitution. You do not need to choose to use EBP — once your state is in presence tables, every system naturally iterates them. The filtered-iteration version does not even arise.

Exercises

1. Re-read your alive-fraction numbers. From §18 exercise 2 you have measurements for AoS, bool mask, and presence at five alive-fractions. The same numbers tell the EBP story: the presence column is the EBP dispatch path. Confirm by mapping the §18 row labels to the §19 vocabulary — “presence” = “EBP,” “bool mask” = “filtered iteration.”
2. Implement both, on creatures. Implement drive_hunger_filtered(creatures, is_hungry, dt) (walks creatures, checks the bool column, applies the burn) and drive_hunger_ebp(hungry, energy, id_to_slot, dt) (walks the presence table). Run both on a 1M-creature world with 10% hungry. Time both with timeit. Note the ratio.
3. The isinstance trap. Build a list[Creature] where some are Hungry(Creature), some are Sleepy(Creature), some are plain Creature. Implement dispatch via if isinstance(c, Hungry) chains. Time it at 1M creatures with 10% Hungry. Now implement the EBP version: three numpy presence tables, three system functions. Time it. The ratio is the cost of consulting the predicate per entity.
4. The polymorphic-method trap. Convert exercise 3 to class Hungry(Creature): def update(self): ... and a single for c in creatures: c.update(). Time it. Note that the source-code complexity fell (the if/elif is gone), but the runtime cost did not — the predicate moved into Python’s method resolution order, where it is still consulted on every iteration.
5. The list-comprehension filter. Implement hungry = [c for c in creatures if c.is_hungry] followed by for c in hungry: drive(c). Time it. Compare against EBP. Note that the filter pass is the cost of the filtered-iteration version plus a list allocation; the EBP version pays neither, because the hungry table was maintained at state-transition time, not at read time.
6. A multi-state system. A creature can be in any combination of hungry, sleepy, dead. Write three EBP systems: drive_hunger, drive_sleep, drive_death. Each iterates only its own presence table. Compare with a single filtered loop that handles all three with if/elif. Note that the EBP version has no shared state between the three systems and could trivially run them in parallel (§31).
7. (stretch) A naive EBP bug. A system that iterates hungry while also calling hungry.append on the table corrupts iteration. (You knew this from §9 and §15.) Construct a small case that demonstrates the bug — a creature that “becomes hungry” mid-iteration. Then fix it via deferred cleanup: write to to_become_hungry, apply at tick boundary.

Reference notes in 19_ebp_dispatch_solutions.md.

What’s next

§20 — Empty tables are free names the consequence at scale: cost is proportional to active rows, not to population.

Solutions: 19 — EBP dispatch

Exercise 1 — Re-read your alive-fraction numbers

The §18 alive-fraction exhibit is the EBP-vs-filtered comparison:

At 1% sparsity (typical for transient state): EBP is 10× faster than the filtered numpy version, 150× faster than the AoS version. As sparsity rises, the EBP advantage shrinks; at 100% live the bool mask wins because the “filter” is a no-op.

The takeaway: EBP is the right default for sparse states; bool masks are the right default for near-universal states. Both happen to be correct on a wide range of hardware; AoS is wrong at every fraction.

Exercise 2 — Implement both, on creatures

import numpy as np, timeit

n = 1_000_000
rng = np.random.default_rng(0)
energy = rng.uniform(0, 100, n).astype(np.float32)
ids = np.arange(n, dtype=np.uint32)
hungry = ids[energy < 10]                            # 10% sparsity
is_hungry = energy < 10
HUNGER = 0.5
dt = 1/30

def filtered(energy, is_hungry, dt):
    energy[is_hungry] -= HUNGER * dt

def ebp(energy, hungry, dt):
    energy[hungry] -= HUNGER * dt

t_f = timeit.timeit(lambda: filtered(energy.copy(), is_hungry, dt), number=200) / 200
t_e = timeit.timeit(lambda: ebp(energy.copy(), hungry, dt),         number=200) / 200
print(f"filtered: {t_f*1e6:.0f} µs   EBP: {t_e*1e6:.0f} µs   ratio: {t_f/t_e:.1f}×")

filtered: 2756 µs   EBP: 421 µs   ratio: 6.5×

At 10% sparsity, EBP is 6.5× faster on this machine. The filtered version reads is_hungry in full (1M bytes scanned) plus energy at the masked positions. The EBP version reads only hungry (the K = 100K hungry indices, 400 KB) plus energy at those positions. EBP’s working set is 90% smaller.

Exercise 3 — The isinstance trap

from dataclasses import dataclass

@dataclass(slots=True)
class Creature: energy: float
class Hungry(Creature): pass
class Sleepy(Creature): pass

# build a 1M list with three types mixed
ents = []
for i in range(n):
    e = float(energy[i])
    if e < 10:    ents.append(Hungry(e))
    elif e > 80:  ents.append(Sleepy(e))
    else:         ents.append(Creature(e))

def isinstance_dispatch(ents, dt):
    for e in ents:
        if isinstance(e, Hungry):
            e.energy -= HUNGER * dt

t_i = timeit.timeit(lambda: isinstance_dispatch(ents, dt), number=3) / 3
print(f"isinstance chain: {t_i*1e3:.1f} ms")

isinstance chain: 32.4 ms

At 1M entities with 10% Hungry, the isinstance chain costs 77× more than EBP (32.4 ms vs 0.42 ms). The cost is not the isinstance call alone — it’s per-entity interpreter dispatch plus isinstance, plus getattr(e, "energy"), plus the attribute write back to a heap-allocated object. Predicate-per-entity is the structural cost; isinstance is its idiomatic embodiment.

Exercise 4 — The polymorphic-method trap

class Creature:
    __slots__ = ("energy",)
    def __init__(self, e): self.energy = e
    def update(self, dt): pass

class Hungry(Creature):
    def update(self, dt):
        self.energy -= HUNGER * dt

# rebuild ents with subclass instances
ents = [Hungry(e) if e < 10 else Creature(e) for e in (float(x) for x in energy)]

def polymorphic(ents, dt):
    for e in ents:
        e.update(dt)

t_p = timeit.timeit(lambda: polymorphic(ents, dt), number=3) / 3
print(f"polymorphic dispatch: {t_p*1e3:.1f} ms")

Typical: ~50-80 ms. The source-code branching disappeared (no if isinstance in the loop body), but the cost moved into Python’s method resolution. Each call:

1. Looks up update via the MRO chain (one for Creature, one for Hungry).
2. Sets up a Python frame for the method call.
3. Dispatches to a different code path depending on the runtime type — a cache miss every time the type changes.

The “cleaner code” form is more expensive than the visible-branch form — the predicate is consulted as often, and each consultation is more work than isinstance.

Exercise 5 — The list-comprehension filter

def list_comp_dispatch(creatures, dt):
    hungry_list = [c for c in creatures if isinstance(c, Hungry)]      # filter pass
    for c in hungry_list:                                              # work pass
        c.energy -= HUNGER * dt

# Two passes: one to filter, one to work. Plus a list allocation.

The cost is the filtered-iteration baseline plus the list allocation. At 1M entities with 10% hungry, expect ~30-40 ms — comparable to the isinstance chain, with extra allocation pressure.

The shape looks like EBP (a list containing only the hungry ones). The difference is when the filtering happens. EBP’s hungry table is built when the transition occurs (energy crosses below threshold) — once per creature per state change. The list-comp form rebuilds it every read — once per query, on the entire population.

For a simulator with multiple consumers of “the hungry creatures” per tick, this gap compounds: EBP pays 1× the build cost, list-comp pays K× (K = number of consumers).

Exercise 6 — A multi-state system

hungry = ids[energy < 10]
sleepy = ids[energy > 80]
dead   = ids[energy < 0]

def drive_hunger(hungry, energy, dt):
    energy[hungry] -= HUNGER * dt

def drive_sleep(sleepy, energy, dt):
    pass     # sleepy creatures are at rest; no energy change

def drive_death(dead, world):
    world.live_count -= len(dead)

# Each system reads its own table. Disjoint write-sets where possible.

Three EBP systems, three independent write-sets:

• drive_hunger reads hungry, writes energy[hungry slots]
• drive_sleep reads sleepy, writes nothing (or a separate rest_log)
• drive_death reads dead, writes world.live_count (or to_remove)

Now compare to the filtered alternative:

def drive_all_filtered(creatures, dt):
    for c in creatures:
        if c.is_hungry:    c.energy -= HUNGER * dt
        elif c.is_sleepy:  pass
        elif c.is_dead:    c.live = False

The filtered version is one loop with three shared write-sets (energy, live, etc.). The three EBP systems can run in parallel; the filtered loop cannot, because all three branches write through the same Python list.

The §31 multiprocessing pattern is the same systems, run on disjoint slices of hungry. The filtered version cannot be split that cleanly because the consumer can’t tell, before reading each creature, which branch it will take.

Exercise 7 — A naive EBP bug (stretch)

hungry = list(np.arange(5, dtype=np.uint32))     # five creatures hungry
energy = np.array([5.0, 8.0, 3.0, 1.0, 7.0], dtype=np.float32)

# anti-pattern: bad! mutating hungry while iterating it
for cid in hungry:
    energy[cid] -= 1
    if energy[cid] < 2:                          # crossed a deeper threshold
        hungry.append(cid + 100)                 # also become *very_hungry*

The bug: the for loop’s iteration is over hungry’s state at iteration start; appending to hungry mid-iteration may or may not extend the iteration depending on the iterator’s implementation. With a Python list, appending does extend the iteration; with a generator over a numpy slice, it does not. Either way, the behavior is fragile — and reasoning about which creatures end up processed depends on the iteration’s implementation detail.

The fix is the deferred-cleanup pattern from §15:

to_add: list[int] = []
for cid in hungry:
    energy[cid] -= 1
    if energy[cid] < 2:
        to_add.append(cid + 100)

# After the iteration completes, apply the queued changes
hungry.extend(to_add)

The iteration sees a consistent snapshot. Mutations are queued and applied at a clear boundary. This is exactly the §15 and §22 discipline scaling down to a single system.

20 — Empty tables are free

Concept node: see the DAG and glossary entry 20.

If a presence table is empty, the system that iterates it does nothing. No rows, no work. This is the consequence of §19 at the limit, and it is the property that lets the simulator scale gracefully under shifting state.

Concretely: a 1,000,000-creature simulation with zero hungry creatures right now spends zero cycles in drive_hunger. The system is wired into the DAG, runs every tick, takes a numpy array of hungry ids of length 0, executes one bulk op that operates on zero elements, returns. The overhead is one function call and one fancy-index of length zero — measured in microseconds, not milliseconds.

This is not “fast in the empty case as an optimisation”. It is free in the empty case as a structural consequence. The flag-based version runs through the entire creature table even when no flags are set, paying full memory bandwidth to discover that no work is needed. The EBP version is told there is no work by the simple fact of an empty table.

The Python-default failure: Optional fields on every entity

Python’s tutorial reflex when an attribute might be absent is disease: Optional[Disease] = None. Every Creature carries the field; healthy creatures carry None. This looks free — None is a singleton, after all — but every instance still pays one slot, every iteration still pays one getattr, and the storage still scales with population, not with prevalence.

From code/measurement/empty_tables.py, one million creatures with a disease field at four prevalence levels:

Read the 0% row first. With zero diseased creatures, the optional-field layout still costs 105.9 MB of RAM and 7.46 ms per tick of “process disease.” It pays full population price for state that does not exist. The presence layout pays 0.02 ms — function call plus an empty fancy-index — and an extra ~0 KB for the empty diseased array. At zero prevalence, the optional layout is 365× slower than the presence layout, and 4× heavier in memory. The optional layout is not paying for what is happening; it is paying for what might happen.

Read the 10% row. The presence layout pays 0.48 ms — proportional to the 100,000 active rows. The optional layout pays 19 ms — proportional to the full population of one million, because the loop walks every creature to check the is None predicate. The ratio shrinks from 365× to 40× as prevalence rises, but the presence layout always wins, and at typical sparsities (≪ 10% of population is in any specific state at any specific tick) the gap stays large.

The lesson generalises. For every condition you might think of as “optional state” — disease, held_item, target, cooldown_until, aimed_at, fingerprint, last_login_ip, parent_pointer — the disciplined Python form is a separate presence table that contains only the entities that have it right now, not an Optional[X] field on every entity.

Activity-based costs

The effect compounds across many states. A simulation with twenty possible behaviours, each represented as a presence table, pays for the fraction of creatures actually exhibiting each behaviour. Most ticks, most tables are nearly empty. The total work is proportional to the sum of active rows across all tables, not to population × number of behaviours. For a sparsely active world this is one or two orders of magnitude cheaper than the equivalent flag-based design.

A subtle case worth naming: an empty system is not the same thing as a missing system. A drive_hunger system that iterates an empty hungry is still in the DAG, still scheduled, still part of the program’s contract. It is just doing zero rows of work this tick. Removing it from the DAG entirely would change the contract; adding it back when the table next gains a row would require dynamic scheduling, which is harder than a no-op call. EBP gives you cheap idle systems, not absent ones.

Three implications

Activity-based costs. A simulator’s per-tick cost is set by what is active, not by what exists. A million dormant creatures cost nothing to ignore. Only behaving creatures consume budget. Most simulators in production rely on this — game worlds with hundreds of thousands of NPCs but only a few in active play, training simulations with millions of agents but few in critical phases, control systems with thousands of sensors but few in alarmed state.

Structural sparsity. The world is encouraged to be in mostly-resting states. Designs that scatter activity across many small presence tables (lots of cheap idle systems) outperform designs that concentrate activity in a single big “active creatures” flag. The data-oriented mindset is to multiply states (hungry, sleepy, mating, fighting, …) rather than gate behaviour through one master switch.

Persistence is also activity-based. A snapshot of an empty hungry table is one row in the schema and zero rows of data. A snapshot of an is_hungry: np.ndarray of length 1,000,000 is 1 MB regardless of how many bits are set. Backups, replication, and replay all benefit from the same property.

The flag-based mind sees idle objects as “still present, just inactive”. The data-oriented mind sees idle objects as not in the table. The difference is one of cost: the former pays for what exists; the latter pays for what is happening.

Exercises

1. Time the empty case. With your simulator from §19, run a tick where hungry is empty. Time drive_hunger. It should be in the microseconds range — function call plus empty fancy-index, no inner work.
2. Time the same case in flag form. Run the bool-mask version of drive_hunger against a 1,000,000-creature world where is_hungry.sum() == 0. Time it. Should be milliseconds — the mask scan still walks the whole column, even though nothing matches.
3. Run the exhibit. uv run code/measurement/empty_tables.py. Read the 0% row first. Note the absolute cost of the optional layout when nothing is diseased. Note the ratio of optional/presence widening as prevalence drops.
4. The cost-per-active-creature plot. Run the EBP simulator with hungry size ranging over 0, 100, 1,000, 10,000, 100,000, 1,000,000. Time drive_hunger at each. Plot. The line is roughly linear in K, starting at near-zero.
5. Add four more states. Add sleepy, mating, fighting, idle as presence tables, each with its own driver system. Run a tick where most tables are empty (most creatures are in idle). Confirm the per-tick cost is roughly the cost of the idle driver only, plus negligible per-system overhead.
6. Activity histogram. At each tick, log (tick, table_name, len) for every presence table. After 1000 ticks, plot len over time. The plot is the simulator’s activity profile; flat lines mean the world is at rest, bumps mean events are firing.
7. (stretch) Idle systems removed? Argue why removing an empty system from the DAG (rather than running it with zero work) is the wrong move. Hint: it changes the system DAG, breaks determinism if the table is non-empty next tick, and adds dynamic scheduling cost that exceeds the empty-call overhead.
8. (stretch) The Optional[X] sweep. Search any Python project you have. Count Optional[-typed fields on data classes. For each, ask: at runtime, what fraction of instances actually have it set? If the answer is “almost none,” that field is a candidate for a presence table.

Reference notes in 20_empty_tables_are_free_solutions.md.

What’s next

You have closed Existence-based processing. The next phase is Memory & lifecycle, starting with §21 — swap_remove. The simulator is about to start making structural changes to its tables — births and deaths, in production volumes — and the lifecycle phase makes those cheap.

Solutions: 20 — Empty tables are free

Exercise 1 — Time the empty case

import numpy as np, timeit

energy = np.zeros(1_000_000, dtype=np.float32)
hungry = np.empty(0, dtype=np.uint32)            # empty table

def drive_hunger_ebp(energy, hungry, dt):
    energy[hungry] -= 0.5 * dt

t = timeit.timeit(lambda: drive_hunger_ebp(energy, hungry, 1/30), number=10_000) / 10_000
print(f"drive_hunger on empty table: {t*1e6:.2f} µs")

drive_hunger on empty table: ~1-3 µs

A function call, a fancy-index of length zero, an __isub__ on a zero-length view. Microseconds. The system is “in the DAG” but pays almost nothing this tick.

Exercise 2 — Time the same case in flag form

is_hungry = np.zeros(1_000_000, dtype=bool)      # all False — nothing hungry

def drive_hunger_flag(energy, is_hungry, dt):
    energy[is_hungry] -= 0.5 * dt

t = timeit.timeit(lambda: drive_hunger_flag(energy, is_hungry, 1/30), number=1_000) / 1_000
print(f"drive_hunger on all-False mask: {t*1e6:.0f} µs")

drive_hunger on all-False mask: ~150-200 µs

~100× the EBP cost. The mask scan walks all 1M booleans to determine that none are set; numpy still has to materialise the (empty) result of energy[is_hungry]. The “zero work to do” is invisible to the dispatch — the predicate is consulted on every element regardless of the answer.

Exercise 3 — Run the exhibit

uv run code/measurement/empty_tables.py

 prevalence   layout                                  RSS (MB)   tick (ms)
--------------------------------------------------------------------------
     0.00%   list[Creature] with Optional[Disease]      106.4       8.88
     0.00%   numpy SoA + diseased presence              26.6       0.02
     0.10%   list[Creature] with Optional[Disease]      106.3       7.66
     0.10%   numpy SoA + diseased presence              26.7       0.04
     1.00%   list[Creature] with Optional[Disease]      107.1       8.65
     1.00%   numpy SoA + diseased presence              26.8       0.13
    10.00%   list[Creature] with Optional[Disease]      113.8      17.55
    10.00%   numpy SoA + diseased presence              26.5       0.56

The 0% row is the headline: zero diseased creatures, but the optional-field layout costs 8.88 ms per tick to discover this. The presence layout costs 0.02 ms — function-call overhead and an empty fancy-index. The optional layout pays full population price for state that does not exist.

The widening ratio at low prevalence (445× at 0.0%, 191× at 0.1%, 67× at 1%, 31× at 10%) shows that the optional cost is dominated by the iteration, not by the work — the loop walks all 1M creatures regardless of how few have a disease.

Exercise 4 — The cost-per-active-creature plot

import numpy as np, timeit
energy = np.zeros(1_000_000, dtype=np.float32)
results = []
for k in [0, 100, 1_000, 10_000, 100_000, 1_000_000]:
    hungry = np.arange(k, dtype=np.uint32)
    t = timeit.timeit(lambda: drive_hunger_ebp(energy, hungry, 1/30), number=200) / 200
    results.append((k, t * 1e6))
    print(f"K={k:>10}: {t*1e6:>8.1f} µs")

K=         0:       1.5 µs
K=       100:       2.4 µs
K=     1,000:       3.7 µs
K=    10,000:      14.2 µs
K=   100,000:     143.0 µs
K= 1,000,000:    1820.0 µs

Roughly linear in K above ~1000. Below that, the line is dominated by per-call overhead — the work itself disappears into noise. The plot is “y = a + b·K” with a ≈ 1.5 µs (overhead) and b ≈ 1.8 ns (per-active-creature work).

The line starts at near-zero because EBP’s cost depends on K, not N. A flag-based plot would be a flat line at ~150 µs (the mask-scan cost) regardless of K. The two strategies have different shapes.

Exercise 5 — Add four more states

hungry  = ids[energy < 10]
sleepy  = ids[energy > 80]
mating  = np.empty(0, dtype=np.uint32)
fighting = np.empty(0, dtype=np.uint32)
idle    = ids[(energy >= 10) & (energy <= 80)]    # the bulk

def tick(world, dt):
    drive_hunger(world.hungry, world.energy, dt)
    drive_sleep(world.sleepy, world.energy, dt)
    drive_mating(world.mating, world, dt)         # empty — near-zero cost
    drive_fighting(world.fighting, world, dt)     # empty — near-zero cost
    drive_idle(world.idle, world.energy, dt)

If mating and fighting are empty most ticks, the per-tick cost is:

• ~1 µs each for drive_mating and drive_fighting (empty tables)
• The actual work for hungry, sleepy, idle proportional to their sizes

Total: dominated by idle (which holds most of the population) plus small contributions from hungry/sleepy, plus negligible overhead from the empty tables. A simulator can have dozens of dormant systems without paying for them.

Exercise 6 — Activity histogram

activity_log: list[tuple[int, str, int]] = []

for tick_n in range(1000):
    tick(world, dt)
    for name in ("hungry", "sleepy", "mating", "fighting", "idle", "dead"):
        activity_log.append((tick_n, name, len(getattr(world, name))))

import collections
by_table = collections.defaultdict(list)
for t, name, n in activity_log:
    by_table[name].append((t, n))

# plot each name's series — flat lines = resting world, bumps = events

The activity profile is the simulator’s behaviour. A trace where hungry and dead stay flat near 0 means the population is well-fed and stable; bumps mean a food shortage hit; a stairstep up means births are outpacing deaths. The same numbers that drive the per-tick cost are also the simulator’s “vital signs.” Free observability.

Exercise 7 — Idle systems removed? (stretch)

Removing an empty system from the DAG sounds like a free optimisation. It is not. Three reasons:

1. Determinism breaks. The DAG is the contract; a system’s position in the DAG is part of its definition. Run A removes drive_mating because the table is empty; run B (one tick later, after a creature has entered mating) puts it back. The execution order has changed; the world hash changes; replay no longer reproduces.

2. Re-adding it has scheduling cost. When mating next gains a row, the system must be inserted back into the DAG and topo-sorted. Topo-sort is cheap (microseconds for a small DAG) but it is not free, and it pays this cost on every transition between empty and non-empty. The empty-call overhead it was supposed to save was also microseconds. The fix is more expensive than what it fixes.

3. The contract is now dynamic. Static DAG: every run executes the same sequence of systems in the same order. Dynamic DAG: the sequence depends on the run’s state. Reasoning about the simulator (which systems run when, what they read and write, what determinism property holds) becomes much harder. Empty calls are cheap; dynamic schedules are not.

The right move is to keep all systems in the DAG, accept the few microseconds of overhead per empty system per tick, and design states so most are sparse. A simulator with 30 systems and a 30 Hz tick budget can afford 30 µs of empty-call overhead — under 0.1% of the budget.

Exercise 8 — The Optional[X] sweep (stretch)

A quick sweep of any Python project for Optional[-typed fields:

grep -rE 'Optional\[|: [A-Z][a-zA-Z]* \| None|: None \|' src/

For each hit, ask: at runtime, what fraction of instances actually have it set?

• disease: Optional[Disease] — 0-2% of creatures. Strong candidate for a diseased presence table.
• held_item: Optional[Item] — 30-60%. Closer; the trade depends on access pattern. If most systems just need to know whether an item is held, presence wins. If they need the item type, a column might be simpler.
• parent: Optional[Self] — varies. Trees with many leaves and few internal nodes: presence wins. Balanced trees: column wins.
• last_login_at: Optional[datetime] — 99% of users have logged in. Column wins; the Optional wrapper is just defensive coding for the never-logged-in edge case.

The pattern: Optional fields with low fill-rate are presence tables waiting to be discovered. Optional fields with high fill-rate are columns with a sentinel that means “not yet” (a magic timestamp, 255 in a uint8, etc.).

21 — swap_remove

Concept node: see the DAG and glossary entry 21.

The presence-replaces-flags substitution from §17 raised a problem we deferred. When a creature stops being hungry, you remove its id from hungry. When a creature dies, you remove its row from every table. Removing rows from the middle of an array is expensive — every later row has to shift left by one, costing O(N).

For a 1,000,000-creature simulator with 1,000 deaths per tick, naive remove costs roughly 10⁹ moves per tick — far past the budget of any real-time loop.

Python gives you four options. Two are wrong, two are right — and the right two are right in different situations.

Four options, ranked

# anti-pattern: bad!
lst.pop(i)            # O(N) — shifts every subsequent element left
np.delete(arr, i)     # O(N) plus a fresh allocation — usually the slowest

# disciplined: per-element swap_remove with an active counter
arr[i] = arr[n_active - 1]   # move last live element into the freed slot
n_active -= 1                # the "table" is now arr[:n_active]

# disciplined and faster: bulk filter when you have K indices in hand
keep_mask = np.ones(n_active, dtype=bool)
keep_mask[indices_to_remove] = False
arr = arr[keep_mask]         # one C-level pass; survivors keep original order

The mechanism for the per-element version is small: take the last live element, move it into the deleted slot, decrement the active count by one. Two memory writes and a counter decrement. O(1) regardless of N. The “active counter” pattern means you allocate a numpy column once at the maximum size you need, and n_active tells you how many rows are currently in use. The table is the prefix arr[:n_active]. Removing a row never resizes the backing storage; only the counter changes. Inserting a row writes to arr[n_active] and increments. (Insertion details in §24 — Append-only and recycling.)

The bulk-filter version takes a batch of indices and processes them in a single numpy call. It allocates a fresh column of size n_active - K, but pays the allocation only once for the whole batch instead of once per row. It is the natural pair to §22 — Mutations buffer, which is exactly the pattern of “collect K removes during the tick; apply them all at once at the boundary.” The batch is the unit of work; the single numpy call is the application.

The SoA reminder from §6 still applies. Both the per-element swap_remove and the bulk filter are single-column operations as shown above — and a creature table is six or eight columns, not one. Removing creature i is pos_x[i] = pos_x[-1]; pos_y[i] = pos_y[-1]; ...; n_active -= 1 across every column with the same i. The bulk-filter form is the same shape — one keep_mask computed once, applied to every column with the same indices, in lockstep. Apply it to half the columns and rows go out of alignment, exactly the bug from §9. The discipline is the same as it was for sort: every operation that reorders any column reorders all columns of that table together.

Cost, measured

From code/measurement/swap_remove.py, removing 100,000 mid-table rows from a 1,000,000-row table on this machine:

Four readings.

np.delete is the worst. This will surprise readers who reach for it because it sounds like the “numpy way” to remove a row. It is not — np.delete returns a new array with the element removed, allocating fresh memory and copying the surviving elements every call. At 100,000 sequential deletes from a 1M-row array, you allocate 100,000 progressively-shrinking arrays. The bytes are typed, the operation is C-level, and it is still 7,151× slower than the bulk filter because the algorithmic shape is wrong.

list.pop(i) is the AoS middle ground, but only because Python lists are pointer arrays — shifting an N-element list is N pointer copies, which is faster than shifting and reallocating an N-element typed numpy array. Either way: O(N) per remove, 1,129× slower than the bulk filter.

Sequential swap_remove processes 5.5 million removes per second. Each remove is O(1), but the loop that drives it crosses the Python-numpy boundary 100,000 times — one bounds check, one assignment, one n_active -= 1 per iteration. That overhead is the only thing keeping it from being the fastest line in the table.

Bulk filter processes 29.5 million removes per second — 5× faster than sequential swap_remove. The boolean-mask pass and the compress are both single C-level operations over the whole array. The Python interpreter is touched once, not 100,000 times. This is the version the simulator’s cleanup pass should use whenever it has a buffer of indices to remove.

Reading the table together: per-element swap_remove is the right tool when you genuinely have one row to remove (rare). Bulk filter is the right tool when you have a buffer of K indices (the typical case once buffering is in place — §22). Both forms beat the AoS reflexes by orders of magnitude. The choice between them is set by whether the buffering pattern from §22 has happened upstream.

Cost paid

Order is sacrificed. If your code depended on rows being in any particular order, swap_remove reorders them. Two specific consequences:

• Iteration corruption. If you iterate the table and call swap_remove during iteration, the slot you just visited now holds a different row, but your loop counter has moved past it. Half the rows after a swap_remove get skipped or revisited inconsistently. (The same iterate-and-mutate footgun from §15.)
• External references break. Any code holding a slot index into the table now refers to a different row. This is the same bug as §9: rearrangement breaks slot-based references.

Both problems have fixes already named in the book. The iteration corruption is fixed by §22 — Mutations buffer: swap_remove never runs during iteration; it runs during cleanup at the tick boundary, when no system is iterating. The external-reference problem is fixed by §23 — Index maps: an id_to_slot map is updated whenever a row moves, so id-based references survive.

When the lifecycle phase matters

This whole phase — Memory & lifecycle — only matters for variable-quantity tables. Constant-quantity tables like the 52-card deck never grow or shrink, never need swap_remove, never need any of the machinery in this phase. The card game ran for ten chapters without it. The simulator from §11 onward needs all of it, because creatures are born and die every tick.

The constant vs variable distinction is what determines whether a programmer reaches into the lifecycle toolbox at all. Once you have a table whose row count varies at runtime, every tool in this phase becomes load-bearing.

Exercises

1. Compare timings, simple case. Build a list of length 1,000,000. Time 1,000 calls to lst.pop(0) (front delete, the worst case). Time the same with the swap_remove pattern (lst[0] = lst[-1]; lst.pop()). The ratio is roughly N.
2. Mid-table delete. Build a numpy int64 array of length 1,000,000. Time 1,000 calls to np.delete(arr, 500_000) (rebinding arr each time). Time 1,000 calls to the swap_remove pattern (arr[500_000] = arr[n_active - 1]; n_active -= 1). The ratio is enormous — np.delete allocates a fresh array each call.
3. Run the §21 exhibit. uv run code/measurement/swap_remove.py. Note the order of the four rows. Confirm np.delete is the slowest, not the fastest, despite being the “numpy way.” Note the gap between sequential swap_remove and bulk filter — both are O(K) algorithmically, but the bulk version pays the Python-loop overhead once instead of K times.
4. The iteration hazard. Build a numpy int64 array of length 100 with values 0..100 and an n_active = 100. In a forward loop, iterate i in range(n_active) and apply swap_remove whenever arr[i] % 2 == 0. Compare with the expected output (only odd values remaining). What did you actually get? (Spoiler: you missed half the evens.)
5. The fix in one shape: iterate backwards. Repeat exercise 4, but iterate range(n_active - 1, -1, -1). Does it work now? Why does it work?
6. The fix in another shape: deferred cleanup. Repeat exercise 4, but instead of calling swap_remove inside the loop, append the index to to_remove. After the loop, sort to_remove in reverse order and apply swap_remove. This is the §22 pattern in miniature.
7. Aligned per-element swap_remove. Build the simulator’s six creature columns (pos_x, pos_y, vel_x, vel_y, energy, id). Write def delete_creature(world, slot) that calls swap_remove on every column in lockstep. Verify all columns remain aligned after a sequence of deletes.
8. Aligned bulk filter. Take the same six creature columns. Write def delete_batch(world, indices_to_remove) that builds one keep_mask and applies it to every column. Verify alignment by spot-checking row 17 (the (pos_x[17], pos_y[17], ..., id[17]) tuple) before and after the batch — its values should match the original row whose id is now at slot 17. Now write the broken version that applies the mask to only some columns; verify that row alignment is destroyed exactly as §9 predicted. The single-column bulk filter shown in the prose is for clarity; the table version always reads the mask once and uses it everywhere.
9. (stretch) The bandwidth cost. Compute the bytes moved by np.delete(arr, 0) on a 1 GB int64 array: roughly the whole 1 GB (the source array, copied minus the deleted element). Compute the same for the swap_remove pattern: roughly 8 bytes (one int64 move). The ratio is N / 1. Verify with tracemalloc or psutil.

Reference notes in 21_swap_remove_solutions.md.

What’s next

§22 — Mutations buffer; cleanup is batched is the rule that makes swap_remove safe to use: it never runs while any system is iterating.

Solutions: 21 — swap_remove

Exercise 1 — Compare timings, simple case

import time
N = 1_000_000

# pop(0) — worst case
lst = list(range(N))
t0 = time.perf_counter()
for _ in range(1_000):
    lst.pop(0)
t1 = time.perf_counter()
print(f"pop(0) × 1000:       {(t1-t0)*1000:.1f} ms")

# swap_remove at front
lst = list(range(N))
t0 = time.perf_counter()
for _ in range(1_000):
    lst[0] = lst[-1]; lst.pop()
t1 = time.perf_counter()
print(f"swap_remove × 1000:  {(t1-t0)*1000:.3f} ms")

pop(0) × 1000:       380 ms
swap_remove × 1000:    0.2 ms

~2000× ratio. pop(0) shifts every element down one slot — N pointer copies per call, K × N total. swap_remove writes one slot and pops — 2 ops per call, K total. The ratio scales with N.

Exercise 2 — Mid-table delete

import time, numpy as np
N = 1_000_000

# np.delete returns a fresh array each call
arr = np.arange(N, dtype=np.int64)
t0 = time.perf_counter()
for _ in range(1_000):
    arr = np.delete(arr, 500_000 if len(arr) > 500_000 else 0)
t1 = time.perf_counter()
print(f"np.delete × 1000:    {(t1-t0)*1000:.0f} ms")

# swap_remove
arr = np.arange(N, dtype=np.int64)
n_active = N
t0 = time.perf_counter()
for _ in range(1_000):
    i = 500_000 if i < n_active - 1 else 0
    arr[i] = arr[n_active - 1]
    n_active -= 1
t1 = time.perf_counter()
print(f"swap_remove × 1000:  {(t1-t0)*1000:.3f} ms")

np.delete re-allocates the entire array each call — N int64s = 8 MB copied per delete. After 1000 calls, 8 GB has been written. swap_remove writes 8 bytes per call. ~10⁵-10⁶× ratio.

Exercise 3 — Run the §21 exhibit

uv run code/measurement/swap_remove.py

Source: code/measurement/swap_remove.py. Removing 100,000 mid-table rows from a 1M-row table:

layout                                              time      remove rate
-------------------------------------------------------------------------
Python list, list.pop(i)                            3.59 s          28K ops/s
numpy, np.delete(arr, i)                           23.09 s           4K ops/s
numpy active counter, sequential swap_remove       0.011 s        8.0M ops/s
numpy bulk filter, arr[keep_mask]                  0.004 s       25.4M ops/s

Surprises that calibrate intuition:

• np.delete is the slowest — 6500× slower than the bulk filter. The “numpy way” sounds right but is algorithmically wrong: it reallocates on every call.
• Python list pop(i) beats np.delete at this scale, because pointer-shifts in a Python list are ~8 bytes each whereas reallocation copies the whole int64 array.
• Bulk filter is 3× faster than sequential swap_remove, even though both are O(K). The Python loop crossing the C boundary 100K times has measurable overhead; the bulk version pays the boundary cost once.

For a simulator’s cleanup pass: collect to_remove indices during the tick (cheap, append-only), then apply with one bulk-filter call at the boundary. This is the §22 pattern.

Exercise 4 — The iteration hazard

import numpy as np
arr = np.arange(100, dtype=np.int64)
n_active = 100

i = 0
while i < n_active:
    if arr[i] % 2 == 0:
        arr[i] = arr[n_active - 1]
        n_active -= 1
    else:
        i += 1

print(arr[:n_active].tolist())     # should be [1, 3, 5, ..., 99]

Without the else: i += 1 (i.e. plain for i in range(...)), the bug is: after a swap, the slot at i now holds a different value (the one that was at the end). The forward for loop has already moved past i and won’t re-check it. Half the evens get skipped.

The version above with the explicit while and conditional increment is correct: when a swap happens, don’t advance i — the slot has new contents that need re-checking. This is the canonical fix when iterating-while-mutating cannot be avoided.

Exercise 5 — The fix in one shape: iterate backwards

arr = np.arange(100, dtype=np.int64)
n_active = 100
for i in range(n_active - 1, -1, -1):
    if arr[i] % 2 == 0:
        arr[i] = arr[n_active - 1]
        n_active -= 1
print(arr[:n_active].tolist())     # all odds

Why it works: when you swap arr[i] = arr[n_active - 1], the slot at i now holds the old last element, but i is decreasing, so we move to i - 1 next — a slot we have not yet visited. We never re-encounter a swapped slot. The “old last” element gets to be checked for evenness at its new position because that position was never visited by the iteration.

Exercise 6 — The fix in another shape: deferred cleanup

arr = np.arange(100, dtype=np.int64)
n_active = 100

to_remove = []
for i in range(n_active):
    if arr[i] % 2 == 0:
        to_remove.append(i)

# Apply at end — reverse order so swap_remove indices stay valid
for i in sorted(to_remove, reverse=True):
    arr[i] = arr[n_active - 1]
    n_active -= 1

print(arr[:n_active].tolist())

The for loop is now read-only — no swap during iteration. Mutations are buffered and applied at the boundary (for i in sorted(to_remove, reverse=True)). This is the §22 pattern: filter (read-only) and apply (single batch) are separate phases.

For very large to_remove buffers, the bulk-filter form (arr = arr[~np.isin(np.arange(n_active), to_remove)]) is faster than per-index swap_remove. Both forms are correct; the bulk one wins on speed.

Exercise 7 — Aligned per-element swap_remove

class World:
    def __init__(self, n):
        self.pos_x   = np.zeros(n, dtype=np.float32)
        self.pos_y   = np.zeros(n, dtype=np.float32)
        self.vel_x   = np.zeros(n, dtype=np.float32)
        self.vel_y   = np.zeros(n, dtype=np.float32)
        self.energy  = np.zeros(n, dtype=np.float32)
        self.id      = np.arange(n, dtype=np.uint32)
        self.n_active = n

def delete_creature(world: World, slot: int) -> None:
    last = world.n_active - 1
    if slot != last:
        for arr in (world.pos_x, world.pos_y, world.vel_x,
                    world.vel_y, world.energy, world.id):
            arr[slot] = arr[last]
    world.n_active -= 1

Each column gets the same slot and last. Forgetting to apply this to one column produces the §9 misalignment bug. The discipline is: the function above is the only place that does swap_remove on a creature; no caller writes to one column without going through it.

Exercise 8 — Aligned bulk filter

def delete_batch(world: World, indices_to_remove: np.ndarray) -> None:
    keep = np.ones(world.n_active, dtype=bool)
    keep[indices_to_remove] = False
    for name in ("pos_x", "pos_y", "vel_x", "vel_y", "energy", "id"):
        col = getattr(world, name)
        # in-place compress: copy survivors to the front
        n_keep = int(keep.sum())
        col[:n_keep] = col[:world.n_active][keep]
    world.n_active -= len(indices_to_remove)

# spot-check alignment
row17_before = (world.pos_x[17], world.pos_y[17], int(world.id[17]))
delete_batch(world, np.array([5, 13, 87]))
# whichever creature is now at slot 17 — its row tuple should still be coherent
row17_after  = (world.pos_x[17], world.pos_y[17], int(world.id[17]))
# verify that row17_after matches the row in the original world whose id == world.id[17] now

The same keep mask is applied to every column. One boolean indexing pass per column, one mask shared across all columns. Forgetting one column lands the broken version: rows misaligned exactly as §9 predicted.

The broken version (apply mask to half the columns):

# anti-pattern: bad! demonstrates the bug
def delete_batch_broken(world, indices):
    keep = np.ones(world.n_active, dtype=bool)
    keep[indices] = False
    world.pos_x = world.pos_x[keep]
    world.pos_y = world.pos_y[keep]
    # forgot vel_x, vel_y, energy, id — they keep their old length and contents

Now pos_x[i] and vel_x[i] are from different rows. Reading “the velocity of the creature at slot i” returns garbage. The fix is structural: one function, all columns, one mask.

Exercise 9 — The bandwidth cost (stretch)

import numpy as np

# np.delete on a 1 GB int64 array — copies (1 GB - 8 bytes)
arr = np.zeros(1_000_000_000 // 8, dtype=np.int64)   # 125 M elements = 1 GB
arr2 = np.delete(arr, 0)                              # bytes moved ≈ 1 GB

# swap_remove — copies 8 bytes
n_active = len(arr)
arr[0] = arr[n_active - 1]
n_active -= 1                                          # bytes moved = 8

Bytes moved per delete:

Ratio: ~125,000,000×. At a 30 Hz tick rate, doing one np.delete per tick on a 1 GB array would mean moving 30 GB/s — past the bandwidth ceiling of most desktop systems. Doing one swap_remove takes 30 × 8 = 240 bytes per second.

The structural cost dominates the constant factors. This is why the chapter’s table shows np.delete losing to list.pop at this scale: the algorithmic shape is wrong regardless of how typed and contiguous the data is.

22 — Mutations buffer; cleanup is batched

Concept node: see the DAG and glossary entry 22.

This rule has been forward-referenced through ten chapters. Time to make it concrete.

Mutations during a tick do not apply immediately; they queue, and a single cleanup pass applies them all at the tick boundary. The shape:

@dataclass
class CleanupBuffer:
    to_remove: list[int]               # creature ids to delete this tick
    to_insert_pos_x: list[float]       # parallel arrays of inserted-row data
    to_insert_pos_y: list[float]
    to_insert_vel_x: list[float]
    to_insert_vel_y: list[float]
    to_insert_energy: list[float]
    to_insert_id: list[int]

(The insert side has one list per column. Per §6, a row is a tuple-at-index, and that’s true of the insert buffer too — it is an SoA buffer, not a list of Creature objects. The reason is the same reason the rest of the simulator is SoA: numpy gets to work on the bytes when cleanup runs.)

During the tick, every system that wants to delete appends an id to to_remove. Every system that wants to add appends one row’s worth of data to the parallel insert columns. No system mutates the live tables.

The cleanup pass

At the end of the tick, one system runs:

def cleanup(world: World, buffer: CleanupBuffer) -> None:
    # 1. Removals: build one keep_mask, apply to every column at once.
    if buffer.to_remove:
        ids_to_remove = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids_to_remove]              # see §23
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[: keep_mask.sum()] = col[: world.n_active][keep_mask]
        world.n_active = int(keep_mask.sum())
        # (Update id_to_slot — covered in §23.)
        buffer.to_remove.clear()

    # 2. Insertions: bulk concatenate parallel insert columns into the table.
    n_inserts = len(buffer.to_insert_id)
    if n_inserts:
        new_n = world.n_active + n_inserts
        # The columns were sized at maximum capacity at startup; we are
        # writing into the previously unused tail [n_active : new_n).
        world.pos_x[world.n_active : new_n] = buffer.to_insert_pos_x
        world.pos_y[world.n_active : new_n] = buffer.to_insert_pos_y
        world.vel_x[world.n_active : new_n] = buffer.to_insert_vel_x
        world.vel_y[world.n_active : new_n] = buffer.to_insert_vel_y
        world.energy[world.n_active : new_n] = buffer.to_insert_energy
        world.id[world.n_active : new_n] = buffer.to_insert_id
        world.n_active = new_n
        # (Append the new ids to id_to_slot — §23.)
        for lst in (buffer.to_insert_pos_x, buffer.to_insert_pos_y,
                    buffer.to_insert_vel_x, buffer.to_insert_vel_y,
                    buffer.to_insert_energy, buffer.to_insert_id):
            lst.clear()

Two passes, both bulk operations. The world is in a fully consistent state at the end. The keep_mask is built once and applied to every column; the insert tail is filled with one slice assignment per column. Per §21, the bulk-filter form is 5× faster than per-element swap_remove at K=100,000 mutations per tick — and per the editions-diverge framing in the prose of §10 and elsewhere, this is where the Python edition’s cleanup actually diverges from the Rust edition’s: Rust §22 uses a per-element swap_remove loop because compiled code pays no interpreter-boundary tax; Python §22 uses the bulk-mask form because we measured the boundary cost and it dominates at scale.

What this fixes

The iteration-corruption problem from §21 goes away because the table is never mutated while any system is iterating. By the time cleanup runs, every system has finished. There is no concurrent iteration to confuse. The list-during-iteration and dict-during-iteration footguns from §15 cannot happen — there is no creatures.remove(c) inside a for c in creatures loop, because nothing inside the tick mutates the live tables.

The race-condition problem from concurrent mutation goes away. Two systems may both want to remove a creature; both append to to_remove; cleanup deduplicates with np.unique. Neither system needs to coordinate.

The composition problem from §14 goes away. Systems read consistent snapshots; they read the world as it was at tick start, not the world as some other system half-rewrote it.

What it costs

Every mutation is one extra entry pushed to a side list. For a simulator with 1,000 deaths and 500 reproductions per tick, that is 1,500 entries of bookkeeping per tick — a few thousand bytes, completely negligible against the cost of running the systems themselves.

The cleanup pass is one additional system in the DAG. It is empty (no work) when no mutations are queued (§20); it runs the bulk filter and bulk concatenate when there are. The system is wired in once and never removed.

What it does not fix

Dedup is the system’s job. Two systems may both push the same id to to_remove if they independently detect the same death condition. The cleanup uses np.unique(to_remove) to reduce to distinct ids before computing slots. The cost is one O(K log K) sort on a small array — irrelevant against the bulk filter.

Order matters. Inside cleanup, deletions run first, then insertions. If you insert first, an inserted row might land in a slot you are about to delete. Deleting first frees up tail capacity that subsequent inserts can reuse — though slot recycling is its own decision (§24).

The pattern itself is universal. Database transactions buffer writes and commit at the boundary. Graphics pipelines render to a back buffer and swap. Version-controlled file systems collect changes and commit. They all solve the same problem: how do you let many independent operations modify shared state without stepping on each other? The answer is always the same — accumulate, then apply atomically.

Exercises

1. Implement the side buffers. Add to_remove: list[int] and the parallel insert lists (one per column) to your simulator’s world. They are empty at the start of every tick.
2. Push from apply_starve. Modify your starvation system to append to to_remove instead of any direct table mutation. Verify the system no longer touches the live creatures columns.
3. Push from apply_reproduce. Modify reproduction to append the parent’s offspring rows to the parallel insert lists. Verify reproduction no longer mutates creatures directly.
4. Implement bulk cleanup. Write the cleanup system as in the prose. Apply removals first (one keep_mask, applied to every column), then insertions (one slice-write per column). Run a tick with both kinds of mutations; verify the world is consistent after.
5. Compare cleanup forms. Implement a second cleanup that uses per-element swap_remove in a Python loop instead of the bulk mask. Time both at 1,000,000 creatures with 1,000 mutations per tick. The bulk form should win by ~5× per the §21 numbers — confirm on your machine.
6. The dedup question. Push id 42 to to_remove from two different systems in the same tick. Run cleanup without the np.unique step. What happens? (Hint: id_to_slot[42] is looked up twice; the second lookup may produce garbage if the first removal moved another row to that slot.) Now add the np.unique and re-run. The result is correct.
7. Tick-delayed visibility. A creature inserted in tick 5 (via the to_insert_* lists) does not appear in the live columns during tick 5’s systems — only at the end, in cleanup. Verify by adding an age_in_ticks column that increments at the end of each tick; the new creature’s value starts at 0 in tick 6, not tick 5.
8. (stretch) A graphics pipeline analogy. A rendering pipeline draws to a “back buffer” while the “front buffer” is being displayed. At the boundary of one frame to the next, the buffers swap. Argue why this is the same pattern as to_remove / to_insert plus cleanup. (Hint: it is the same atomic-commit shape; the back buffer is exactly the side table.)

Reference notes in 22_mutations_buffer_solutions.md.

What’s next

§23 — Index maps is the missing piece for swap_remove and bulk-filter cleanup to be useful: a parallel data structure that tracks where every id currently lives, updated whenever the columns move.

Solutions: 22 — Mutations buffer; cleanup is batched

Exercise 1 — Implement the side buffers

from dataclasses import dataclass, field

@dataclass
class CleanupBuffer:
    to_remove: list[int] = field(default_factory=list)
    to_insert_pos_x:   list[float] = field(default_factory=list)
    to_insert_pos_y:   list[float] = field(default_factory=list)
    to_insert_vel_x:   list[float] = field(default_factory=list)
    to_insert_vel_y:   list[float] = field(default_factory=list)
    to_insert_energy:  list[float] = field(default_factory=list)
    to_insert_id:      list[int]   = field(default_factory=list)

# tick boundary: clear everything
buffer = CleanupBuffer()

The insert side is parallel column lists, not a list of objects. The whole point of the simulator’s SoA discipline is that “a row to insert” is six values across six lists with the same index — exactly like the live tables, just on the side.

For tighter packing, the insert lists could be pre-allocated numpy arrays with their own n_pending counter; for typical mutation rates (hundreds to thousands per tick), Python lists are plenty fast.

Exercise 2 — Push from apply_starve

def apply_starve(world: World, buffer: CleanupBuffer) -> None:
    """Read-set: world.energy, world.id, world.n_active.
       Write-set: buffer.to_remove (only)."""
    starvers = np.where(world.energy[: world.n_active] <= 0)[0]
    starver_ids = world.id[starvers]
    buffer.to_remove.extend(starver_ids.tolist())

The system does not call world.delete_creature(). It does not modify world.energy or world.n_active. It writes only to buffer.to_remove — the live world is untouched until cleanup. A diff between this version and the previous shows: every line that mutated a live column is gone; one extend line replaces all of them.

Exercise 3 — Push from apply_reproduce

THRESHOLD = 100.0

def apply_reproduce(world: World, buffer: CleanupBuffer, rng) -> None:
    """Read-set: world.energy, world.pos_x, world.pos_y, world.id, world.n_active.
       Write-set: buffer.to_insert_* (only). Parent's energy unchanged here;
                  splitting energy is a separate consideration handled by cleanup
                  or a follow-on system."""
    parents = np.where(world.energy[: world.n_active] > THRESHOLD)[0]
    if parents.size == 0:
        return
    n = parents.size
    # offspring inherit parent pos with tiny jitter
    jitter_x = rng.uniform(-0.1, 0.1, n).astype(np.float32)
    jitter_y = rng.uniform(-0.1, 0.1, n).astype(np.float32)
    new_ids  = world.next_ids(n)                              # see §24

    buffer.to_insert_pos_x.extend((world.pos_x[parents] + jitter_x).tolist())
    buffer.to_insert_pos_y.extend((world.pos_y[parents] + jitter_y).tolist())
    buffer.to_insert_vel_x.extend([0.0] * n)
    buffer.to_insert_vel_y.extend([0.0] * n)
    buffer.to_insert_energy.extend([world.energy[parents].mean()] * n)  # half-energy variant in §13
    buffer.to_insert_id.extend(new_ids.tolist())

Reproduction has no direct effect on the world during the tick. The offspring exist as parallel entries in the buffer lists. Cleanup will materialise them.

Exercise 4 — Implement bulk cleanup

def cleanup(world: World, buffer: CleanupBuffer) -> None:
    # 1. Removals (deletes first so freed slots can host inserts in §24's recycling)
    if buffer.to_remove:
        ids = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids]                     # see §23
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False
        n_keep = int(keep_mask.sum())
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[:n_keep] = col[: world.n_active][keep_mask]
        world.n_active = n_keep
        buffer.to_remove.clear()
        # update id_to_slot — see §23

    # 2. Insertions (one slice-write per column)
    n_inserts = len(buffer.to_insert_id)
    if n_inserts:
        new_n = world.n_active + n_inserts
        world.pos_x[world.n_active : new_n]  = buffer.to_insert_pos_x
        world.pos_y[world.n_active : new_n]  = buffer.to_insert_pos_y
        world.vel_x[world.n_active : new_n]  = buffer.to_insert_vel_x
        world.vel_y[world.n_active : new_n]  = buffer.to_insert_vel_y
        world.energy[world.n_active : new_n] = buffer.to_insert_energy
        world.id[world.n_active : new_n]     = buffer.to_insert_id
        world.n_active = new_n
        # update id_to_slot for new ids — see §23
        for lst in (buffer.to_insert_pos_x, buffer.to_insert_pos_y,
                    buffer.to_insert_vel_x, buffer.to_insert_vel_y,
                    buffer.to_insert_energy, buffer.to_insert_id):
            lst.clear()

Two bulk ops. The world is consistent at the end. Spot-check after a tick:

assert len(set(world.id[: world.n_active].tolist())) == world.n_active     # no duplicates

Exercise 5 — Compare cleanup forms

import time, numpy as np
N, K = 1_000_000, 1_000

# Bulk cleanup: arr[keep_mask]
def bulk_cleanup(arr, indices_to_remove):
    keep = np.ones(len(arr), dtype=bool)
    keep[indices_to_remove] = False
    return arr[keep]

# Per-element swap_remove in a Python loop
def per_element_cleanup(arr, indices_to_remove):
    n = len(arr)
    for i in sorted(indices_to_remove, reverse=True):
        arr[i] = arr[n - 1]
        n -= 1
    return arr[:n]

arr = np.arange(N, dtype=np.int64)
indices = np.random.default_rng(0).choice(N, size=K, replace=False)

t = time.perf_counter()
for _ in range(100):
    bulk_cleanup(arr.copy(), indices)
print(f"bulk:        {(time.perf_counter()-t)*10:.2f} ms / call")

t = time.perf_counter()
for _ in range(100):
    per_element_cleanup(arr.copy(), indices.tolist())
print(f"per-element: {(time.perf_counter()-t)*10:.2f} ms / call")

Typical ratio at K=1000: bulk ~3-5× faster. At K=100,000: bulk ~5-10× faster (the boundary-crossing cost grows linearly with K for the per-element version, while the bulk form pays it once).

The bulk form is the right default for the Python edition. If you find yourself writing a per-element swap_remove loop inside cleanup, consider whether you have a buffer of indices in hand — if you do, use the mask.

Exercise 6 — The dedup question

# anti-pattern: bad! no dedup
buffer.to_remove.append(42)                      # apply_starve appends it
buffer.to_remove.append(42)                      # apply_disease appends it too
# both systems independently noticed creature 42 should die

# cleanup without np.unique:
slots = world.id_to_slot[buffer.to_remove]      # [slot_of_42, slot_of_42] — same slot twice
keep_mask = np.ones(world.n_active, dtype=bool)
keep_mask[slots] = False                         # idempotent — same slot zeroed twice is fine

For removals via mask, dedup happens to be implicit — assigning False to the same index twice is the same as once. So the boolean-mask form is robust to duplicate to_remove entries.

The risk is for per-element swap_remove: removing slot 42 once moves the last row into 42; removing it again moves the new last row into 42, deleting an unintended row. The cleanup function above protects via np.unique regardless of which deletion form is used.

Exercise 7 — Tick-delayed visibility

@dataclass
class World:
    age_in_ticks: np.ndarray = ...
    # ...

def end_of_tick(world):
    """Increment all live ages."""
    world.age_in_ticks[: world.n_active] += 1

# Tick 5: parent reproduces; offspring goes into to_insert with age_in_ticks=0
buffer.to_insert_age_in_ticks.append(0)
cleanup(world, buffer)                            # offspring now in live columns

end_of_tick(world)                                # offspring goes 0 → 1 (counts as full tick of life)

# Tick 6: age_in_ticks of newborn is 1 at start of tick
print(world.age_in_ticks[-1])                     # 1

The offspring did not live a partial tick of tick 5. It became part of the world between tick 5 and tick 6. Tick 6 is its first full tick; end_of_tick on tick 6 makes its age_in_ticks = 2.

Whether the increment happens before or after cleanup is a policy decision. The convention here: increment after cleanup, so newborns start at 0 and reach 1 at the end of their first tick. The choice should be written down once (in the simulator’s contract) and applied consistently.

Exercise 8 — A graphics pipeline analogy (stretch)

A double-buffered renderer:

• Front buffer: the framebuffer the display reads.
• Back buffer: the framebuffer the renderer writes.
• At vsync (the frame boundary), the buffers swap. The display now reads what the renderer just wrote; the renderer starts writing what the display previously had.

Map to the simulator:

The shapes are identical. Both solve “many independent operations want to mutate shared state; how do they not step on each other?” by accumulating in a side buffer and applying atomically at a boundary. Database transactions, version-controlled file systems, audio engines (frame buffers for samples), and real-time-safety control systems (double-buffered set-points) all share this pattern.

A simulator that buffers its mutations is a simulator that has discovered transaction processing without naming it. Once you see the shape, every “atomic commit” boundary in software is a tick-boundary in disguise.

23 — Index maps

Concept node: see the DAG and glossary entry 23.

The presence-replaces-flags substitution from §17 had a sting in its tail. A presence query — “is creature 42 hungry?” — costs O(K) when implemented naively as np.any(hungry == 42). At a 1,000,000-creature simulator with thousands of such queries per tick, that is too slow.

The fix is a parallel data structure: an index map that maps every id to its current slot in the table. Lookup is now O(1).

Python gives you two reasonable shapes for the map, and one trap.

Two shapes that work

A numpy array, when ids are dense. If your ids are integers in [0, N_max) and most are in use, a single typed column does the job:

INVALID = np.iinfo(np.uint32).max  # 4_294_967_295
id_to_slot = np.full(N_max, INVALID, dtype=np.uint32)

def slot_of(id_to_slot: np.ndarray, creature_id: int) -> int | None:
    slot = int(id_to_slot[creature_id])
    return None if slot == INVALID else slot

The sentinel value (np.iinfo(np.uint32).max) marks “no slot — this id has no current row”. 4 MB at 1,000,000 ids; a single C-level memory read per lookup; bulk lookups via fancy indexing (id_to_slot[ids_to_remove]) run at numpy speed and are exactly what cleanup uses (§22). One cache line per 16 ids; cleanup streams through it sequentially.

A dict[int, int], when ids are sparse. If the id space is large but few are in use — id is a hash of a string, an external system’s UUID-as-int, a timestamp truncated to a slot — a Python dict is the right pick:

id_to_slot: dict[int, int] = {}

def slot_of(id_to_slot: dict[int, int], creature_id: int) -> int | None:
    return id_to_slot.get(creature_id)

Dict lookup is O(1) amortised, ~30-40 million ops/sec for integer keys (per code/measurement/float_or_int_tuple.py — note that which integer matters; int keys are 2.4× faster than float-tuple keys at the same map size). Dict pays for hash machinery on every lookup and one pointer chase per access; numpy pays neither. But dict pays only for ids that actually exist, which is the right shape for a sparse id space.

The choice is set by id density, not by taste. The simulator’s surrogate ids from §10 are dense — a fresh integer per creature, recycled when slots are reused. The numpy array is the right pick. An audit log indexed by 64-bit hash would be sparse — the dict is the right pick.

One shape that is wrong

# anti-pattern: bad!
from scipy.sparse import csr_matrix
m = csr_matrix(...)            # built for sparse 2D matrix arithmetic
slot = m[creature_id, 0]        # used here as a 1D point-lookup map

The scipy.sparse family — CSR, CSC, COO — are not index maps. They are sparse-matrix data structures, optimised for matrix-vector products and slicing entire rows or columns. Used for individual point lookups, they are very slow. From code/measurement/csr_matrix or python dict.py at 1,000 × 1,000 with 1% density, a Python dict is roughly 108× faster than CSR at random scalar lookups.

The exhibit’s headline reads “CSR matrix is 108× slower than Python dict.” That is true for the access pattern in the file — and it is the wrong reading. The right reading is: scipy gave you a sparse matrix, not a sparse map. Pick the structure that matches your access pattern. CSR is excellent at SpMV (sparse-matrix-vector-product, the common dense-vector-multiplied-by-sparse-matrix operation in scientific computing). It is poor at point-and-shoot lookups because its internal layout — three indices, indptr, data arrays — is optimised for stride-skipping, not for O(1) random access. The lesson is not “CSR is slow”; it is “wrong tool for this job, every time, by design.”

Maintenance

The map must be updated whenever a row moves. The events that move rows in this book are exactly three:

• Bulk filter cleanup (§22). Every removed slot’s id is set to INVALID. Every surviving id whose slot changed has its entry rewritten — exactly the rows that moved during the keep-mask compress.
• Append. When a new row lands at slot n, set id_to_slot[new_row.id] = n. The cleanup pass writes this in lockstep with the insert tail.
• Sort or reshuffle (for locality, §28). When the table is reordered, every slot moves. The full map is rewritten in lockstep with the sort. In numpy this is one assignment: id_to_slot[ids[order]] = np.arange(n_active).

The cleanup system from §22 is the natural home for these updates. Every removal and every insertion goes through cleanup; cleanup keeps the map in step.

Cost

The numpy array adds one uint32 per id ever issued, including ids that are currently dead but whose slots have not been recycled. For a simulator that issues a million ids over its lifetime but has 100,000 alive at any moment, the map is 4 MB. That is a real cost — bigger than the alive table itself if the table has narrow columns. Mitigations:

• Generational ids (§10) plus a separate id allocator that recycles dead ids bound the map’s size to the high-water mark of live ids, not the total ever issued. With recycling, the map stays at 100,000 × 4 = 400 KB.
• A dict-of-int-to-int trades a constant-factor lookup overhead for tighter memory; useful when ids are sparse, as named above.

For most simulators with recycling, the dense np.ndarray is the right shape. One cache line per 16 ids; the bulk lookup id_to_slot[ids] is bandwidth-bound at numpy speed.

The pattern in the wild

Every ECS engine ships an index map. Bevy’s Entity (Rust) is a 64-bit handle whose unpacking is essentially a slot lookup with a generation check. slotmap’s SlotMap keeps an internal map. Database engines maintain index maps as B-trees over primary keys. The shape — id-to-slot lookup, maintained on every move — is universal.

Combined with §10’s stable ids and §24’s slot recycling, the index map is the third piece of the generational arena — the canonical handle-based data structure in modern systems software.

Exercises

1. Build the map. Add id_to_slot = np.full(N_max, INVALID, dtype=np.uint32) to your simulator. When a creature is appended at slot N, set id_to_slot[id] = N. When a creature’s slot changes during cleanup, update accordingly.
2. O(1) presence query. Add a parallel hungry_membership = np.zeros(N_max, dtype=bool) set to True when an id is in hungry. Now is_hungry(id) is two array lookups, both O(1).
3. Maintain on bulk-filter cleanup. Modify your §22 cleanup to update id_to_slot after the keep_mask compress. The fastest form: after id[: new_n] = id[: n_active][keep_mask], run id_to_slot[id[:new_n]] = np.arange(new_n, dtype=np.uint32) — one bulk write, every surviving id’s slot rewritten in one pass.
4. Time the difference. Rerun the simulator at 1M creatures, calling is_hungry(random_id) 100,000 times per tick. Compare the linear-scan version (§17 exercise 6) and the indexed version. The ratio is roughly N — about a million.
5. Run the exhibit (honestly). uv run "code/measurement/csr_matrix or python dict.py". Read the file’s headline (“CSR matrix is 108× slower”). Then read the chapter’s reframing. Confirm with one small experiment of your own that scipy’s CSR is fast at its job — csr.dot(some_dense_vector) for a 1000×1000 matrix — and slow at the job the file gave it.
6. The bandwidth cost. At 1M ids, id_to_slot is 4 MB. Cleanup’s bulk update on a tick with 1,000 swap_removes and 500 inserts writes ~1,500 entries — 6 KB. Compute the cleanup cost in microseconds for those writes against a 30 Hz budget.
7. Sort-for-locality compatibility. When creatures is sorted (a preview of §28), every slot moves. Rewrite id_to_slot in lockstep with one bulk numpy assignment: id_to_slot[ids[order]] = np.arange(n_active). Verify external references (held as ids) are still correct after the sort.
8. (stretch) A from-scratch generational arena. Combine §10’s gens: np.ndarray, §22’s deferred cleanup, and §23’s id_to_slot map into a SlotMap class. Provide insert(row) -> CreatureRef, remove(ref), get(ref) -> int | None. Compare the shape with slotmap::SlotMap (Rust) — same machinery, organised differently.

Reference notes in 23_index_maps_solutions.md.

What’s next

§24 — Append-only and recycling names two strategies for what happens to a slot after it has been freed. The choice is decided by access pattern, not by taste.

Solutions: 23 — Index maps

Exercise 1 — Build the map

import numpy as np
INVALID = np.iinfo(np.uint32).max

class World:
    def __init__(self, capacity: int, n_ids: int):
        self.capacity = capacity
        self.n_active = 0
        self.id           = np.zeros(capacity, dtype=np.uint32)
        # ... other columns ...
        self.id_to_slot   = np.full(n_ids, INVALID, dtype=np.uint32)

    def append(self, new_id: int, **fields):
        slot = self.n_active
        self.id[slot] = new_id
        for k, v in fields.items():
            getattr(self, k)[slot] = v
        self.id_to_slot[new_id] = slot
        self.n_active += 1

Adding the map is one extra column and one extra line in append. Removal updates happen in cleanup (next exercise).

Exercise 2 — O(1) presence query

class World:
    hungry: np.ndarray = np.empty(0, dtype=np.uint32)
    hungry_member: np.ndarray = np.zeros(N_max, dtype=bool)

def become_hungry(world, creature_id: int):
    world.hungry = np.concatenate([world.hungry, [creature_id]])
    world.hungry_member[creature_id] = True

def stop_being_hungry(world, creature_id: int):
    world.hungry = world.hungry[world.hungry != creature_id]
    world.hungry_member[creature_id] = False

def is_hungry(world, creature_id: int) -> bool:
    return bool(world.hungry_member[creature_id])

Two parallel structures: hungry (the iteration list — O(K) walk) and hungry_member (the membership map — O(1) check). The list is for iterating; the bool array is for asking “is this id in the table?”. Both updated together; one read for each access pattern.

The cost is one byte per id ever issued (~1 MB at 1M ids), which is the memory price of constant-time membership.

Exercise 3 — Maintain on bulk-filter cleanup

def cleanup(world, buffer):
    if buffer.to_remove:
        ids = np.unique(np.array(buffer.to_remove, dtype=np.uint32))
        slots = world.id_to_slot[ids]
        keep_mask = np.ones(world.n_active, dtype=bool)
        keep_mask[slots] = False

        # mark the removed ids as no longer in the table
        world.id_to_slot[ids] = INVALID

        # compress every column
        n_keep = int(keep_mask.sum())
        for col_name in world.column_names:
            col = getattr(world, col_name)
            col[:n_keep] = col[: world.n_active][keep_mask]
        world.n_active = n_keep
        # rewrite id_to_slot for survivors — one bulk numpy assignment
        world.id_to_slot[world.id[:n_keep]] = np.arange(n_keep, dtype=np.uint32)
        buffer.to_remove.clear()

    # ... insertions: append new ids and write id_to_slot[new_id] = slot ...

The id_to_slot[ids[:n_keep]] = np.arange(n_keep) line is the keystone. It rewrites every surviving id’s slot in one bulk numpy assignment — exactly the same shape as the column compress, applied to the index map.

Exercise 4 — Time the difference

import time, numpy as np

world = build_world(n=1_000_000, hungry_count=100_000)
ids = np.random.default_rng(0).choice(1_000_000, size=100_000)

# Linear scan version (§17 ex 6)
def is_hungry_scan(hungry, target):
    return bool(np.any(hungry == target))

t = time.perf_counter()
for cid in ids:
    is_hungry_scan(world.hungry, int(cid))
print(f"linear scan × 100K: {time.perf_counter()-t:.2f} s")

# Indexed version
t = time.perf_counter()
for cid in ids:
    bool(world.hungry_member[int(cid)])
print(f"indexed × 100K: {time.perf_counter()-t:.3f} s")

Typical: linear scan ~5-10 minutes (10⁵ × 10⁵ = 10¹⁰ ops). Indexed: ~30 ms (one C-level read per call, plus Python loop overhead). Ratio: ~10⁵-10⁶×.

For a real simulator that does many membership queries per tick, the index map is the difference between workable and unsalvageable. Without it, presence-replaces-flags would only be defensible for whole-table operations, not individual queries.

Exercise 5 — Run the exhibit (honestly)

uv run "code/measurement/csr_matrix or python dict.py"

Benchmarking with a 1000x1000 matrix, 1.0% density (9954 non-zero elements).
Performing 10000 random lookups.

CSR Matrix lookup time:        0.0616 s
Python Dictionary lookup time: 0.00072 s

Python Dictionary is faster for lookups by approximately 85.62 times.

The headline (“Dict is 86× faster”) is true for the access pattern in the file (random scalar lookups). The right reading is that scipy gave you a sparse matrix, not a sparse map. CSR is excellent at:

import numpy as np
from scipy.sparse import csr_matrix

mat = csr_matrix((1000, 1000))
# ... populate ...
v = np.zeros(1000)
result = mat @ v               # SpMV — what CSR is actually for

For SpMV at 1000×1000 with 1% density, CSR is dramatically faster than naive dense or dict-based approaches — nine thousand multiplications instead of a million. That’s the operation it’s optimised for.

The lesson: pick the structure that matches your access pattern. A dict is a sparse point-lookup map. CSR is a sparse matrix. They share the word “sparse” and almost nothing else.

Exercise 6 — The bandwidth cost

1M id_to_slot entries × 4 bytes = 4 MB total
1500 cleanup writes per tick × 4 bytes = 6 KB written
At ~10 GB/s memory bandwidth: ~0.6 µs to write 6 KB
30 Hz tick budget: 33 ms

The cleanup map-update cost is 0.002% of the tick budget at typical mutation rates. The id_to_slot maintenance is invisible against the rest of the work. The 4 MB total memory cost is the dominant concern at scale, not the bandwidth — which mitigates to 400 KB once recycling caps the high-water id count.

Exercise 7 — Sort-for-locality compatibility

def sort_for_locality(world, key_col_name: str):
    """Sort the table in-place by some key (e.g., spatial bucket).
       Updates id_to_slot to reflect the new positions."""
    key = getattr(world, key_col_name)[: world.n_active]
    order = np.argsort(key, kind="stable")

    for col_name in world.column_names:
        col = getattr(world, col_name)
        col[: world.n_active] = col[: world.n_active][order]

    # the keystone again — one bulk update
    world.id_to_slot[world.id[: world.n_active]] = np.arange(world.n_active,
                                                              dtype=np.uint32)

After the sort, world.id[k] is some new id, and id_to_slot[world.id[k]] == k. External code holding a reference to id 42 looks up id_to_slot[42], gets the new slot, reads the (now-relocated) row.

The sort changed every slot. The map update changed every entry of id_to_slot. Both are O(N) bulk numpy operations — fast enough to do every tick if needed.

Exercise 8 — A from-scratch generational arena (stretch)

import numpy as np
from typing import NamedTuple

class CreatureRef(NamedTuple):
    id:  int
    gen: int

INVALID = np.iinfo(np.uint32).max

class SlotMap:
    """Generational arena: stable handles, O(1) lookup, slot recycling, generation checks."""

    def __init__(self, capacity: int = 65536, n_ids: int = 1_000_000):
        self.capacity = capacity
        self.n_active = 0
        self.id    = np.zeros(capacity, dtype=np.uint32)
        self.gens  = np.zeros(capacity, dtype=np.uint32)
        self.value = np.zeros(capacity, dtype=np.float32)
        self.id_to_slot = np.full(n_ids, INVALID, dtype=np.uint32)
        self.next_id = 0

    def insert(self, value: float) -> CreatureRef:
        if self.n_active >= self.capacity:
            raise MemoryError("SlotMap full")
        slot = self.n_active
        new_id = self.next_id
        self.next_id += 1
        self.id[slot]    = new_id
        self.gens[slot]  = 0
        self.value[slot] = value
        self.id_to_slot[new_id] = slot
        self.n_active += 1
        return CreatureRef(id=new_id, gen=0)

    def remove(self, ref: CreatureRef) -> bool:
        slot = self._slot_of(ref)
        if slot is None: return False
        last = self.n_active - 1
        moved_id = int(self.id[last])
        if slot != last:
            self.id[slot]    = self.id[last]
            self.gens[slot]  = self.gens[last]
            self.value[slot] = self.value[last]
            self.id_to_slot[moved_id] = slot
        self.id_to_slot[ref.id] = INVALID
        self.gens[last] += 1                      # bump generation for next reuse
        self.n_active -= 1
        return True

    def get(self, ref: CreatureRef) -> float | None:
        slot = self._slot_of(ref)
        return None if slot is None else float(self.value[slot])

    def _slot_of(self, ref: CreatureRef) -> int | None:
        slot = int(self.id_to_slot[ref.id])
        if slot == INVALID: return None
        if int(self.gens[slot]) != ref.gen: return None
        return slot

Compare with slotmap::SlotMap (Rust): same machinery, different organisation. Rust packs (index, generation) into one Key (a u64); we use a NamedTuple. Rust uses Vec<Slot> with an internal free list; we use an active counter and bump generations on remove. The structural pieces — id allocator, generation array, id_to_slot map, swap_remove on delete — are identical.

Combined with §22’s deferred cleanup, this SlotMap is the simulator’s table primitive. Once you have it, every variable-quantity table in the book reuses the shape — creatures, food, pending events, transition log entries — each one a SlotMap with different columns.

24 — Append-only and recycling

Concept node: see the DAG and glossary entry 24.

When a row is removed from a table, its slot is freed. There are two strategies for what happens to that slot.

Append-only. Old slots stay valid forever. The table grows monotonically. New rows always go to the end.

Recycling. Freed slots are reused. The table’s length stays bounded. New rows go into freed slots before the table grows.

Each is correct; they have very different access patterns and costs.

When you have to think about slot reuse

A short Python aside before the strategies. Most Python code never thinks about slot reuse because the language hides it: del obj lets the garbage collector reclaim the memory, and the next obj = something() may or may not land in the same address — you do not know and do not care. The runtime decides.

Numpy columns are the opposite. You allocated np.empty(N_max, dtype=...) once, at startup. The slots are positional: slot 17 is the bytes at offset 17 * 4. There is no GC to reclaim them; there is just n_active and a discipline about whether slot 17, once freed, gets reused or sits empty until the table is rebuilt. The Python edition’s lifecycle phase is exactly the work the runtime usually does for you, made explicit because numpy will not.

Append-only

Use append-only when:

• History matters. The simulator’s eaten, born, dead logs from code/sim/SPEC.md are all append-only — they record what happened. Removed entries would be lost history.
• Old references must remain valid forever. Some slot-as-pointer designs assume the table never shrinks.
• Total volume is bounded by elapsed time, not by population. A 30-second 30 Hz simulation produces at most 900 frames; an append-only frame log is at most 900 rows. No need to recycle.

The cost is monotonic memory growth. A long-running simulator with append-only eaten accumulates millions of rows over hours. Mitigations:

1. Periodic snapshot + truncate (the log is replaced by a recent slice).
2. Tiered storage — recent in memory, older streamed to disk (§30).
3. Just accept the memory, if the run is short.

Recycling

Use recycling when:

• Steady-state size is small even though total inserted is large. The simulator’s creatures table at 100,000 alive with 100,000 deaths and 100,000 births per second — net flow zero, but total ever issued grows linearly. Recycling keeps memory bounded.
• Memory matters. Recycling caps the table at the high-water mark of live rows.

The cost is reference-stability complications. A new row in a recycled slot has the same slot as a previous, removed row. Code holding an old slot reference would silently dereference the new row. The fix is generational ids (§10): each slot has a generation counter that increments on every recycle. References hold (id, gen); dereference checks the generation. A stale reference fails its check.

A slot allocator looks like:

class SlotPool:
    """Allocates fixed-capacity slot indices, recycling freed ones.
       Generation increments on every free, so old (slot, gen) refs
       can detect they are stale."""

    def __init__(self, capacity: int) -> None:
        self.capacity = capacity
        self.free_slots: list[int] = []          # stack of freed slots
        self.next_slot: int = 0                  # high-water mark
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int]:
        if self.free_slots:
            slot = self.free_slots.pop()         # reuse a freed one
        else:
            slot = self.next_slot                # grow
            self.next_slot += 1
            assert self.next_slot <= self.capacity, "pool exhausted"
        return slot, int(self.gens[slot])

    def free(self, slot: int) -> None:
        self.gens[slot] += 1                     # invalidate old refs
        self.free_slots.append(slot)

allocate pops a freed slot if any are available, otherwise grows. free bumps the generation and adds the slot back to the free list. Stale references (with the old generation) cannot dereference the recycled row.

The free list is a Python list used as a LIFO stack — append and pop are both O(1). The generation column is numpy because it is touched in lockstep with cleanup (§22) and benefits from bulk numpy ops when many slots are freed together.

Choosing between them

Match the strategy to the table’s role:

Mixing strategies in one simulator is normal. The discipline is to be explicit about which table is which, and apply the right machinery to each.

Exercises

1. Two append-only logs. Implement eaten and born as append-only numpy columns with their own n_active counters. After 1,000 ticks, examine the lengths and verify they grow monotonically.
2. A recycling pool. Implement the SlotPool above. Allocate 1,000 slots, free 500, allocate 500 more. Print the slot indices the second allocate batch returns. Did the pool reuse the freed slots, or grow?
3. Stale reference detection. Allocate a slot with (slot, gen=0). Free it. Allocate a new row in the same slot — its gen is 1. Try to dereference the old (slot, 0) against the live gens column; confirm the check fails.
4. Switch creatures to append-only. Run the simulator with creatures as append-only (no recycling, every birth grows the table). Run for 10,000 ticks with steady birth and death. Plot n_active and next_slot over time — n_active is roughly flat (deaths balance births), next_slot grows monotonically. Memory cost: next_slot * row_size.
5. Switch eaten to recycling. Run with eaten recycled. After 100 ticks, all “what did this creature eat at tick 50” queries fail because the rows were reused. The history is gone. This is the failure mode that makes append-only the right pick for logs.
6. (stretch) A capacity-aware allocator. Modify SlotPool.allocate to return None when the pool is full instead of asserting. The simulator now has to handle “no slot available” as a real condition — what does it mean? (Hint: the world has hit its population cap; either rebuild bigger, drop the new entity, or delete the oldest one to make room.)

Reference notes in 24_append_only_and_recycling_solutions.md.

What’s next

§25 — Ownership of tables is the rule that makes every other discipline in the phase work: each table has exactly one writer.

Solutions: 24 — Append-only and recycling

Exercise 1 — Two append-only logs

import numpy as np

class AppendLog:
    def __init__(self, capacity: int, dtype):
        self.tick      = np.zeros(capacity, dtype=np.uint32)
        self.creature  = np.zeros(capacity, dtype=np.uint32)
        self.value     = np.zeros(capacity, dtype=dtype)
        self.n_active  = 0
        self.capacity  = capacity

    def append(self, tick: int, creature_id: int, value):
        if self.n_active >= self.capacity:
            raise MemoryError("log full — snapshot and truncate")
        self.tick[self.n_active]     = tick
        self.creature[self.n_active] = creature_id
        self.value[self.n_active]    = value
        self.n_active += 1

eaten = AppendLog(capacity=1_000_000, dtype=np.float32)
born  = AppendLog(capacity=1_000_000, dtype=np.uint32)

# After 1000 ticks of the simulator
print(f"eaten: {eaten.n_active} entries (monotonic — never shrinks)")
print(f"born:  {born.n_active} entries (monotonic)")

Both n_active counters only ever increment. Once entries are written, they stay. Capacity is the high-water-mark of total events ever recorded, not of current population.

Exercise 2 — A recycling pool

class SlotPool:
    def __init__(self, capacity: int):
        self.capacity = capacity
        self.free_slots: list[int] = []
        self.next_slot: int = 0
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int]:
        if self.free_slots:
            slot = self.free_slots.pop()
        else:
            slot = self.next_slot
            self.next_slot += 1
        return slot, int(self.gens[slot])

    def free(self, slot: int):
        self.gens[slot] += 1
        self.free_slots.append(slot)


pool = SlotPool(capacity=10_000)
first_batch = [pool.allocate()[0] for _ in range(1_000)]
print(f"first 1000 slots: {first_batch[:5]}...{first_batch[-3:]}")

# Free 500
for slot in first_batch[:500]:
    pool.free(slot)

# Allocate 500 more
second_batch = [pool.allocate()[0] for _ in range(500)]
print(f"second 500 slots: {second_batch[:5]}...{second_batch[-3:]}")
print(f"all reused?      {set(second_batch).issubset(set(first_batch[:500]))}")
print(f"next_slot now:   {pool.next_slot}")  # still 1000 — no growth

first 1000 slots: [0, 1, 2, 3, 4]...[997, 998, 999]
second 500 slots: [499, 498, 497]...[2, 1, 0]
all reused?      True
next_slot now:   1000

The second batch reuses the freed slots in LIFO order (the most recently freed slot is allocated next). Total next_slot stays at 1000 — the pool did not grow.

Exercise 3 — Stale reference detection

pool = SlotPool(capacity=100)
slot, gen = pool.allocate()                           # slot=0, gen=0
old_ref = (slot, gen)                                  # save
pool.free(slot)                                        # gens[0] = 1
new_slot, new_gen = pool.allocate()                    # reuses slot 0, gen=1
new_ref = (new_slot, new_gen)

def deref(pool, ref):
    slot, gen = ref
    return None if int(pool.gens[slot]) != gen else slot

print(f"new ref deref:  {deref(pool, new_ref)}")      # 0
print(f"old ref deref:  {deref(pool, old_ref)}")      # None — stale!

The old reference’s generation is stale. Even though the slot is alive again, the generation check correctly identifies that the holder of old_ref is looking at a different row than they expect. The reference is rejected; the holder must re-fetch.

Exercise 4 — Switch creatures to append-only

# After 10,000 ticks, steady-state birth/death:
# n_active ≈ 100,000 (live population, oscillates around equilibrium)
# next_slot = total ever issued = (births_per_tick × 10,000)
#           ≈ 100 × 10,000 = 1,000,000 (10× the live population)

# Memory cost: next_slot × row_size = 1M × ~32 bytes = 32 MB
# Live data:    n_active × row_size = 100K × 32 bytes = 3.2 MB
# Wasted:       28.8 MB sitting in dead slots

The append-only creatures table has 90% of its memory occupied by tombstones — slots whose previous occupants are dead. Reading n_active is correct, but the table’s allocated bytes grow with elapsed time. For a 1-hour simulation, the wasted memory might be 100× the live data.

For history tables this is fine (the tombstones are the history). For the live population it’s a memory leak waiting to be named. Recycling is the structural fix.

Exercise 5 — Switch eaten to recycling

# eaten is now a SlotPool-managed table with capacity 10_000
# After 100 ticks at 50 eats/tick, 5000 events recorded into 10_000 slots
# After 200 ticks: free list starts being used; old eat events are overwritten
# After 300 ticks: ~10,000 events have been recycled into existing slots

# Query: "what did creature 42 eat at tick 50?"
# Search eaten.tick[:n] == 50 → finds it (tick 100)
# Search after tick 250: finds nothing (the row was recycled at ~tick 250)

The history is gone once a slot is recycled. There is no record that creature 42 ate at tick 50 — the slot now holds tick 273’s eat event for creature 91. Recycling for a history table is category error.

This is exactly the failure mode that makes append-only correct for logs. Logs grow forever; you handle that with snapshot-and-truncate or tiered storage, not by recycling slots.

Exercise 6 — A capacity-aware allocator (stretch)

class CapacityAwarePool:
    def __init__(self, capacity: int):
        self.capacity = capacity
        self.free_slots: list[int] = []
        self.next_slot: int = 0
        self.gens = np.zeros(capacity, dtype=np.uint32)

    def allocate(self) -> tuple[int, int] | None:
        if self.free_slots:
            slot = self.free_slots.pop()
        elif self.next_slot < self.capacity:
            slot = self.next_slot
            self.next_slot += 1
        else:
            return None                              # full!
        return slot, int(self.gens[slot])

Returning None from allocate is the simulator’s signal that the world has hit its population cap. Three reasonable policies:

1. Drop the new entity. “Sorry, no room.” A reproduction event silently fails. Simplest, hides the resource limit, may distort the simulation’s behaviour.
2. Delete the oldest one. A LRU-style eviction. The pool needs an oldest-tracking scheme (a tick column, a queue). The simulation continues at capacity but loses identifiers; not appropriate for logs but reasonable for active scenes.
3. Resize the pool. Allocate a larger backing buffer, copy, retry. Most flexible, but introduces a slow-path that may blow a tick budget; consider doubling the pool every time it fills, like Vec::push’s amortised growth.

The book’s simulator picks whichever fits the table:

• creatures: option 3 with periodic doubling. The scenario should never hit the cap in practice, but if it does, the simulation continues.
• pending_event: option 1. Events that don’t fit the cap are dropped; the simulation makes do with what it has.
• eaten/born (append-only): option 1, but with snapshot+truncate as the recovery, not silent drop.

The choice is per-table. Document it next to the table’s allocation.

25 — Ownership of tables

Concept node: see the DAG and glossary entry 25.

Every table has exactly one writer.

The rule is small. Its consequences are everything.

Why it works. A row is a tuple (§6) — its fields are aligned by index. A table’s columns must be modified together to maintain alignment. A single writer guarantees this: only one place in the code mutates the table, so only one place can violate alignment, so testing one place is enough.

A table with two writers has two places where alignment can be violated. If they run concurrently, alignment is violated nondeterministically. If they run sequentially, the order matters and must be specified. Either way, the cost of getting it right grows superlinearly with the number of writers.

The Python-specific problem: nothing enforces it

Rust has a borrow checker. &mut [T] is the type-level expression of single-writer ownership; only one mutable reference can exist at a time, and the compiler rejects code that violates it. Python has no equivalent. There is no &mut, no exclusive-access type, no compile-time check. Anyone who has a reference to a numpy array can mutate it. The single-writer rule is a discipline you enforce by convention, not a constraint the language enforces for you.

This makes the rule more important in Python, not less. Without compile-time enforcement, the violations show up at runtime as the bugs the rule was supposed to prevent: intermittent, silent, late-binding. The discipline is what stands between the architecture and the bug.

The numpy view trap

The hardest version of the violation in Python is the numpy view. A slice of a numpy array is not a copy — it is a view into the same underlying bytes. Writing through the view mutates the parent:

# anti-pattern: bad!
arr = np.zeros(10)
view = arr[2:5]    # looks like a new array; is actually a view
view[0] = 42       # also writes arr[2] = 42

A function receiving view has no way to know from the variable’s name or its np.ndarray type that it shares memory with someone else’s table. There is no compile-time signal. Mutating view looks local; the side effect on arr is invisible until something else reads it. This is the single-writer rule violated at the byte level, hidden behind a slice that looks like a fresh allocation.

Three mitigations:

# explicit copy when handing data to a function that may mutate it
foreign_function(arr[2:5].copy())

# read-only flag on the parent (writes via any view raise ValueError)
arr.flags.writeable = False

# document the ownership in the function signature and let it live in the contract
def motion(pos_x: np.ndarray, vel_x: np.ndarray, dt: float) -> None:
    """Read-set: vel_x, dt.   Write-set: pos_x.
       pos_x and vel_x must not alias each other or any other column."""

The first two are runtime mechanisms. The third is the convention this book lives on. A function’s docstring declares the read-set and write-set (§13); the caller is responsible for not handing aliasing arrays into a function that assumes none. If the caller cannot guarantee non-aliasing, they pass a copy.

The disciplines that depend on it

All of these need single-writer ownership to work:

• §31 — Disjoint write-sets parallelize freely. Two systems with disjoint write-sets can run on different processes. The rule guarantees no shared mutation.
• §22 — Mutations buffer. A side-table writer (cleanup) is the only writer of creatures. All other systems push to to_remove and to_insert, which they own.
• §43 — Tests are systems. A test system reads everything and writes nothing. The ownership rule is what guarantees its reads see consistent state.
• The InspectionSystem pattern. A debug inspector holds read-only references to every table. Read-only access composes with single-writer ownership to make races structurally impossible.

What the rule looks like in practice

def motion(pos_x: np.ndarray, pos_y: np.ndarray,
           vel_x: np.ndarray, vel_y: np.ndarray, dt: float) -> None:
    """Read-set: vel_x, vel_y, dt.   Write-set: pos_x, pos_y."""
    pos_x += vel_x * dt
    pos_y += vel_y * dt

def next_event(pos_x: np.ndarray, food_x: np.ndarray,
               pending: np.ndarray) -> None:
    """Read-set: pos_x, food_x.   Write-set: pending."""
    ...

def apply_eat(pending: np.ndarray, food: np.ndarray,
              to_remove: list[int], energy: np.ndarray) -> None:
    """Read-set: pending, food.   Write-set: to_remove (append), energy."""
    ...

For each table, exactly one writer is allowed:

• pos_x, pos_y: written only by motion.
• pending: written only by next_event.
• to_remove, to_insert: written by many systems, but each system appends only its own queued mutations; no one reads them until cleanup.
• creatures, food: written only by cleanup, which materialises every other system’s queued changes.

Multiple systems may contribute to a table by appending to its side buffer; the actual single writer of the live table is cleanup. The architecture preserves the rule even as many systems propose mutations.

Bugs that arise from violations

Two systems writing the same column produce inconsistent state. The bug is usually intermittent (depends on schedule), silent (no error reported, just bad data), and late-binding (manifests far from the cause). They are among the hardest bugs in any concurrent system. The single-writer rule eliminates them by construction. In Python, where the language will not catch the violation, the rule is the only thing standing between you and the bug.

The rule applies recursively. A view table whose entries are derived from another table inherits the ownership rule: a hungry: np.ndarray is owned by the system that classifies hunger; no other system writes to it.

This is the rule that closes Memory & lifecycle. Without it, the buffering, swap_remove, index maps, and slot recycling are all unsafe in any concurrent or parallel context. With it, everything composes.

Exercises

1. Identify the writers. For each table in your simulator (creatures, food, food_spawner, pending_event, eaten, born, dead, hungry, to_remove, to_insert), name the one system that writes it. If you find a table with two writers, the rule is violated — investigate.
2. The view trap, in your fingers. Build arr = np.arange(10). Take view = arr[2:5]. Set view[0] = 999. Print arr. Confirm arr[2] == 999. Now take cpy = arr[2:5].copy(), set cpy[0] = 0, print arr — confirm arr is unchanged. The slice was a view; the .copy() was not.
3. The read-only-flag mitigation. Build arr = np.arange(10). Set arr.flags.writeable = False. Try to assign arr[3] = 42. Catch the ValueError. Now derive view = arr[2:5] from the read-only parent — note that view.flags.writeable is also False. Read-only-ness propagates.
4. A constructed violation. Write two functions that both mutate energy. Call them in sequence on the same array; the result is whatever the second one wrote. Now run them in two multiprocessing.Process workers sharing the array via multiprocessing.shared_memory; observe that no error is raised and the bug is silent. This is the failure mode the single-writer rule prevents — Python will not warn you.
5. Refactor with a buffer. Take one of the violations from exercise 4 and add a side buffer that one function writes and the other reads. The two functions are now writer-disjoint, even though they touch the same logical concept.
6. Build an InspectionSystem. Write a function that takes a World (a dataclass holding all the tables), reads every column, and returns a snapshot dictionary. Mark every input array read-only via arr.flags.writeable = False for the duration of the call. The system is read-only by construction and cannot violate the rule.
7. (stretch) The cleanup system as canonical writer. In your simulator, audit: every mutation of creatures, food, etc. flows through cleanup. Every other system writes only to to_remove, to_insert, or its own outputs. Verify the audit holds for the simulator end-to-end. Note this is harder in Python than in Rust because nothing checks it for you — write a unit test that asserts no system other than cleanup mutates the live tables.

Reference notes in 25_ownership_of_tables_solutions.md.

What’s next

You have closed Memory & lifecycle. The simulator’s machinery is now complete: it can grow, shrink, recycle, parallelise, and replay. The next phase is Scale, starting with §26 — Hot/cold splits. The simulator’s per-tick cost goes under the microscope.

Solutions: 25 — Ownership of tables

Exercise 1 — Identify the writers

Audit any simulator project for tables with two writers — that’s the rule violated. Common sources of the violation:

• An “update” function that also validates and corrects.
• A logging side-effect that mutates state.
• Two systems both setting a derived flag.

The fix is always one of: split into two systems with an intermediate buffer, or designate one as the writer and have the other request changes via a side buffer.

Exercise 2 — The view trap, in your fingers

import numpy as np

arr = np.arange(10)
view = arr[2:5]                      # view, NOT a copy
view[0] = 999
print(arr)
# [  0   1 999   3   4   5   6   7   8   9]   — arr was mutated through the view!

cpy = arr[2:5].copy()                # explicit copy
cpy[0] = 0
print(arr)
# [  0   1 999   3   4   5   6   7   8   9]   — arr unchanged

A slice of a numpy array is a view into the same backing buffer. view[0] = 999 writes to byte offset 16 of arr (since int64 × index 2). The .copy() allocates a new buffer; mutations there are isolated.

This is the classic memory-aliasing trap. The variable name (view vs cpy) gives no signal. The dtype gives no signal. The only ways to know: check arr.base (view.base is arr is True; cpy.base is None) or pass through the convention *.copy() whenever ownership transfers.

Exercise 3 — The read-only-flag mitigation

arr = np.arange(10)
arr.flags.writeable = False

try:
    arr[3] = 42
except ValueError as e:
    print(f"caught: {e}")
# caught: assignment destination is read-only

view = arr[2:5]
print(view.flags.writeable)          # False — read-only-ness propagates to views

Setting writeable = False is a runtime guard. Anyone with a reference to the array — including any view derived from it — can read but not write. This is the closest Python has to Rust’s &[T] (immutable borrow). It does not guarantee correctness across function calls (a careless caller can still set writeable = True back), but it catches accidental writes loudly.

For library functions that accept arrays from outside, locking the input via writeable = False for the function body is a defensive practice. The cost is one attribute set; the protection is real.

Exercise 4 — A constructed violation

import numpy as np

def system_a(energy):
    energy[:] += 1.0                # writer 1

def system_b(energy):
    energy[:] -= 0.5                # writer 2 — same column!

# Sequentially: result depends on order
energy = np.zeros(10)
system_a(energy); system_b(energy)
print(energy)        # [0.5, 0.5, ...] — A first, then B

energy = np.zeros(10)
system_b(energy); system_a(energy)
print(energy)        # [0.5, 0.5, ...] — same end state because additions commute
                      # but the per-step state would differ

Sequentially: order matters and must be specified. With multiprocessing/shared_memory:

# anti-pattern: bad! two writers, no synchronisation
from multiprocessing import Process
from multiprocessing.shared_memory import SharedMemory
import numpy as np

shm = SharedMemory(create=True, size=80)
energy = np.ndarray((10,), dtype=np.float64, buffer=shm.buf)
energy[:] = 0

def worker_a(shm_name):
    s = SharedMemory(shm_name)
    e = np.ndarray((10,), dtype=np.float64, buffer=s.buf)
    for _ in range(1_000_000): e[:] += 0.0001

def worker_b(shm_name):
    s = SharedMemory(shm_name)
    e = np.ndarray((10,), dtype=np.float64, buffer=s.buf)
    for _ in range(1_000_000): e[:] -= 0.0001

# Run them simultaneously
p1, p2 = Process(target=worker_a, args=(shm.name,)), Process(target=worker_b, args=(shm.name,))
p1.start(); p2.start(); p1.join(); p2.join()
print(energy)   # not [0, 0, ...] — race conditions ate some updates

Each += involves a read, an add, and a write. Two processes interleaving these without coordination produce lost updates: process A reads x; process B reads x; A writes x+1; B writes x-1; the result is x-1 (or x+1) instead of x. No ValueError, no warning. Just silently wrong arithmetic.

The single-writer rule is the structural prevention. Two writers to the same column means coordination is required, and Python provides no enforcement. The rule eliminates the need for coordination at the architectural level.

Exercise 5 — Refactor with a buffer

def system_a(energy, energy_delta):
    energy_delta[:] += 1.0           # writer of energy_delta only

def system_b(energy, energy_delta):
    energy_delta[:] -= 0.5           # writer of energy_delta only

def cleanup(energy, energy_delta):
    energy[:] += energy_delta        # the SOLE writer of energy
    energy_delta[:] = 0

Now system_a and system_b are writer-disjoint with respect to energy; both write to energy_delta (which is also a violation, but a contained one — energy_delta is a side buffer, not load-bearing world state).

The architectural fix is one more level of buffering: each system writes to its own delta column.

def system_a(energy_delta_a):  energy_delta_a[:] += 1.0
def system_b(energy_delta_b):  energy_delta_b[:] -= 0.5
def cleanup(energy, energy_delta_a, energy_delta_b):
    energy += energy_delta_a + energy_delta_b
    energy_delta_a[:] = 0; energy_delta_b[:] = 0

This is the canonical pattern for parallel mutation: each writer has its own column; the merge happens in cleanup, single-threaded, on disjoint inputs. §31 — Disjoint write-sets parallelize freely develops it further.

Exercise 6 — Build an InspectionSystem

from contextlib import contextmanager

@contextmanager
def read_only_world(world):
    """Locks every column read-only for the duration of the inspection."""
    columns = (world.pos_x, world.pos_y, world.vel_x, world.vel_y, world.energy, world.id)
    for c in columns:
        c.flags.writeable = False
    try:
        yield
    finally:
        for c in columns:
            c.flags.writeable = True

def inspect(world) -> dict:
    """A read-only system; returns a snapshot."""
    with read_only_world(world):
        return {
            "n_active": world.n_active,
            "energy_min": float(world.energy[: world.n_active].min()),
            "energy_max": float(world.energy[: world.n_active].max()),
            "centre_of_mass": (float(world.pos_x[: world.n_active].mean()),
                               float(world.pos_y[: world.n_active].mean())),
        }

The system reads everything, writes nothing, locks the world for the duration. Any accidental write inside the inspection raises ValueError immediately. The lock is dropped on exit, so subsequent (non-inspection) systems can mutate normally.

This is the §43 test as system shape. A test that “verifies the world is consistent” runs in the same shape: lock, read, assert, unlock.

Exercise 7 — The cleanup system as canonical writer (stretch)

def write_audit(world, system_func):
    """Record which columns each system wrote during one tick."""
    snapshot_before = {name: getattr(world, name).tobytes() for name in world.column_names}
    system_func(world)
    written = []
    for name, before in snapshot_before.items():
        after = getattr(world, name).tobytes()
        if after != before:
            written.append(name)
    return written

# After running each system, assert which ones it should have written
expected = {
    "motion":         {"pos_x", "pos_y"},
    "next_event":     {"pending_event"},
    "apply_eat":      {"to_remove", "energy_delta"},     # buffers, not live tables
    "apply_starve":   {"to_remove"},
    "cleanup":        {"pos_x", "pos_y", "vel_x", "vel_y", "energy", "id", "n_active",
                       "id_to_slot", "to_remove", "to_insert_pos_x", ...},  # cleanup writes everything
}

for name, func in systems:
    written = write_audit(world, func)
    assert set(written) <= expected[name], f"{name} wrote {written} — unexpected: {set(written) - expected[name]}"

The audit is itself a system. It runs once per tick (or in a CI-only build) and asserts the structural property: every system writes only what it claims to write. A drift between expected and actual is the signal that someone added a side-effect — exactly the violation the single-writer rule forbids.

In Python this is the closest you get to a borrow checker. It runs at runtime, with O(N) overhead per tick (the byte snapshots), and it catches violations at the smallest mutation. Disable it in production; keep it on in CI.

26 — Hot/cold splits

Concept node: see the DAG and glossary entry 26.

The simulator’s creature table has six columns: pos, vel, energy, birth_t, id, gen. The motion system reads three of the six (pos, vel, energy). The starvation system reads only energy. The cleanup system reads id and gen. The births log reads birth_t. No system reads all six.

If the columns are stored together — same memory region, same prefetcher pulls — every load brings in fields the inner loop ignores. At cache-spilling sizes, the ignored fields cost real bandwidth.

The fix is a split: fields touched on the hot path go in one table; fields read rarely go in another. Two tables, same length, same id alignment.

Why this lesson is gentler in Python+numpy

In a Rust struct-of-fields-per-creature layout, pos, vel, energy, birth_t, id, gen all sit adjacent in memory. When the motion system reads pos, the cache line it pulls also contains birth_t and id and gen — the prefetcher loads them whether you want them or not. The hot/cold split breaks this adjacency by moving the cold fields to a different memory region.

In Python with numpy SoA, the situation is already different. Each column is its own contiguous numpy array, allocated by its own np.empty(...) call. When the motion system reads pos_x, the cache line it pulls contains only pos_x values. It does not touch birth_t’s memory at all. The columns are already physically separated. The hot/cold split is, for SoA-in-numpy, largely organisational — a way of naming and grouping columns that share an access pattern — not a memory-layout optimisation.

The split does matter when the layout is something other than SoA-in-numpy:

• AoS dataclass lists (list[Creature] with attributes). Reading c.pos_x from each instance pulls the full Creature object into cache, including c.birth_t and c.id. Splitting into two parallel lists of dataclasses (a hot Creature and a cold Creature) saves cache bandwidth — but you have already paid the bigger cost of being AoS in the first place. From §11’s tick_budget.py, the AoS form costs 28 ms per tick at 1M creatures vs 0.6 ms for SoA. The hot/cold split inside the AoS form might recover some of that gap; switching to numpy SoA recovers all of it.
• Numpy structured arrays — np.dtype([('pos', np.float32, 2), ('vel', np.float32, 2), ('birth_t', np.float64), ...]). This is AoS in numpy clothing — the bytes for one creature are adjacent. Reading arr['pos'] strides through the buffer, skipping past birth_t’s bytes one row at a time. The strided access is faster than a Python loop but slower than a contiguous numpy column. Splitting helps; using non-structured columns helps more.

The SoA-in-numpy discipline this book has built since §7 means most of the bandwidth win the hot/cold split offers in Rust, Python+numpy already gives you for free. The chapter exists for two reasons that are still load-bearing.

What the split still buys you

1. Code-organisational clarity. A reader of motion(pos_x, pos_y, vel_x, vel_y, energy, dt) should not also have to know where birth_t lives. Putting the hot columns under one CreatureHot namespace and the cold columns under CreatureCold makes the read-set/write-set declarations from §13 shorter and the dependency graph from §14 sparser. The compiler does not enforce it; the discipline does.

2. Cleanup amortisation. Cleanup (§22) writes every column when slots move. Six columns means six bulk-filter operations per cleanup. Splitting into hot (4 columns) and cold (2 columns) does not reduce the total work, but it lets you skip the cold-table cleanup between creatures-affecting and creatures-not-affecting cleanup phases. If a tick has only food deaths (no creature deaths), the creature_cold cleanup runs at zero cost (§20) — the empty-tables-free property compounds with the split.

3. Persistence and inspection. A snapshot for replay §37 needs every column. A live debug inspector might want only the cold metadata. Splitting lets the inspector read only what it needs and avoid loading the hot columns for an interactive query.

The cost of the split is the cost of an extra table: one more name, one more bookkeeping point in cleanup, one more place where alignment must be maintained. Two tables of the same length share an id allocator; updates that affect both must be applied in lockstep.

When the split is wrong

• Pure SoA-in-numpy with sub-millisecond inner loops. If the existing layout already has every column as its own numpy array and the inner loops are bandwidth-bound at numpy speed, splitting will not measurably help. The bandwidth wasn’t being wasted to begin with.
• All-fields workloads. A debug-inspect system that reads every field reads everything; the split adds organisational overhead without reducing access cost.
• Tiny rows. If the full row is already 16-24 bytes, the split’s overhead exceeds its benefit.
• Frequently rebalancing. If which fields are “hot” changes from tick to tick, a fixed split becomes unhelpful. Hot/cold is a static decision, made once for a given target workload.

The decision rests on measurement. Profile the simulator at the target size; identify the inner loop’s actual touched columns; split when the split changes a measurable number. The split is earned by data, not by aesthetics.

A useful test: name the split before writing it. “I am moving birth_t into a cold table because no inner loop reads it” is a sound design choice. “I am moving birth_t into a cold table because that’s how ECS engines do it” is not.

Exercises

These extend the simulator’s creature table.

1. Audit access patterns. For each system in your simulator, list which columns it reads and which it writes. Columns read every tick are hot; the rest are cold.
2. Build the split, organisationally. Refactor creature into creature_hot (a class holding pos_x, pos_y, vel_x, vel_y, energy) and creature_cold (a class holding birth_t, id, gen). Both share the id allocator. Verify each row’s fields stay aligned across the two classes.
3. Time motion at 1M creatures. Pre-split: time motion. Post-split: time motion. The two should be near identical if you started from numpy SoA. The split was organisational, not bandwidth-saving.
4. Time motion in numpy structured-array form. Build the same world using arr = np.zeros(N, dtype=np.dtype([('pos_x', 'f4'), ('pos_y', 'f4'), ('vel_x', 'f4'), ('vel_y', 'f4'), ('energy', 'f4'), ('birth_t', 'f8'), ('id', 'u4'), ('gen', 'u4')])). Run motion as arr['pos_x'] += arr['vel_x'] * dt. Time it. Compare to the unsplit SoA version. The structured-array version is slower because it strides past every cold field on every read — this is the layout where the hot/cold split would actually help.
5. Cleanup must touch both. Modify cleanup to apply the keep_mask (§22) to both creature_hot and creature_cold columns when a creature dies. Verify alignment after.
6. A bad split. Construct a split where the wrong fields go cold (e.g. energy in cold). Time motion. The cost of the cache-trip on energy per tick should bury any savings elsewhere.
7. (stretch) The all-fields case. Write a system that reads every field (e.g. a serialiser). Time the split version. Discuss why the split’s overhead is real here, and why this is a fine tradeoff: most ticks do not run this system.

Reference notes in 26_hot_cold_splits_solutions.md.

What’s next

§27 — Working set vs cache puts numbers on the question this section was implicitly asking: how big is the inner loop’s footprint, and what cache level does it fit in?

Solutions: 26 — Hot/cold splits

Exercise 1 — Audit access patterns

A typical simulator audit:

Hot columns: pos_x, pos_y, vel_x, vel_y, energy — read by 4-5 systems per tick. Cold columns: birth_t, id, gen — read only by cleanup and logging.

food and pending_event are their own tables; not part of the creature split.

Exercise 2 — Build the split, organisationally

class CreatureHot:
    def __init__(self, capacity):
        self.pos_x  = np.zeros(capacity, dtype=np.float32)
        self.pos_y  = np.zeros(capacity, dtype=np.float32)
        self.vel_x  = np.zeros(capacity, dtype=np.float32)
        self.vel_y  = np.zeros(capacity, dtype=np.float32)
        self.energy = np.zeros(capacity, dtype=np.float32)

class CreatureCold:
    def __init__(self, capacity):
        self.birth_t = np.zeros(capacity, dtype=np.float64)
        self.id      = np.zeros(capacity, dtype=np.uint32)
        self.gen     = np.zeros(capacity, dtype=np.uint32)

class Creatures:
    def __init__(self, capacity):
        self.hot  = CreatureHot(capacity)
        self.cold = CreatureCold(capacity)
        self.n_active = 0

Both tables share the same capacity and n_active. Cleanup must apply rearrangement to both in lockstep.

Exercise 3 — Time motion at 1M creatures

unsplit numpy SoA:   ~0.6 ms per call
split numpy SoA:     ~0.6 ms per call

Identical. Each numpy column is already its own contiguous buffer; the split changed no bytes’ addresses, only their namespace. The bandwidth win the chapter’s prose describes for Rust does not materialise in Python+numpy because the layout is already optimal.

This is the chapter’s main point: the split is organisational, not bandwidth-saving, in the Python edition.

Exercise 4 — Time motion in numpy structured-array form

import numpy as np, time
n = 1_000_000
dtype = np.dtype([('pos_x', 'f4'), ('pos_y', 'f4'),
                  ('vel_x', 'f4'), ('vel_y', 'f4'),
                  ('energy', 'f4'), ('birth_t', 'f8'),
                  ('id', 'u4'), ('gen', 'u4')])
arr = np.zeros(n, dtype=dtype)
arr['vel_x'] = 1.0; arr['vel_y'] = 1.0

t = time.perf_counter()
for _ in range(100):
    arr['pos_x'] += arr['vel_x'] / 30.0
    arr['pos_y'] += arr['vel_y'] / 30.0
print(f"structured array motion: {(time.perf_counter()-t)*10:.2f} ms/call")

SoA columns:       0.62 ms
structured array:  4.93 ms
ratio:             8×

The structured array is 8× slower than separate columns. Why? Each arr['pos_x'] returns a strided view — it walks the buffer with stride 32 bytes (the size of the full row), reading 4-byte pos_x values every 32 bytes. The prefetcher pulls 32 bytes per row even though only 4 are used; the remaining 28 are wasted bandwidth (and the cold fields, especially birth_t at 8 bytes, are dragged through cache anyway).

This is the AoS-pattern in numpy clothing. The split would help here — splitting the structured array into two structured arrays, hot and cold, reduces the stride from 32 to 20 (40% bandwidth saving). But the simpler fix is to leave the structured-array world and use one numpy column per field, which is what the simulator does.

Exercise 5 — Cleanup must touch both

def cleanup(world: Creatures, to_remove: list[int]):
    if not to_remove: return
    ids = np.unique(np.array(to_remove, dtype=np.uint32))
    slots = world.id_to_slot[ids]
    keep_mask = np.ones(world.n_active, dtype=bool)
    keep_mask[slots] = False

    # Apply to both hot and cold columns in lockstep
    for col_name in ("pos_x", "pos_y", "vel_x", "vel_y", "energy"):
        col = getattr(world.hot, col_name)
        col[:keep_mask.sum()] = col[:world.n_active][keep_mask]
    for col_name in ("birth_t", "id", "gen"):
        col = getattr(world.cold, col_name)
        col[:keep_mask.sum()] = col[:world.n_active][keep_mask]
    world.n_active = int(keep_mask.sum())

The same keep_mask, applied to every column of every sub-table. Missing one column → misalignment between hot and cold (§9 bug across tables).

Exercise 6 — A bad split

# anti-pattern: bad! energy is hot, but we put it in cold
class CreatureHot:  pos_x, pos_y, vel_x, vel_y      # missing energy!
class CreatureCold: energy, birth_t, id, gen        # energy stored here

# motion now reads from cold:
def motion_bad(hot, cold, dt):
    hot.pos_x += hot.vel_x * dt
    hot.pos_y += hot.vel_y * dt
    # apply_starve still reads cold.energy every tick — extra column-access overhead

In numpy SoA the timing penalty is small (cold.energy is still its own contiguous array). In structured-array layout the penalty would be real — energy would be at stride-32-bytes inside the cold record. The categorical error is naming hot fields cold: code that follows the convention “cold means rarely read” will draw wrong conclusions about which fields can be omitted from inspection-time reads, persistence, etc.

The lesson: hot/cold is not about names; it is about access frequency. Audit, then split.

Exercise 7 — The all-fields case (stretch)

def serialize_world(world):
    """Read every column to disk via np.savez."""
    np.savez("snapshot.npz",
             pos_x  = world.hot.pos_x[: world.n_active],
             pos_y  = world.hot.pos_y[: world.n_active],
             vel_x  = world.hot.vel_x[: world.n_active],
             vel_y  = world.hot.vel_y[: world.n_active],
             energy = world.hot.energy[: world.n_active],
             birth_t = world.cold.birth_t[: world.n_active],
             id     = world.cold.id[: world.n_active],
             gen    = world.cold.gen[: world.n_active])

The split’s overhead here is the function signature — eight columns spread across two namespaces — but the runtime cost is identical to the unsplit version: every column gets read once, written to disk once. The serialiser does not benefit from the split.

This is a fine tradeoff: serialisation runs once per checkpoint (every minute? every hour?), not once per tick. Paying eight extra characters in the function call to keep the inner loop’s namespace clean is cheap. The split is earned by the hot path, not the cold path.

27 — Working set vs cache

Concept node: see the DAG and glossary entry 27.

The working set of a loop is the data it touches per pass. The cache hierarchy (§1) is what holds that data. The two together decide the loop’s speed — once you are in numpy. In pure Python, the interpreter-dispatch tax dominates and the cliff is invisible. The moment your inner loop drops into a bulk numpy op, the cliff is real and exactly where the hardware says it is.

If the working set fits in L1 — typically 32 KB per core — the loop runs near memory-bandwidth speed: ~0.1-0.5 ns per element. If it fits in L2 — typically 1-2 MB per core — it is ~0.5-2 ns. If it fits in L3 — typically 16-32 MB shared — it is ~1-5 ns. If it spills to RAM, sequential access drops to ~3-10 ns (prefetcher helping); random access drops to 50-200 ns (no prefetcher help).

These ranges are not theoretical. They are what your machine actually does, measured in §1’s cache_cliffs.py exhibit. The numbers from that exhibit, on this machine:

The cliff is in the gather column. The 10K and 100K rows fit in L1 / L2 (gather ratio ~2-16×); the 10M and 100M rows spill to RAM (ratio 54-72×). The numpy sequential row stays roughly flat because the prefetcher reaches forward and amortises the cost — that is what bandwidth-bound looks like on this machine.

Computing your working set

The arithmetic is mechanical. Motion’s inner loop reads pos_x: float32 = 4 bytes, pos_y: float32 = 4 bytes, vel_x: float32 = 4 bytes, vel_y: float32 = 4 bytes, energy: float32 = 4 bytes. Total: 20 bytes per creature. At N creatures, working set = 20 × N bytes.

Each transition costs roughly 3-5× in per-element time when the access pattern is random. Sequential access is largely insulated by the prefetcher, but only up to RAM bandwidth — at 10M creatures and beyond, the prefetcher is no longer hiding latency, just keeping pace with what RAM can deliver.

This is what §4’s “cliff” was about, made concrete for your simulator. The transition points are not magic — they are arithmetic over your cache sizes. From §1 exercise 1 you have those numbers written down.

Why this lesson still matters when numpy hides it

Most numpy code never thinks about cache size because the inner loops are bandwidth-bound and “fast enough.” That intuition holds until the working set leaves L3 — at which point per-element cost rises 5-10× with no change to the source code. A simulator written for 1M creatures and tested at 100K never notices the cliff; it shows up the day the simulator is sized to 10M and the deadline is missed.

The hot/cold split (§26) shrinks the working set. Motion’s working set goes from 40 bytes per creature (full row) to 20 bytes (hot columns only). This pushes the cliff outward by a factor of 2: a 2M-creature simulator now runs at L3-resident speeds instead of RAM-resident. In pure SoA-in-numpy, this is the chief tangible benefit of the split — and the §26 caveat applies: only when the inner loop is genuinely hitting the bandwidth ceiling does the split move the cliff.

Design discipline

• Decide the target N before the schema. The schema must fit the cache that fits N.
• Audit the inner loops. Sum the bytes per row touched. Compare to your cache sizes.
• When you cross a transition, measure — do not assume. The prefetcher and the OS will sometimes save you, sometimes not. Numpy’s bulk-op threshold also shifts with version; benchmark on the exact stack you ship.
• The narrowest dtype that holds the value (§2) is not aesthetic; it is the cliff’s distance. np.float32 over np.float64 doubles the headroom; np.uint8 for indices in [0, 256) packs 64 to a cache line.

This is not premature optimisation. It is layout-aware design — making the schema fit the machine that will run it. A schema that ignores the cache works for small N and breaks at the scales the simulator was meant for.

Exercises

1. Compute your working sets. For each system in your simulator, compute bytes per row × N for N = 1K, 10K, 100K, 1M, 10M. Note which cache level each falls into on your machine (use lscpu | grep -i cache from §1 exercise 1).
2. Find your cliff. uv run code/measurement/cache_cliffs.py (the §1 exhibit) gives you ns/element across sizes for sequential and gather access. Plot the gather column. The transitions should match your cache sizes.
3. Reduce the working set. Apply the hot/cold split organisationally (§26) so motion reads only the hot columns. Time motion at the cliff size you found in exercise 2. Did the cliff move? In pure SoA-in-numpy, the answer is “no, because the columns were already separated” — see §26’s framing.
4. A wider dtype. Change energy: float32 to energy: float64. Recompute the working set. Time motion. The cliff should move inward (closer to smaller N).
5. Random vs sequential, your machine. Re-read the gather/seq ratio in the cache_cliffs table for your output. The factor 2.7× → 72× growth across sizes is your machine’s cache-vs-RAM cost gap. Memorise this number; it is the answer to “how much does a random access cost compared to a sequential one on this hardware?”.
6. (stretch) The L1 sweet spot. Find the N at which motion’s working set fills L1 to roughly 75%. Run the motion loop in tight repetition (call it 1,000 times in a row, no other work between calls). The L1-resident loop should run at a stable ~0.2 ns/element for the entire run. The closest L2-only neighbour should be 3-5× slower.

Reference notes in 27_working_set_vs_cache_solutions.md.

What’s next

§28 — Sort for locality puts the cache to work explicitly: rearrange your rows so accesses become more sequential.

Solutions: 27 — Working set vs cache

Exercise 1 — Compute your working sets

For the motion system (pos_x, pos_y, vel_x, vel_y, energy at float32):

The cliff is at the L3 → RAM transition. The exact size depends on your CPU’s L3 (run lscpu from §1 exercise 1 to confirm).

For the starvation system (reads energy only — 4 bytes per creature):

The starvation system fits more creatures per cache level than motion, because it touches fewer bytes per row. Smaller working set, larger headroom.

Exercise 2 — Find your cliff

uv run code/measurement/cache_cliffs.py

From §1 — gather column (random access):

N           gather (ns/elem)
10,000          1.62
100,000         2.24
1,000,000       3.69
10,000,000      7.60
100,000,000     7.78

Transitions visible: 10K → 100K (L1 → L2, ~1.4×), 100K → 1M (L2 → L3, ~1.6×), 1M → 10M (L3 → RAM, ~2.1×). The cliff is shallowest at the L1/L2 boundary and steepest at L3/RAM on this machine.

Exercise 3 — Reduce the working set

Splitting the motion’s row from (pos_x, pos_y, vel_x, vel_y, energy, birth_t, id, gen) = 36 bytes to (pos_x, pos_y, vel_x, vel_y, energy) = 20 bytes:

• Motion’s working set at 1M creatures: 36 MB → 20 MB. Still fits L3.
• At 2M: 72 MB → 40 MB. The 72-MB version spilled to RAM; the 40-MB version fits L3.

So the cliff moves outward by ~1.8× — exactly the bytes-ratio. But in pure SoA-in-numpy, motion already reads only the hot columns because each column is its own buffer; reading pos_x does not touch birth_t’s memory. The split is organisational, not a working-set reduction. The chapter’s framing applies: timing does not change.

The split does reduce working set in structured array layout, where reading arr['pos_x'] strides past birth_t’s bytes. Confirmed by exercise 4 of §26: structured array is 8× slower than SoA columns at the same N.

Exercise 4 — A wider dtype

energy = np.zeros(n, dtype=np.float64)         # was float32 — doubles the bytes

Working set per creature: 20 → 24 bytes (one column doubled). Cliff moves inward by ~20%. At N=1M, working set 24 MB → still fits typical L3. At N=1.5M, 36 MB → starts to spill. The motion timing rises proportionally to the bytes read (sequential access is bandwidth-bound; bytes moved is the cost).

This is §2’s narrowest-dtype discipline re-applied at scale. Choosing float32 over float64 doubles your population headroom in cache. The choice is not aesthetic — it is “how many creatures can my simulator host at L3-resident speed?”

Exercise 5 — Random vs sequential, your machine

From your cache_cliffs.py output:

The 100M figure is your machine’s L1-to-RAM cost gap on this run. On modern desktops 50-80×; on Pi 4 / 2012 Intel, closer to 30-40×; on Apple Silicon, somewhere in between.

Memorise the number. When a colleague says “the data structure I wrote does random lookups; I think it’s fast,” ask them for N. If N puts the working set past L3, multiply their best-case estimate by your machine’s gather/seq ratio. That’s the real cost.

Exercise 6 — The L1 sweet spot (stretch)

L1 is ~48 KB on this CPU; 75% = 36 KB. At 20 bytes per row, that’s ~1,800 creatures. Closest power-of-10-ish: 1,500-2,000.

import time, numpy as np

for n in (1_500, 1_800, 2_000, 10_000):
    pos_x  = np.zeros(n, dtype=np.float32)
    pos_y  = np.zeros(n, dtype=np.float32)
    vel_x  = np.ones(n,  dtype=np.float32)
    vel_y  = np.ones(n,  dtype=np.float32)
    energy = np.zeros(n, dtype=np.float32)
    dt = 1/30.0
    # warm up
    for _ in range(50):
        pos_x += vel_x * dt; pos_y += vel_y * dt
    t = time.perf_counter()
    for _ in range(1_000):
        pos_x += vel_x * dt; pos_y += vel_y * dt
    elapsed = (time.perf_counter() - t) / 1_000
    print(f"N={n:>5}: motion {elapsed*1e6:.2f} µs ({elapsed*1e9/n:.2f} ns/elem)")

Expected pattern: N=1500 and N=1800 stay around 0.2 ns/elem (L1-resident). N=10,000 jumps to 0.5-0.8 ns/elem (L2-resident — 3-5× slower).

The L1-resident regime is where you want hot inner loops to live. Any code path that runs every tick over a small data set should be sized so the data fits L1 — that’s the difference between “fast” and “very fast.” For the simulator, this matters most for per-creature derived columns (an urgency_score of length N_hot) that are computed and consumed within a single system.

28 — Sort for locality

Concept node: see the DAG and glossary entry 28.

In §9 you learned the sort-breaks-indices bug. In §10 you fixed it with stable ids. In §23 you made id-to-slot lookup O(1). With those three pieces in place, the simulator can now do something it could not before: rearrange its rows for locality.

The principle is simple. Rows accessed near each other in time should sit near each other in memory. Two creatures that interact (collide, query a neighbour, broadphase against each other) should land on adjacent cache lines.

The classic technique is a spatial sort. Each creature’s position is hashed to a spatial cell; the creatures table is sorted by cell. Reading “all creatures in cell C” becomes a contiguous range read.

def spatial_cell(pos_x: np.ndarray, pos_y: np.ndarray,
                 cell_size: float) -> np.ndarray:
    """Returns a uint32 cell id for each creature. Pack (x, y) cells
       into one integer. Z-order or Hilbert curves work too."""
    cx = (pos_x / cell_size).astype(np.int32)
    cy = (pos_y / cell_size).astype(np.int32)
    return ((cx & 0xFFFF) << 16) | (cy & 0xFFFF)


def sort_creatures_for_locality(world, cell_size: float) -> None:
    cells = spatial_cell(world.pos_x, world.pos_y, cell_size)
    order = np.argsort(cells, kind="stable")
    # Apply the same permutation to every column, in lockstep — §6's rule.
    world.pos_x[:] = world.pos_x[order]
    world.pos_y[:] = world.pos_y[order]
    world.vel_x[:] = world.vel_x[order]
    world.vel_y[:] = world.vel_y[order]
    world.energy[:] = world.energy[order]
    world.id[:]     = world.id[order]
    # Rebuild id_to_slot in lockstep — §23.
    world.id_to_slot[world.id] = np.arange(world.id.size, dtype=np.uint32)

Two creatures in the same spatial cell are now adjacent in pos_x and pos_y. The next-event system, which checks every creature against its spatial neighbours, strides through pos_x and reads neighbours from the same cache line.

Why this matters in numpy

The locality gap is not theoretical. From §1’s cache_cliffs.py, at 100M elements the gather (random-index) read is 72× slower than sequential on this machine. That ratio is the cost of every cache-unfriendly access pattern — every iteration that visits creatures in a non-spatial order pays it. Spatial sort converts gather-shaped reads into sequential ones, which is exactly the operation that ratio measures.

The cost is the sort itself. At 1M uint32 keys, np.argsort takes ~10-30 ms depending on input distribution. Done every tick this would be too expensive — but typically the sort is done every ~100 ticks (or when accumulated motion exceeds a threshold), amortising to ~0.1-0.3 ms per tick. The savings on the inner loop dwarf the cost.

Other sort orders, when they pay off

• Sort by id. Stable across runs; nice for debugging; but no locality benefit unless ids correlate with access patterns.
• Sort by access frequency. Hot creatures first; cold last. Useful only when the inner loop respects the order — and most numpy bulk ops do not, they walk the whole column.
• Sort by behaviour. All hungry creatures together; all sleepy together. Mostly redundant in a presence-based system (§19) where the hungry-driver iterates hungry directly.

Sort cadence is its own decision. Sorting every tick is wasted work if the world is mostly stationary. Sorting once at startup is wrong if the world drifts. Most simulators trigger a re-sort when accumulated motion since the last sort exceeds a fraction of the cell size.

The pieces this lesson assumes

The sort interacts with three earlier lessons:

• Lockstep reordering (§6, §9). Every column gets the same permutation applied. The world.pos_x[:] = world.pos_x[order] form is in-place rebinding to the same backing array — it does not allocate, and it does not break aliases held elsewhere. Doing this column-by-column for every column is the disciplined form.
• Stable ids (§10). Code outside the sort holds ids, not slots; the sort moves slots, and the id_to_slot map (the last line of sort_creatures_for_locality) keeps the ids correct.
• Index maps (§23). The id_to_slot rebuild is one bulk numpy assignment that runs in O(N) once per sort, not O(N) per id. The pieces compose.

This is the pattern Bevy, Unity DOTS, Unreal’s Mass Entities, and most production ECS engines use under the hood. Locality is paid up front (one sort) and amortised over many cache-friendly inner loops.

Exercises

1. Compute spatial cells. Write spatial_cell(pos_x, pos_y, cell_size) as in the prose. Apply it to a 1,000-creature world with random positions. Print np.bincount of the cell ids; this is the histogram of how many creatures land in each cell.
2. Sort by cell. Implement sort_creatures_for_locality with the lockstep column reorder. Run it. Verify: print the first ten (pos_x, pos_y) after the sort — these should be near-neighbour positions, not random ones.
3. Maintain id_to_slot. Confirm the id_to_slot[world.id] = np.arange(N) rewrite resolves correctly. Take a held id from before the sort; look up its slot after; confirm pos_x[slot] is the same value as before (it has just moved).
4. Time next_event before and after. Write a next_event system that, for each creature, scans the next 100 entries of pos_x, pos_y for collisions. Time it pre-sort vs post-sort at 100,000 creatures. The post-sort version should be measurably faster — by how much depends on how much the scan happens to land in the same cache line.
5. Sort cadence. Run a 100-tick simulation, sorting every tick. Run the same simulation, sorting every 10 ticks, and every 100 ticks. Compare total cost (sort cost + neighbour-scan cost). Find the cadence where sort cost equals neighbour-scan savings — that is your sweet spot.
6. (stretch) Z-order curve. Replace the simple (x, y) packing with a Z-order (Morton) hash — interleave the bits of cx and cy. Compare next_event timings. Z-order keeps spatially close cells close in the linear order; it usually outperforms simple stripe packing because two-cell horizontal neighbours stay adjacent.

Reference notes in 28_sort_for_locality_solutions.md.

What’s next

§29 — The wall at 10K → 1M is where these techniques start to bind. Code that ran fine at 10K stops running fine at 1M; the chapter is about finding out where and why.

Solutions: 28 — Sort for locality

Exercise 1 — Compute spatial cells

import numpy as np

def spatial_cell(pos_x: np.ndarray, pos_y: np.ndarray, cell_size: float) -> np.ndarray:
    cx = (pos_x / cell_size).astype(np.int32)
    cy = (pos_y / cell_size).astype(np.int32)
    return ((cx & 0xFFFF) << 16) | (cy & 0xFFFF)

rng = np.random.default_rng(0)
n = 1_000
pos_x = rng.uniform(0, 100, n).astype(np.float32)
pos_y = rng.uniform(0, 100, n).astype(np.float32)

cells = spatial_cell(pos_x, pos_y, cell_size=10.0)
unique, counts = np.unique(cells, return_counts=True)
print(f"{len(unique)} cells occupied, max {counts.max()} creatures per cell")
print(f"first 5 cells: {unique[:5].tolist()}")
print(f"histogram of cell counts: {np.bincount(counts)[:20]}")

For uniformly distributed creatures in a 100×100 world with 10-unit cells, expect ~100 cells (10×10 grid), ~10 creatures per cell on average. The histogram is Poisson-shaped — most cells have 5-15 creatures, a few have 0 or 25+.

Exercise 2 — Sort by cell

def sort_for_locality(world, cell_size: float):
    cells = spatial_cell(world.pos_x, world.pos_y, cell_size)
    order = np.argsort(cells, kind="stable")
    for col in ("pos_x", "pos_y", "vel_x", "vel_y", "energy", "id"):
        arr = getattr(world, col)
        arr[:] = arr[order]
    # rebuild id_to_slot
    world.id_to_slot[world.id[:world.n_active]] = np.arange(world.n_active, dtype=np.uint32)

sort_for_locality(world, cell_size=10.0)
print(f"first 10 positions after sort:")
for i in range(10):
    print(f"  ({world.pos_x[i]:.2f}, {world.pos_y[i]:.2f}) cell={spatial_cell(world.pos_x[i:i+1], world.pos_y[i:i+1], 10.0)[0]}")

After the sort, the first 10 positions belong to creatures in the same (or adjacent) cells — their (pos_x, pos_y) values cluster instead of scattering randomly.

Exercise 3 — Maintain id_to_slot

# Before sort: held_id is at some slot
held_id = int(world.id[42])
before_slot = 42
before_pos  = (float(world.pos_x[42]), float(world.pos_y[42]))

sort_for_locality(world, cell_size=10.0)

# After sort: look up by id
after_slot = int(world.id_to_slot[held_id])
after_pos  = (float(world.pos_x[after_slot]), float(world.pos_y[after_slot]))

print(f"before: slot={before_slot}, pos={before_pos}")
print(f"after:  slot={after_slot}, pos={after_pos}")
assert before_pos == after_pos, "data moved but is the same value"

The held id resolves to a new slot. The position at the new slot equals the position at the old slot. The id_to_slot map is the bridge; without it, the held reference would dereference garbage.

Exercise 4 — Time next_event before and after

import time, numpy as np

def next_event_scan(pos_x, pos_y, radius=1.0):
    """For each creature, count neighbours within radius among the next 100 entries."""
    n = len(pos_x)
    count = np.zeros(n, dtype=np.uint32)
    for i in range(n):
        end = min(i + 100, n)
        dx = pos_x[i+1:end] - pos_x[i]
        dy = pos_y[i+1:end] - pos_y[i]
        count[i] = int(np.sum(dx*dx + dy*dy < radius*radius))
    return count

# Pre-sort timing
t = time.perf_counter()
next_event_scan(world.pos_x[:10_000], world.pos_y[:10_000])
t_pre = time.perf_counter() - t

sort_for_locality(world, cell_size=10.0)

# Post-sort timing
t = time.perf_counter()
next_event_scan(world.pos_x[:10_000], world.pos_y[:10_000])
t_post = time.perf_counter() - t

print(f"pre-sort:  {t_pre*1000:.2f} ms")
print(f"post-sort: {t_post*1000:.2f} ms")
print(f"ratio:     {t_pre/t_post:.2f}×")

Expect a ~1.5-3× speedup on the post-sort version. The reason: post-sort, the pos_x[i+1:end] slice is more likely to contain creatures in the same spatial cell — so the boolean mask has more True values clustered together, and the subsequent indexing operations are more cache-friendly.

The exact ratio depends on the spatial distribution and the scan-window size. A scan window of 100 might capture exactly one cell (if cells average 10 creatures) or several adjacent cells; the locality benefit is biggest when the scan window matches the typical cell occupancy.

Exercise 5 — Sort cadence

results = {}
for cadence in (1, 10, 100, 1_000_000):       # last one = "never"
    world = build_world(n=10_000)
    t0 = time.perf_counter()
    for tick in range(100):
        motion(world, dt=1/30)
        if tick % cadence == 0:
            sort_for_locality(world, cell_size=10.0)
        next_event_scan(world.pos_x, world.pos_y)
    results[cadence] = time.perf_counter() - t0
for c, t in results.items():
    print(f"sort every {c:>8} ticks: {t:.2f} s total")

Typical shape:

sort every       1 ticks: 0.85 s   (sort cost dominates)
sort every      10 ticks: 0.62 s   (sweet spot, often)
sort every     100 ticks: 0.75 s   (scan cost grows as positions drift)
sort every 1000000 ticks: 1.20 s   (no resort; scan cost stays high)

The optimum is wherever the sort’s amortised cost balances the scan’s per-tick savings. For most simulators that’s “resort every 10-100 ticks,” depending on motion speed. A re-sort triggered by accumulated drift (resort once total motion since last sort exceeds half a cell width) generalises this to scenarios with variable motion rates.

Exercise 6 — Z-order curve (stretch)

def _spread2(v: int) -> int:
    """Interleave 16 bits of v with zeros (Morton helper)."""
    v &= 0xFFFF
    v = (v | (v << 8)) & 0x00FF00FF
    v = (v | (v << 4)) & 0x0F0F0F0F
    v = (v | (v << 2)) & 0x33333333
    v = (v | (v << 1)) & 0x55555555
    return v

def morton_cell(pos_x: np.ndarray, pos_y: np.ndarray, cell_size: float) -> np.ndarray:
    cx = np.clip((pos_x / cell_size).astype(np.int32), 0, 0xFFFF)
    cy = np.clip((pos_y / cell_size).astype(np.int32), 0, 0xFFFF)
    return np.array([(_spread2(int(x)) | (_spread2(int(y)) << 1)) for x, y in zip(cx, cy)],
                    dtype=np.uint32)

For pure-numpy efficiency, vectorise _spread2:

def spread_vec(v: np.ndarray) -> np.ndarray:
    v = v & 0xFFFF
    v = (v | (v << 8)) & 0x00FF00FF
    v = (v | (v << 4)) & 0x0F0F0F0F
    v = (v | (v << 2)) & 0x33333333
    v = (v | (v << 1)) & 0x55555555
    return v

def morton_cell(pos_x, pos_y, cell_size):
    cx = np.clip((pos_x / cell_size).astype(np.int32), 0, 0xFFFF)
    cy = np.clip((pos_y / cell_size).astype(np.int32), 0, 0xFFFF)
    return spread_vec(cx) | (spread_vec(cy) << 1)

Compared to the simple (cx << 16) | cy packing, Z-order keeps cells (1,0), (0,1), (1,1) close to (0,0) in the linear order — instead of (1,0) being adjacent to (0,0) but (0,1) being far away. The result is that 2D adjacency is approximately preserved in 1D adjacency.

For typical simulator workloads where next_event_scan looks at horizontal-and-vertical neighbours, Z-order outperforms simple packing by 10-30%. The difference is biggest for densely-packed simulations where vertical neighbours within the scan window matter.

The full Hilbert curve preserves 2D locality even better but is more expensive to compute. For most simulators, Z-order is the sweet spot — close to optimal, vectorisable in numpy.

29 — The wall at 10K → 1M

Concept node: see the DAG and glossary entry 29.

A simulator that runs cleanly at 10,000 creatures often grinds to a halt at 1,000,000. Not because the algorithm changed — because constant factors that were invisible at the smaller scale now bind.

This chapter is about finding the wall. The fixes are techniques you already have: hot/cold splits (§26), working-set discipline (§27), sort for locality (§28), pre-sized buffers, batched cleanup. The chapter’s job is to teach the reader to measure — to find which constant factors blew up.

Walls Python hits, named

• Pre-allocation skipped. A to_insert: list[CreatureRow] that grew lazily was fine at 100 appends per tick (10K creatures × 1% reproduction). At 10K appends per tick (1M × 1%), Python list append is amortised O(1) but each capacity doubling is an N-byte copy; at this scale the doublings dominate. Fix: pre-size with [None] * estimated_max plus an n_inserts counter, the same pattern §22 already uses.
• Linear scans in pure Python. A list comprehension [c for c in creatures if c.id == target_id] was 0.1 ms at 10K, but tens of milliseconds at 1M. Fix: the id_to_slot map (§23) plus parallel presence flags. In Python the linear-scan cost is sharper than in Rust — you pay interpreter dispatch on every iteration, ~5 ns per step from §1.
• Cache spillover. A creature working set at 10K is 200 KB (L2-resident). At 1M it is 20 MB (L3-resident). Per-element time triples. Fix: hot/cold splits + narrower numpy dtypes.
• The pandas wall. A pandas.DataFrame of 10M rows × 20 columns at default dtypes occupies 1.6 GB+ before any operation. A DataFrame.merge allocates intermediate copies; a groupby.apply materialises Python objects per row; both can OOM long before the data itself would. Fix: drop pandas. Either move to numpy SoA (when the working set still fits in RAM with explicit columns) or to sqlite (when it doesn’t, or won’t long-term). code/measurement/sqlite_performance_test.py shows sqlite delivers ~830K-900K random lookups per second on disk — fast enough to be the production answer for many workloads that pandas was struggling with. The migration is usually a one-day project that gives back days of OOM debugging per quarter.
• Per-tick allocation. A system that calls np.zeros(N) per tick was fine when N was 10,000 (40 KB). At N = 1,000,000 it is 4 MB allocated and zero-filled every tick — the malloc cost alone is significant. Fix: allocate the buffer once at startup, fill or reuse in place.
• Logging. A print(f"creature {i} ate") per event was tolerable at 10K. At 1M events it is the simulator’s bottleneck — print flushes, formats, dispatches the GIL. Fix: write to a numpy event log per §37, flush in bulk; or simply turn it off.

The pattern: any cost that was O(1) per creature, multiplied by 1M, is no longer free. Anything that was O(N) per tick at 10K is now O(N²)-equivalent in wall time. The fixes are local — each cost is a single-line change — but finding them requires measurement.

Measurement tools

The right tool is a profiler. In Python, three good options:

• cProfile (stdlib). python -m cProfile -o profile.out my_sim.py records every Python-level function call. Read with python -m pstats profile.out or snakeviz. Fine for finding hot Python functions; opaque to numpy internals (numpy ops show up as one C call).
• py-spy (third-party). py-spy record -o flame.svg -- python my_sim.py produces a flame graph similar to perf. Sees the C stack inside numpy ops, which cProfile does not. The right tool when the bottleneck is inside numpy.
• perf (Linux). The same tool the Rust edition uses. perf record -- python my_sim.py; perf report reads at the OS level; sees everything but interprets nothing — you read raw symbols.

The same simulator at 10K and 1M produces different flame graphs; the wall is the difference.

Calibration

A useful exercise: run your simulator at 10K for 1,000 ticks; time it. Run at 1M for 100 ticks (same total entity-ticks); time it. The 1M version should take roughly 10× longer, not 100×. If it takes 100×, something has crossed a constant-factor wall and the profiler will show you what.

The fix is structural. Apply the techniques: hot/cold, working set, sort for locality, pre-sized buffers, batched cleanup, deterministic structures. Each is a chapter you have already read. The wall is the moment they all become non-optional.

Exercises

1. Calibration. Run your simulator at N = 10,000 for 1,000 ticks. Time it. Note the wall-clock total.
2. Scale up. Run at N = 1,000,000 for 100 ticks (same total entity-ticks). Time it. Compute the ratio.
3. Profile with cProfile. python -m cProfile -s cumulative my_sim.py | head -30. Identify the top three hottest functions.
4. Profile with py-spy. py-spy record -o flame.svg -- python my_sim.py. Open the flame graph in a browser. Identify hot regions inside numpy that cProfile did not surface.
5. Pre-size cleanup buffers. Replace to_insert = [] plus to_insert.append(...) with a pre-sized array plus an n_inserts counter (the §22 pattern). Re-run; re-profile. The list-resize calls should disappear from the hot list.
6. Hot/cold split. Apply the §26 split organisationally. Re-run; re-profile. In numpy SoA you may see no change in the profile (per §26’s framing); in numpy structured-array form you should see a visible improvement.
7. Use index maps. Replace any linear np.where(arr == target)[0] lookup with the §23 id_to_slot form. Re-run; re-profile.
8. The pandas wall, hands-on. Build a pandas DataFrame of 5M rows × 10 float64 columns. Note its memory (df.memory_usage(deep=True).sum() / 1e6 MB). Now move the same data into 10 numpy float32 columns; note the memory ratio. Now move it into a sqlite table; note the disk size and a sample lookup time using sqlite_performance_test.py as a template. Decide consciously which form fits your workload.
9. (stretch) Find one new wall. Pick any system in your simulator and find one constant factor that scales worse than expected. The fix is usually one of the techniques above; identifying which one is the lesson.

Reference notes in 29_wall_10k_to_1m_solutions.md.

What’s next

§30 — Moving beyond the wall takes the next step: when even your fastest, tightest, hot/cold-split, sorted-for-locality simulator no longer fits in RAM, the architecture itself shifts.

Solutions: 29 — The wall at 10K → 1M

These exercises ask you to find the wall, not to remove it abstractly. The fixes are techniques you have from §26-§28; the diagnostic is the new content.

Exercises 1 & 2 — Calibration and scale-up

time python my_sim.py --n 10000 --ticks 1000
time python my_sim.py --n 1000000 --ticks 100

Both runs do the same total entity-ticks (10⁷). The wall-clock ratio is the diagnostic:

A 1.5-3× wall is normal and the chapter’s techniques close it. A 100× wall is a structural bug; nothing in this chapter fixes it short of recognising it.

Exercise 3 — Profile with cProfile

python -m cProfile -o profile.out -s cumulative my_sim.py
python -m pstats profile.out
> sort cumulative
> stats 30

Typical hot-list culprits at 1M:

• list.append — to_insert.append in a loop; pre-size to fix.
• numpy.ndarray.__getitem__ — accidental Python-level fancy indexing.
• <dict iteration> — id lookup via dict.get per creature when an id_to_slot array would be O(1).
• One named system that wasn’t supposed to be hot but is.

cProfile sees Python-level calls. Numpy primitives show up as one C-call entry (numpy.add or similar) regardless of how many elements they process. For numpy-internal hot spots, use py-spy.

Exercise 4 — Profile with py-spy

pip install py-spy
py-spy record -o flame.svg -- python my_sim.py
# then open flame.svg in a browser

py-spy samples the C stack, which surfaces numpy hot spots that cProfile lumps together. Typical findings:

• A np.where(...) over a column that could be a presence table.
• A bool-mask reduction ((arr > 0).sum()) that compiles to a slow path on int8.
• A np.argsort inside the tick that should run every 10 ticks (§28 cadence).

The flame graph’s width is wall time. Widest function is your bottleneck.

Exercise 5 — Pre-size cleanup buffers

# Before
class CleanupBuffer:
    to_insert: list[CreatureRow] = field(default_factory=list)

# After
class CleanupBuffer:
    def __init__(self, capacity: int):
        self.to_insert_pos_x = np.zeros(capacity, dtype=np.float32)
        self.to_insert_pos_y = np.zeros(capacity, dtype=np.float32)
        # ...
        self.n_inserts = 0

    def add_insert(self, pos_x, pos_y, ...):
        i = self.n_inserts
        self.to_insert_pos_x[i] = pos_x
        self.to_insert_pos_y[i] = pos_y
        self.n_inserts += 1

The Python list append is amortised O(1) but each doubling is an N-byte copy. At 10K inserts per tick that’s a 80K-byte copy every few ticks (negligible). At 100K inserts per tick the doublings happen often enough to be one of the hottest calls in the profile. Pre-sized arrays remove the doubling entirely.

Exercise 6 — Hot/cold split

In pure numpy SoA (where every column is its own array), splitting the row organisationally does not change the profile — the bytes were already separated. §26’s framing applies: the split is naming, not bandwidth.

If the simulator uses numpy structured arrays (one combined dtype for the whole row), the split shows up immediately. Motion’s arr['pos_x'] += arr['vel_x'] * dt runs at structured-array stride; splitting into hot_arr['pos_x'] += hot_arr['vel_x'] * dt runs at SoA speed. Expect ~8× improvement at 1M creatures.

Exercise 7 — Use index maps

# Before
def find_creature(world, target_id):
    return np.where(world.id == target_id)[0]   # O(N) per call

# After  
def find_creature(world, target_id):
    return int(world.id_to_slot[target_id])     # O(1) per call