The exam › What you now own
Three ideas and one formula
Nine chapters gave you three tools. One: latency, bandwidth, and capacity are three different quantities with three different costs (0.3). Never let one number impersonate another. Practitioners keep the first two apart operationally: latency is how fast one byte comes back, bandwidth is how many bytes per second flow once the stream is going. Two: the hierarchy exists because fast memory cannot be big (0.2), so every system you will ever meet is a stack of small-fast over big-slow (0.5, 0.8). Three: for anything that streams data,
time ≈ max(bytes ÷ bandwidth, work ÷ compute)
And when access is dependent and irregular, latency × count takes over instead (0.9). This capstone is spine idea three rehearsed until it is a reflex. The estimation protocol is always the same four moves: state your assumptions (bytes moved, access pattern); compute both bounds, the bandwidth bound (bytes ÷ bandwidth) and the latency bound (dependent accesses × latency); keep whichever binds; then sanity-check against the reference cardbelow. Predict on paper, then measure, then explain the gap. No reading ahead to the answer; that's the whole pedagogy.
The reference card › Latency numbers
The numbers everyone should know
Jeff Dean's famous list, in its maintained modern form: Colin Scott's interactive, year-adjusted version (linked in this chapter's resources). Bookmark this table; every estimate in this course starts from it or from the reference machine's spec sheet.
| L1 cache reference | 1 ns | |
| L2 cache reference | 4 ns | |
| Main memory reference | 100 ns | |
| Read 1 MB sequentially from memory | 10 µs | |
| NVMe SSD random read | 20 µs | |
| Read 1 MB sequentially from SSD | 200 µs | |
| Round trip within same datacenter | 500 µs | |
| Read 1 MB sequentially from disk | 5 ms | |
| Packet CA → Netherlands → CA | 150 ms |
These numbers are not folklore: they are measurable, and 0.3 taught the method. Idle latencyis measured by pointer chasing (each load's address depends on the previous load: 0.9's dependency chain, weaponized as an instrument), while bandwidth is measured with independent streaming loads. On a current Alder Lake P-core, that method clocks DRAM idle latency at ≈90ns with a large buffer, which is exactly where our reference machine's spec-sheet number comes from.
Two habits when you use the card. Ratios matter more than values: one DRAM reference costs ~100 L1 hits, and that ratio survives hardware generations even as both numbers drift. And units are where napkin math dies: keep everything in ns and bytes until the last line.
Predict › Warm-up
Two estimates, no calculator
Predict first
memcpy(dst, src, 1 GB) on the reference machine (76.8 GB/s DRAM). Roughly how long?
Predict first
From the table: reading 1 MB sequentially from memory vs from an SSD. What's the ratio?
The number › The finale
The 0.6 experiment, derived from first principles
In 0.6 you watched row-major and column-major traversal diverge. Now run the full protocol on it, before touching a keyboard. Sum a 4096 × 4096 float matrix. Assumptions:64 MiB of data; row-major streams it once (bandwidth bound binds); column-major strides 16 KiB between consecutive accesses (every access a fresh line, prefetcher defeated), so the latency bound binds (0.9's serialized-miss regime).
Compute the number › matrix sum, both traversal orders
| matrix | 4096 × 4096 × 4 B = 64 MiB |
| row-major: bytes ÷ bandwidth | 67 MB ÷ 76.8 GB/s ≈ 0.9 ms |
| column-major: one miss per element | 16.8 M misses |
| misses × DRAM latency | 16.8 M × 90 ns ≈ 1.5 s |
| Result | 0.9 ms vs ~1.5 s: three orders of magnitude, on paper |
The honest footnote: 1.5 s is the serialized upper bound; it assumes zero overlap between misses. Real out-of-order hardware keeps several misses in flight at once (0.9), so measured gaps land closer to 10–50× than 1000×. That is napkin math working as designed: the two bounds bracket reality, and they always get the direction and the mechanism right.
Measure › Close the loop
Now run it, properly
Predictions unmeasured are just vibes. Write the two loops (any language with real arrays), and measure the way performance engineers actually measure. Warm up and discard the first run: it pays page faults (0.8) and cold caches; but know that a dry run doesn't purge every bias, since even things like environment-variable size and link order have been shown to skew results unpredictably. Repeat 5–10 times and look at the distribution, not one number; if the spread is on the same order as the mean, don't compute a speedup at all; fix the noise first. Stabilize the machine (plugged in, cool, nothing heavy in the background): dynamic frequency scaling can hand one run a turbo boost the next run never gets, a classic way two identical binaries differ by hundreds of milliseconds, especially on laptops.
Your numbers will not match this page's. They were sanity-run on Apple M4 (author's MacBook), whose memory system differs from the reference spec sheet and from your machine. And that is the point: the order of magnitude is the deliverable, and the ratio between your two loops is the lesson. If your gap is 10× instead of 40×, explain it with the tools you now own. That explanation is passing the capstone.
interactive · coming soon
NapkinQuiz: predict-then-measure engine
Planned interactive: a quiz deck where every question locks until you commit an estimate; then it reveals the derivation, and hands you a runnable snippet to measure the real value on your own machine and log the gap.
SourcesBakhvalov 2e Ch. 2 · pp. 24–32Bakhvalov 2e §4.10 · pp. 88–90Dean/Scott interactive latency numbers
The full story › when the napkin lies to you
Napkin estimates land within ~2× when one resource clearly dominates and access is regular. They miss when the model omits a mechanism: memcpy can beat your estimate (non-temporal stores skip the read-for-ownership traffic) or lose to it (page faults on first touch); random access can cost double when TLB misses stack on cache misses (0.8); loaded systems add queueing that idle-latency numbers never see, so measured latency under load is a different, larger number than the idle latency we quote. The professional habit is not a better formula; it is treating every 3×-off measurement as a mechanism you haven't named yet, then going and naming it.
You can now put a number on a loop before running it: bytes over bandwidth when access streams, latency times count when it chains. And you know to measure, expect ±2×, and interrogate anything worse. That reflex is Topic 0's entire payload, and in Topic 1 it gets a name: the roofline.