Simulations
Each one runs the same code the videos are rendered from. Watch it play out, or take the parameter that causes the failure and try to undo it yourself.
Congestion collapse
That backing the retries off undoes the collapse. The audience expects the system to be a function of its current input, and it is a function of its history — the queue is by then deep enough to time out its own contents.
play it →
Coordinated omission
That a percentile is a property of the service. It is a property of the instrument: the same service measured by a closed-loop generator and by a fixed schedule produces p99s that differ by two orders of magnitude, and only one of them is what a user would have felt.
play it →
The GPU is full
That serving capacity is a compute number. The audience reads 'the GPU is full' as 'the GPU is working', and here it is neither — the arithmetic unit is idle enough to serve thirty-five times as many people, and the only thing that is full is a reservation nobody checked.
play it →