Give the same work to four different methods — what changes, and by how much? The same 24 customers pass one counter four ways, with the time taken and the price paid written side by side.
See this first in FoundationsWhy does a busy server fall behind?These numbers are simulator output, not hand-written: one thread (chapter 7's world), pools of 4 and 64 (chapter 10), and the event loop (chapter 11), all under identical conditions.
Makespan is total time to finish all 24; switches counts context switches; memory is thread stacks; CPU busy is the fraction of time the core actually computed. One thread is dramatically slow, four is much better, and sixty-four buys no more speed while costing sixteen times the memory.
Light per-request computation with very many connections favours the event loop on memory. Heavy computation, or unavoidable blocking libraries, strongly favours the thread pool — it survives the traps of chapters 13 and 14. The answer depends on the workload, not on taste.
Makespan is the time to finish all twenty-four. But a user experiences only their own request. So in practice you look at the distribution of per-request latency. Requests that arrive early skip the queue and finish fast; later ones carry their queue time on top. Averages erase that difference, which is why the tail — p99 — is tracked separately.
Requests in flight equal throughput multiplied by response time. Throughput here is capped by the core. If the core is already saturated, more threads leave throughput unchanged and only raise the number of requests waiting — which shows up purely as longer response times. Nearly equal makespan at sixteen times the memory, in the table above, is exactly this effect.
Three questions. How much CPU does one request use — light computation with mostly waiting favours the event-loop family. Are there blocking libraries you cannot avoid — then a thread pool is safer. And can your team observe and debug the model — asynchronous code with broken stack traces bills you during incidents.
| model | makespan | switches | thread memory | CPU busy |
|---|---|---|---|---|
| Thread pool · 1 | 192.0 tu | 0 | 1.0 | 25% |
| Thread pool · 4 | 65.8 tu | 45 | 4.0 | 73% |
| Thread pool · 64 | 62.1 tu | 47 | 64.0 | 77% |
| Event loop · 1 | 48.0 tu | 0 | 1.1 | 100% |
Try it in the simulator
Terms in this chapter
Why it evolved this way
The four rows are not four different technologies but four answers to one question: what do you hold on to while waiting? One thread overlaps nothing — simplest and slowest. A pool attaches a whole stack to each wait: the code stays ordinary, and memory is the price. An event loop attaches nothing and merely registers with the kernel: almost no memory, provided everyone yields quickly. Coroutines and virtual threads keep that third answer while returning the code to the shape of the first. So this is not a performance comparison but a choice of cost structure — decided by what is expensive for your team and your service.
Choosing a model = choosing where the waiting lives, and what you pay for it.