The trapYou opened two hundred places to take a request — so why did throughput not rise to match?
0.0s · queue 0 · serving 0 · done 0 — nobody has arrived yet
Follow the sentences
1/48Two hundred counters are open. Why do only ten of them work?
Text mode advances to the next sentence every 3 seconds.
This bank has two hundred windows — in this chapter a window is one place that takes a request, which is a thread. The picture above draws six of them; the ×200 badge stands for the rest. But there are only two stamps for the paperwork.
A window without a stamp cannot call anyone up. So four of the six windows on screen stay empty, waiting for a stamp to come back, while the line grows — from outside it looks like plenty of staff and a queue that will not move. The picture keeps those customers in the line; on a real server the request is already holding a thread while it waits for a connection.
Raise the stamps from 2 to 6 above. Not one window was added, and the line all but disappears. What went up was not the number of windows but the number of jobs that can be started at once.
Now switch the call on. While a teller holds the receiver, that window cannot take the next customer — and the line grows again even though there are six stamps — each one stays in the hand of a customer who is still on the phone.
measured · computed just now
Now the same bank at production scale — more customers and more stamps than the picture: 200 windows, 200 milliseconds of work per job, 10 stamps, and 300 customers arriving every second. Only 50.0 customers a second actually finished and left; the other 250 a second piled into the line.
That number is not an accident. One stamp sends 5 customers a second on their way, so 10 stamps cap the bank at 50 customers a second — and the throughput measured above is exactly that ceiling. It stays true whether the bank has 200 windows or two thousand. Raise the stamps to 50 and the ceiling becomes 250 a second, measured at 250: exactly 5 times more.
Give the bank one stamp per window (200) and the ceiling goes to 1,000 customers a second, so nobody piles up in the line any more: 300 customers a second come in and 299 a second go out, with the windows busy only 29.9% of the time. Stamps are no longer the bottleneck.
Now let each teller hold one phone call for 560 milliseconds. The time a customer occupies a window goes from 200 to 760 milliseconds and the ceiling drops to 263 customers a second — lower by a factor of 3.8, with the stamp count still at 200. The stamps are not idle either: customers on the phone hold them while they wait, so stamp utilization is 99.8%. What sets throughput now is not how many stamps there are but how long each one is held. Even if the local work took no time at all, 200 windows ÷ a 560 ms call = 357 a second is the hard ceiling.
These numbers are not written into the sentence. They come from a simulation run while this page rendered.
In one sentence
How much can happen at once is set by the scarcest resource, not the number of seats — and anything that holds a seat longer (an external call) lowers that number too.
Progress is stored in this browser only (no account, no sync between devices).
1.With 200 seats and 10 stamps, how do you raise throughput?
2.One stamp per window, and throughput still stops at 263 customers a second. Why?
3.When an external API slows down, what runs out first on your server?