Why is one fetch instant and another slow?

The trapBoth are just "fetch some data" — so why does one take 0.000001 seconds and the other 0.1?

See the picture

The ladder of waiting — if the desk drawer took one second

If reaching into the nearest desk drawer took one second, everything else stretches by the same factor.

desk drawerread from L1 cache0.5 ns1 s
bookshelf in the roomread from main memory100 ns3.3 min
storeroom downstairsone SSD read150 µs3.5 days
the building next doorround trip inside one datacenter0.5 ms11.6 days
warehouse across townhard disk head seek10 ms7.6 months
another cityintercontinental round trip150 ms9.5 years

While you wait for one external call, the CPU could have run hundreds of millions of instructions.

Source: Jeff Dean · Peter Norvig, “Latency Numbers Every Programmer Should Know” (human-scale conversion: hellerbarde). Bars are on a log scale.

Follow the sentences

1/39

Text mode advances to the next sentence every 3 seconds.

Picture it this way

Reaching into your desk drawer, walking to the bookshelf in the room, going down to the storeroom, stepping into the building next door, driving to a warehouse across town, sending someone to another city — six different errands. The thing you fetch may be identical; the time is not.

A computer has exactly the same ladder. Nearer rungs are faster but smaller; farther rungs are slower but bigger. The device that keeps copies of hot values on a near rung is a cache.

In the table above, the badge on the left is that distance and the right-hand column is the same duration stretched into human time. The trap is that in code, every rung looks like one ordinary line.

Now the numbers

computed · just now

Say that reaching into the nearest desk drawer took 1 second. Then the bookshelf in the room (memory) is 3.3 minutes away, and the storeroom downstairs (disk) is 3.5 days away.

The trip to another city — a round trip across an ocean — is 9.5 years. That is 300,000,000 times farther than opening the drawer. In code, both are one line.

These numbers are not written into the sentence. They are worked out while this page renders.

Try it in the simulator

1.Open the 'add cache' preset — the Redis node's cache hit rate (%) is 95. Run it for 10 seconds and Redis answers 13,915 requests instantly while MySQL behind it sees only 6,685.
2.Drop that number to 50 in the inspector. Misses — trips to the far shelf instead of the near drawer — go up 10x, and MySQL takes the extra traffic. That is this chapter's ladder showing up as throughput.

Watch how many requests the MySQL card receives as you drop cache hit rate from 95 to 50.
Every request the cache misses travels one rung farther down the ladder.

Open it in the simulator →

In one sentence

The difference in speed comes from how far the trip is, not from how hard anything is trying.

Progress is stored in this browser only (no account, no sync between devices).

Quick check

0 / 3 correct

1.Which best explains why we put a cache in front of a database?
2.As you step one rung farther down the ladder, what grows?
3.If the cache hit rate falls from 95% to 50%, what happens to the traffic reaching the database behind it?

New words here

cache: A small nearby store holding copies of the values you reach for most often.
cache hit rate: How often what you came for is already in the cache, so no long trip is needed. At 95%, only one visit in twenty goes far.
latency: The time you wait for the value you asked for to arrive. A different question from throughput, which counts how many finish per second.