Server method ② — the event loop

Does every wait really need a person of its own? One clerk who places several calls and answers whichever bell rings can hold many conversations from a single seat.

See this first in FoundationsWhen does an event loop win, and when does it lose?

Turn it around: must a thread hold the waiting? Write down "when this finishes, run that function" and go do other work. Waiting becomes a line in a list instead of a sleeping thread. That is an event loop.

① Callbacks — handing over a function for later

Below, db.query() returns immediately and takes a function to run when the result arrives — a callback. The handler finishes instantly and the thread moves to the next request.

② The loop is one while statement

The second code box is the whole thing: pull a finished event off the queue, run its callback, repeat. A new connection is an event; a database reply is an event. The structure is startlingly simple.

③ Help from the kernel

Asking 10,000 sockets one by one whether data arrived would itself burn the CPU, so the OS offers "tell me which of these are ready" (epoll on Linux, kqueue on macOS). That is what made event loops practical.

④ What improved — compare with chapter 10

Same request load, one thread. Completion is comparable or better, thread memory drops from 4.0 to about 1, and switches fall to near zero. This model does not count changing requests inside a loop as switching OS threads; a real loop thread can still be preempted or sleep while waiting for events.

⑤ What epoll actually does for you

If the loop checked every connection on each pass it would slow down in proportion to the connection count — which is exactly what select did. epoll and kqueue register the sockets you care about with the kernel and hand back only the ready ones. With ten thousand connections you receive just the ten that are ready. The event loop became practical only after this kernel facility existed.

⑥ Backpressure — accepting fast is not always good

A loop can accept requests very quickly. The question is whether it can finish them just as quickly. When intake outruns processing, pending callbacks and buffers accumulate until memory gives out. Reactive stacks therefore let a consumer request only as much as it can handle. That mechanism is backpressure.

Handler (callback style) — register and return

1handle(req) {
2  id = parse(req)① CPU works
3  db.query(id, (user) => {② just registers a callback
4    res.send(toJson(user))③ the loop runs this later
5  })
6}  // returns immediately!

The event loop itself — this is all it is

1while (true) {
2  ev = queue.take()   // finished IO, new requests...
3  ev.callback()
4}

SpeedRequests

Try it in the simulator

1.Select WebFlux in the 'reactive stack' preset. ioWorkerCount is the number of event-loop threads. Other requests can progress while one waits for a database response.
2.Keep the load, seed and run duration fixed, then record CPU, throughput and latency at the defaults. Lower ioWorkerCount, reset and run for the same duration to compare.
3.Too few workers can queue CPU work; adding workers when enough are available may do little. Comparing different frameworks also changes request costs and settings, so their CPU difference cannot be attributed to thread count alone.

Try it in the simulator →

Terms in this chapter

non-blocking: Returning without waiting when an operation cannot proceed now. A socket read can return EAGAIN, and the caller retries later.
callback: A function handed over now to be run when something finishes.
event queue: Where finished IO and new requests line up for the loop.
C10K problem: The 1999 question of serving ten thousand concurrent connections from one machine.
epoll / kqueue: Kernel APIs reporting readiness of registered sockets. Readiness is distinct from I/O completion; the program still processes each returned event.
backpressure: Limiting intake to consumption speed; without it queues grow without bound.

Why it evolved this way

This is why nginx outperformed Apache and how Node.js and Redis hold tens of thousands of connections on a single thread: buy concurrency with a data structure instead of with threads.

Event loop = move the waiting out of threads and onto a list.

The remaining problem

Not free — the code shatters into callbacks. Let us fix that first →