The Path Has Texture

Courtyard works across the city now. Mira’s connection is fast and so is yours, and the chat is a little laggy all the same. There is a beat between pressing enter and seeing the tick appear. The video call stutters now and then. Downloads, on the other hand, are quick.

You telephone your provider and pay for a faster plan. Downloads get faster still. The chat is precisely as laggy as it was, and the call stutters just as often as before. You have bought something, and it was not the thing you were missing. To see what you were missing, the word “fast” has to be taken apart into three quantities that are usually mashed together into one.

The first is how much the path can carry per second. The field calls this bandwidth, or throughput when it means what you actually got rather than what the link is rated for. If you think of the path as a pipe, this is the pipe’s width. A gigabit link can carry a hundred and twenty-five megabytes a second, if everything else cooperates.

The second is how long one trip takes, from the moment your machine puts a piece on the wire to the moment it arrives at Mira’s. This is latency. Because almost everything in Part I was a question followed by an answer, the number you will usually care about is the round trip, there and back. Latency is the length of the pipe. Across a courtyard it is well under a millisecond. Across a city it is a few milliseconds. Across an ocean it is a good fraction of a second.

The third is how much that trip time varies, and it is called jitter. Latency is not one number but a distribution. Piece one takes twelve milliseconds, piece two takes nine, piece three takes forty. Jitter is how wide that spread is.

The three are independent of one another. A very wide pipe can be a very long one. A truck full of hard drives driving across the country has staggering bandwidth, many terabytes an hour, and a latency of a day. A courtyard wire has trivial bandwidth by comparison and a latency you cannot perceive. When you paid for the faster plan you widened the pipe. You did not shorten it, and you did not make it any steadier.

Now take the three things you were doing and ask which quantity each of them depends on.

The download is a three-hundred-megabyte video file. What matters for it is how many bytes per second you can pull, and once Chapter 16’s slow start has ramped up, that is bandwidth. Latency matters a little, since it sets how quickly slow start ramps and it caps throughput when the window cannot cover the pipe, but for a big file the width of the pipe wins. You bought bandwidth and the download got faster, which is what should have happened.

The chat message is a hundred bytes. It fits in a single piece, and bandwidth has nothing to say about it; the pipe could be a millionth as wide and a hundred bytes would still cross it in an instant. What you are waiting for is the round trip: your piece to Mira’s server, the acknowledgement back, the tick appearing. That is latency. It is set by the distance and by the boxes between you, and no plan you can buy shortens it. The lag you feel in the chat is the length of the pipe.

The call sends thirty frames a second, and each ought to arrive thirty-three milliseconds after the last. If latency were constant, with every frame taking, say, forty milliseconds, the call would be smooth, only delayed. The stutter comes from variation. One frame takes thirty milliseconds and the next takes ninety, and the player either shows nothing while it waits or gives up on the late frame. That is jitter. The jitter buffer of Chapter 17 smooths it by holding frames for a moment before playing them, and the larger the jitter, the longer it must hold them, and the more the call lags. Large jitter forces you to choose between stutter and delay.

So bandwidth is for bulk, latency is for back-and-forth, and jitter is for anything live. Three applications, three different complaints, and the plan you bought addressed one of them. This is why “is your internet fast?” is a question that cannot be answered as asked.

Each of the three quantities comes from a different place. Bandwidth is set by the narrowest link on the path, the bottleneck, and by how many other people are sharing it, which was the subject of Chapter 16. It is a product. You can buy more of it, up to what the bottleneck allows.

Latency has three parts. There is distance, which is the subject of the next chapter and which is the floor beneath everything. There are hops: every box on the path takes a little time to read the header and forward the piece. And there is queueing, the time a piece spends waiting in a box’s queue behind other people’s pieces. The first is physics. The second is small and roughly constant. The third is the interesting one.

Jitter is almost entirely queueing. Distance does not vary from piece to piece, and hop count barely does. What varies is how deep the queue was in each box at the instant each piece arrived. A box that is nearly empty passes a piece straight through. The same box a moment later, with a burst of someone’s download in front of you, holds your piece for the length of the burst. That is why jitter tracks load: an empty path is steady and a busy path is spiky. It is also what brings us to the fourth dial, and to a trap.

Boxes drop pieces when their queues are full; Chapters 10 and 16 covered this. Drops are loss, and loss is bad, and a manufacturer of routers, looking at a drop counter, is bound to wonder what would happen if the queue were bigger. Memory is cheap. Give the box a queue so deep that it never fills, and it will never drop anything. Let’s build that box and watch it.

Under light load nothing changes, because the queue is mostly empty either way. Under heavy load, with your download running flat out, the queue starts to fill. Before, it filled to a modest depth and then dropped. Now it fills, and goes on filling, and does not drop, because there is room. It holds a second’s worth of pieces, then two seconds’ worth. Every piece that arrives, your chat message and your video frames among them, goes to the back of a two-second queue and waits its turn. Your chat’s latency has gone from twelve milliseconds to two seconds, not because anything is far away, but because everything is waiting inside a box that was built never to say no.

The situation is worse than that, because of Chapter 16. The download’s sender is probing for capacity, and its only signal to back off is loss. With the giant queue there is no loss; pieces are held rather than dropped. So the sender never backs off. It keeps the queue permanently full, because nothing tells it not to. The big buffer did not merely add delay. It removed the one signal that would have prevented the delay. Loss was information, and the buffer swallowed it.

The field calls this bufferbloat, and for years it made most home connections far laggier under load than they had any physical reason to be. The fix follows directly from the diagnosis and is counterintuitive all the same. Make the queues smaller, or build boxes that deliberately drop, or mark, a few pieces before they are full, so that senders learn early and the queue never grows deep. A little loss, on purpose, keeps latency low.

You have seen this shape before. Chapter 14 gave a producer three choices when its consumer is slow: it can wait, it can drop, or it can buffer without bound. The box that never drops chose the third, and got the result the third always gets. A buffer converts a rate problem into a delay problem; it does not solve it. That is true of a router’s queue, of a program’s write queue, of a message broker, and of every other place where you will be tempted to add memory instead of making a decision.

It is fair to ask, then, what you can and cannot fix. Bandwidth you can buy, up to the bottleneck, and you can also reduce how much of it you need by sending smaller things. Jitter you can partly tame, by keeping queues short where you control the box, and by smoothing at the receiver with a small buffer at the cost of a little latency. Latency you cannot buy. You can trim the queueing part by keeping paths uncongested, and you can trim the hop part slightly. The distance part, which is the largest part across anything bigger than a city, is fixed by where the two machines are. The only way to reduce it is to move one of them. That sentence is the pivot the rest of the book turns on, and the next chapter puts numbers to it.