The Network Is Not Yours

Mira has a new laptop, and it is fast. Her window is wide open; she is advertising room for a megabyte at a time and reading as quickly as you can send. You start the video transfer again, and pieces go missing anyway.

You check. Her acknowledgements say, in effect, up to here, and room for plenty. Her buffer is nearly empty and her program is keeping up. The mechanism from Chapter 14 is working as designed, and it is telling you, truthfully, that the receiver has capacity to spare. And yet piece 800 is lost. Pieces 1210 through 1215 are lost. Piece 1900 is lost. Your timer fires, you resend, the resends mostly get through, and the transfer limps along at a fraction of what the wire should allow.

The pieces are not being dropped at Mira’s end. They are being dropped on the way. Consider what lies between you now. Mira is across the city, so there are boxes between you, the routers from Chapter 10, with queues in them that occasionally hold a piece too long or let one go. Each queue is a fixed amount of memory. Each router is forwarding not only your pieces but everyone’s. Theo is watching something, Priya is on a call, a hundred people you have never met are doing a hundred things, and all of it passes through the same handful of boxes. When a queue is full and one more piece arrives, that piece is dropped, and it does not matter whose it is.

Your video is being sent as fast as your wire allows, and your wire is fast. The box three hops away is not, or rather it is shared, and its share for you is smaller than what you are sending. The pieces that do not fit are gone.

Two different things are at work in this chapter and the one before it, and they deserve to be kept apart, because they produce the same symptom and are constantly confused. Chapter 14 was about the receiver’s limit. Mira’s machine could hold only so much unread data, so she told you how much room she had, and you never sent more than that. The signal was explicit, a number in every acknowledgement, and it came from the far end. This chapter is about the path’s limit. The boxes between you can carry only so much at once, that capacity is shared with strangers, and it changes from moment to moment. Nobody sends you a number. There is no field in any header that says the third router has room for forty more. The only signal you get is that pieces disappear.

The symptom is the same in both cases, loss followed by retransmission. The cause is different, being either the far end or the middle, and the signal is different, being either a number or silence. The remedy, as you are about to see, is different as well, and it runs alongside the first. A sender is bounded by both limits. It may never exceed the receiver’s window, and it may never exceed what the path can take, and whichever of the two is smaller is the one that binds.

So how much can the path take? You cannot ask, because the boxes do not answer questions; they forward or they drop. You cannot look it up, because the answer depends on what a hundred strangers are doing right now and will be different in a second. You cannot even measure it directly, because the only way to find out whether a box has room is to send something and see if it survives. That, in fact, is the method.

Send a little. If it all arrives, send a bit more, and if that arrives, more still. Keep increasing until something is lost, because loss is the path telling you, in the only language it has, that you have found its limit. Then back off, though not to zero, since you know the level just below was fine. Back off some way and start creeping up again. You will hit the limit again, back off again, and creep up again. You will never know the number, but you will be sending at roughly the right rate, continuously adjusting to a capacity you cannot see.

That is the whole idea, and what you have built is what the field calls congestion control. The amount you are allowing yourself to have in flight on the path’s account, alongside the receiver’s, is the congestion window, and it goes up and down forever in a pattern that looks like the teeth of a saw.

A few details of the shape matter, because you will see their effects. The probe starts small. When a connection begins you know nothing about the path, which might be a courtyard or an ocean, so you start with a handful of pieces and double every round trip while nothing is lost: two, four, eight, sixteen, thirty-two. Doubling finds the right order of magnitude quickly. This phase is called slow start, which is a strange name for the fastest part of the curve. It is slow only relative to sending at full speed from the first byte, which is what senders did before 1988, and which is why 1988 matters.

The probe slows down as it nears the limit. Doubling would overshoot badly, so past a certain point the growth changes from doubling to adding, one more piece per round trip. This creeping finds the edge without blowing past it. When a piece is lost, the window is cut, traditionally in half. That is a big step, and it is big on purpose. If the box is full it is full for everyone, and a gentle reduction would not clear it. Then, from the halved window, the probe goes back to adding one per round trip. Up to the edge, loss, halve, creep. That is the sawtooth.

You might wonder whether all this politeness is necessary. The path drops your pieces, you resend them, and eventually they get through. Why not keep sending at full speed and let the boxes sort it out? The answer is that everyone else is asking the same question.

In October 1986 the link between two university campuses in California, a link that had carried a steady thirty-two kilobits per second, abruptly dropped to forty bits per second. Nothing had failed; the wire was fine. The link had collapsed under its own traffic. Every sender was sending as fast as it could. The queues filled and pieces were dropped. Every sender’s timer fired and every sender resent, into queues that were still full, so those pieces were dropped too, and resent again. The link was running at capacity, and almost every piece on it was a retransmission of something that had already been dropped, being dropped again. Useful throughput fell by a factor of nearly a thousand. This is called congestion collapse, and it is what a shared path does when senders do not back off.

The backoff is not courtesy. It is the difference between a network that carries traffic and one that carries only retries. Each sender halving its window on loss is what lets the queues drain, which is what lets anyone’s pieces get through, including that sender’s own. Being polite is the fast strategy.

There is a fairness result buried in here as well. If two senders share a bottleneck and both follow the rule of adding one and halving on loss, they converge on an equal split of the capacity, regardless of how fast either of them started. The one with the bigger window loses more when it halves, so the gap closes a little every cycle. Nobody coordinates this. It falls out of everyone following the same rule.

All of this happens in the operating system, below your program. You do not set the congestion window, and you cannot see it without asking. But you feel it, and you feel it in one particular place. Every new connection starts slow. Slow start begins from a handful of pieces, and for the first few round trips a connection is sending a tiny fraction of what the path could carry, doubling each time. If the thing you are sending is small, a chat message or a short web page, the transfer finishes before the window ever gets large. You never reached the path’s capacity. You paid the full slow-start ramp for a transfer that did not need it.

This leads directly to the one lesson from this chapter that will change how you write programs, which is to reuse connections. A connection that has been open and carrying traffic has a warm congestion window. It has already probed the path and knows roughly what it can send. A fresh connection knows nothing and starts from nothing. If your program opens a new connection for every message, every message pays the ramp. If it keeps one connection open and sends every message down it, only the first one does. Add the handshake from Chapter 12, a full round trip before any data, and a fresh connection is expensive twice over, once to set up and once to speed up.

You will see this measured in Part II, when a round trip is a hundred milliseconds rather than one and the cost of a fresh connection becomes most of the wait. For now the principle is enough. A connection is a thing that gets better with use, and you should not throw it away.

Your video now crosses a busy city at whatever rate the busiest box along the way can bear, adjusting as strangers’ traffic comes and goes, without anyone ever telling you the number. Mira’s fast laptop is no longer the bottleneck. The path is, and the path is not yours.

Two chapters have given you two limits. The receiver’s you learn by being told; the path’s you learn by probing. Together they are the whole of the question of how fast you may send, and both are answers to the same question you first met with the flipped bit in Chapter 3, which is what to do when a piece does not arrive. The next chapter is about what happens when the answer to that question, which has always been to resend it, is the wrong answer.