When Reliability Hurts

Mira calls you on video. Her camera captures thirty frames a second. Each frame is compressed and sent, a piece or two per frame, thirty times a second, down the reliable stream you have spent six chapters perfecting. It works very well. Her face moves smoothly on your screen, and every byte of every frame arrives intact and in order, because that is what the stream does. Then the picture freezes.

What happened is this. Frame 412, one thirtieth of a second of Mira’s face, was in a piece that a router dropped, and the machinery of Chapter 11 did what you built it to do. Your operating system received the pieces for frames 413, 414, 415 and onward, and placed them by number in the buffer. Then it held them there, because there was a hole at 412, and the stream promises in-order delivery. Your program is never handed byte 413 before byte 412. So your video player, calling read, got nothing. Frame after frame arrived and was held, and the picture stopped.

Meanwhile Mira’s operating system noticed the repeated acknowledgements, up to 411, up to 411, up to 411, and resent 412. That took a round trip across the city, plus a little. Piece 412 arrived, the hole was filled, and your operating system released everything it had been holding, forty frames, all at once. Your player rendered them as fast as it could, and Mira’s face jumped forward two seconds and carried on.

Every byte was delivered, in order, reliably, and the call was unwatchable for two seconds, all so that a frame from two seconds ago could be shown. It was not shown, as it happens. By the time it arrived your player had thirty newer frames and skipped it anyway.

The name for what you just watched is head-of-line blocking. One missing piece sits at the head of the line, and everything queued behind it waits, not because the pieces behind it are missing but because the stream refuses to deliver them out of order.

For a file, that is correct. Byte 413 of a video file is meaningless without byte 412, and you would rather wait than have a corrupt file. For a photo in a chat it is correct, and for a message it is correct. For anything where the bytes have to be all present and in sequence to be worth anything at all, holding back the later ones until the earlier one arrives is the right behaviour. For a live video call it is the wrong behaviour. A frame from two seconds ago has no value, and nobody wants it. What has value is the most recent frame, shown now. If 412 is lost, the right thing to do is show 413, a thirtieth of a second later, with a barely visible flicker, and never think about 412 again.

The stream cannot do that. There is no way, in the agreement from Chapter 11, to say that a piece should be skipped. In-order delivery is not a feature the stream has; it is what the stream is. The numbering, the holding, the filling of holes, the release in sequence, the whole mechanism exists to guarantee that the program sees every byte in order, and you cannot turn off the guarantee without turning off the mechanism. So we turn it off.

Go back through the tower and ask, for each rung, whether a video call needs it. Framing, the check, identity, location and resolution, which were Chapters 2 through 6, are all needed. Those are about getting a frame to the right machine intact, and nothing about video changes that. Ports, from Chapter 7, are needed, because the video has to reach the video program and not the chat.

Conversation identity, from Chapter 8, is needed in part. The four-tuple part stays, since you still need to know which call these bytes belong to. But the part where each conversation gets its own socket and the operating system does the sorting is optional. A video program can perfectly well take every arriving piece, tagged with where it came from, and sort for itself. That was the fork in Chapter 8, and this is the branch you did not take.

Numbering, from Chapter 11, is needed, though in a particular way. The player needs to know that this is frame 413 so that it can show frames in order when they arrive in order and skip when they do not. But the holding, the refusal to deliver 413 until 412 arrives, is not wanted. Nor are acknowledgements and retransmission, since a resent frame is a late frame and late frames are worthless. The handshake from Chapter 12 is not needed either. There is nothing to agree on if there are no sequence numbers to align, and the first piece can simply be the first piece. Flow control from Chapter 14 is not wanted. A receiver that is too slow for a video call should drop frames rather than ask the sender to pause, because pausing a live call makes it not live. Congestion control, from Chapter 16, is a subtle case, and we will come back to it.

What is left is small. Name the program at both ends, check the bytes, and send each piece on its own, with no numbering that the transport cares about, no acknowledgement, no holding and no handshake. Every piece is complete in itself. It arrives or it does not, and either way the next one is independent of it. Such a piece is called a datagram, and the transport that carries it, with an eight-byte header holding two ports, a length and a check and nothing else, is called UDP.

You now have two ways to send anything, and you will need to choose between them for everything you ever build. The choice comes down to a single question, which is whether a late byte still has value. If it does, as with a file, a message, a page, a photo, a transaction, or anything else where you would rather wait than have it wrong or incomplete, you use the stream, and get every byte, in order, however long it takes. If it does not, as with a voice, a video, the position of a player in a game, or a sensor reading that is replaced by a newer one every second, you use datagrams, and get the most recent thing now, forgetting whatever did not make it.

There is a third case that surprises people, which is a single small question with a single small answer. You look up a name and get an address. It is not live, and a late answer is fine. But a stream costs a full round-trip handshake before the question can even be asked, and then a teardown afterwards, all for one message each way. On datagrams, the question is the first message and the answer is the second. If the answer does not come, the program asks again, which is a retransmission, but a trivial one, and one the program controls. For tiny exchanges, the stream’s setup cost outweighs its guarantees.

Choosing datagrams does not make the path any kinder. Pieces still get lost, reordered and duplicated. All you have done is move the responsibility for dealing with that from the operating system into your program, so consider what a video program actually has to do.

When a frame is lost, the program shows the next one. Better, it conceals the gap, by repeating the previous frame or interpolating between neighbours, so that a thirtieth-of-a-second flicker becomes invisible. The program decides, because the program knows what a frame is and the operating system does not. When frames arrive out of order, the program puts a small sequence number of its own in each frame, in its own header, and keeps a tiny buffer a few frames deep, so that a frame arriving a moment late can still be slotted in. This is Chapter 11’s numbering rebuilt at a different scale and with the opposite policy: hold for a few milliseconds at most, then move on. The buffer trades a little latency for a good deal of smoothness, and its depth is one more timeout with no right answer. It is called a jitter buffer. When a frame arrives twice, the same sequence number tells you so, and you drop the second copy.

Congestion is where the question we held comes back. Chapter 16’s backoff lives in the stream’s transport, and datagrams have none. A video program that sends thirty frames a second regardless of what the path can carry is precisely the sender that caused the 1986 collapse, and if every video program behaved that way the path would collapse again. There is no operating system enforcing politeness now, so the program has to do it: watch for loss, or for rising delay, and reduce quality, with smaller frames or fewer of them, until the path recovers. Every well-behaved real-time application does this, and the ones that do not are the ones that make everyone else’s calls worse.

So the video program ends up rebuilding a portion of Chapters 11 through 16 on its own, with its own priorities. That is the honest cost of the second transport. The tower still needs those rungs; you have chosen to build them yourself, differently, because the standard ones were built for a different purpose.

You have two transports. The stream is for everything where completeness matters, and datagrams are for everything where recency matters, with a single question to choose between them. Mira’s video call flickers instead of freezing, and the flicker is invisible. The chat, which is made of messages, which are things where every byte matters, stays on the stream, where it belongs.

Which brings you back to the stream, and to a problem you left behind on the very first floor of the tower. The stream delivers bytes. It does not deliver messages.