Is Anyone There?

You are chatting with Mira. She is in the middle of a sentence, and you can see that she is typing, when her laptop battery dies. This is not a shutdown and not a close. The screen goes black and the machine is off. No FIN was sent, because the operating system that would have sent it stopped existing between one instruction and the next. No reset was sent either. Nothing left her machine at all, and the question is what your machine sees.

It sees nothing, and it will go on seeing nothing. Your connected socket is exactly as it was. The four-tuple is in the table and the buffer is there. Your program, in its loop from Chapter 9, is parked at the top waiting for readiness, and the socket is not ready, because no bytes are arriving, because nothing is sending them. There is no error and no notification. From your side Mira has gone quiet, and quiet is what a socket looks like most of the time. She might be thinking. She might have stepped away, or be typing a long reply, or she might be a dark laptop on a desk. Your operating system cannot tell the difference, because at the level of the tower you have built there is no difference. The wire is idle, and an idle wire looks like an idle wire.

Now suppose you send something. Your piece goes out, Chapter 11’s timer starts, and no acknowledgement comes back. The timer fires, you resend, and nothing happens. The timer doubles, which is sensible since the path might be congested, and fires again; you resend, and again nothing. Your operating system will keep this up for a long time, with the intervals growing, before it finally gives up and tells your program that the connection has failed. On most machines that is somewhere between several minutes and a quarter of an hour. And it only happens if you send. If you are only listening, the connection sits there, apparently healthy, for as long as your machine stays on.

Look at it from the other side as well. Mira plugs in and reboots. Her operating system comes up with no memory of any connection, because the table was in RAM and RAM is gone. Your machine still believes the conversation is open. The two ends now disagree about whether they are connected, and neither knows that the other disagrees. When you eventually send, her operating system sees a piece for a four-tuple it has never heard of and replies with a reset, and then you find out. Until you send, nothing.

This is called a half-open connection, open at one end and gone at the other. It is the natural consequence of a design in which every change of state is announced by a message, because a machine that dies cannot announce anything.

For your chat window this is an annoyance. You will notice eventually, or you will send something and get an error. For Mira’s Courtyard server it is a slow leak. Every person whose laptop dies, whose phone loses signal in a tunnel, whose Wi-Fi drops as they walk out of range, leaves behind a connected socket on her machine that will never carry another byte. Each one holds a buffer and a table entry. Each one is in her program’s list of handles, being watched by the loop, forever. Over a week there are hundreds of them, and over a month thousands. Her machine is holding memory and bookkeeping for a crowd of people who left without saying goodbye, and it has no way to know, because absence and silence are the same thing.

There is a deeper reason too, which you cannot see yet because you have not left the building. In Part II you will meet boxes in the middle of the path that keep their own tables of conversations and forget any conversation that goes quiet for a few minutes. When they forget, the conversation is dead in the middle even though both ends are alive. The only defence against that will be the thing you are about to build.

The problem is that silence carries no information, so the remedy is to stop being silent. You agree with Mira’s machine that, every so often, each side will send a tiny message that means nothing except that it is still here, and that the other side will answer with the same. It is not data, and it is not part of the conversation; it is a pulse. If you send one and get the reply, she is alive. If you send one and get nothing, you send another. If a few in a row go unanswered, she is gone, and you can close the socket, free the buffer, and tell your program.

That is the whole mechanism, and it is called a heartbeat, or, when the operating system does it for you on a connection, a keepalive. It is a small thing. It is also the only way, anywhere in the tower, that one machine can ever learn that another has stopped rather than merely paused. Every failure detector at every layer above this one is a heartbeat wearing different clothes.

There is a reason this is a chat chapter and not only a plumbing chapter. Once Mira’s server is receiving a pulse from every connected client every few seconds, it knows something it did not know before, which is who is here right now. Not who has an open socket, since half-open ghosts have those, but who is actually present, actually reachable, and actually answered within the last few seconds. That is the green dot, the thing the application calls being online. Presence in a chat application is not a feature that someone designed on top of the connection; it is the failure detector read from the other direction. The same pulse that lets Mira’s server clean up dead sockets is the pulse that lets it show you which of your friends are around. One agreement provides two features, and if you have ever wondered why the online indicator in a chat app lags a few seconds behind reality, it is because that is how long the pulse takes to notice.

The heartbeat has a cost, and the cost is two choices with no right answer. The first is how often to send. Send every second and you will know within a couple of seconds that Mira is gone, but every connected client is now sending a message every second, her server is answering all of them, and on a phone the radio is waking up every second to do it, which drains a battery for no reason most of the time. Send every five minutes and the traffic is negligible, but the green dot stays lit for five minutes after someone’s laptop dies, and a dead socket hangs around that long before it is cleaned up.

The second choice is how many misses to tolerate. If one missed pulse means dead, you will declare people gone every time the path hiccups; a congested router delays one heartbeat and you have closed a perfectly good connection. If it takes ten missed pulses, detection takes ten intervals.

This is the timeout question again, of how long to wait before deciding that something has failed, and once again there is no answer, only a trade between being wrong quickly and being wrong slowly. What you pick depends on what you are protecting. A chat application might use fifteen or thirty seconds and tolerate three misses; a system that fails over a database when a machine dies might want two seconds and be willing to pay for it. Neither is correct. Both are chosen.

The second failure mode deserves particular attention, because it is subtle. A heartbeat cannot distinguish dead from slow. Under heavy congestion, pulses are delayed and dropped like any other piece, and a strict threshold will declare a living machine dead. When you build things in Part III that act on that declaration, handing a user’s connection to another server, promoting a replica, taking a machine out of rotation, a false death becomes an expensive mistake. The threshold is not only about the speed of detection. It is about how sure you need to be before you do something you cannot undo.

There are two places the pulse can live. The operating system can do it for you. Ask it to send keepalives on a connection and it will, with no involvement from your program, and it will close the socket and report an error if enough go unanswered. This is convenient and almost useless as configured, because the default interval on most systems is two hours. That figure was chosen decades ago, for a world where the concern was cleaning up truly abandoned connections rather than detecting a dead phone. You can tune it, but it is a per-machine setting, coarse, and invisible to the application.

Or the application can send its own pulse, as ordinary data on the connection, in the form of a tiny message that both ends recognise as not a real message. This is what nearly every chat application actually does. It costs a few lines of code and buys complete control. The interval can be whatever the application wants, it can vary with circumstance, slower on battery and faster during a call, and the reply can carry a little presence information, such as here and typing. When you see a WebSocket ping, an XMPP whitespace keepalive, or a “heartbeat” message in a protocol specification, this is what you are looking at. Both versions are the same agreement. The difference is who keeps it.

You can now tell absence from silence, at a cost you chose, with a delay you chose, and with a chance of being wrong that you chose. That is the honest shape of every liveness check you will ever build, and Courtyard has an online indicator, which fell out of the plumbing at no extra charge. Two chapters ago you saw that the sender keeps a buffer of pieces it has not heard back about. In the next chapter that buffer fills up, because Mira’s machine, though very much alive, cannot keep up with yours.