Safe to Ask Again?

Courtyard holds a good deal by now. There are messages, in dozens of conversations. There are photos, thousands of them, and profiles, and likes on photos, and before long there will be comments on the likes. Before you can ask Mira’s server for any of it, you need a way of naming it, and the naming turns out to be the small problem. The large one is what your client should do when it asks for something and no answer comes back.

The obvious first scheme for the names is a flat number. Photo 42. Message 90,113. Profile 7. It works, in the sense that the server can find things, and it fails in the way every flat namespace in this book has failed. Nobody can say anything about photo 42 without looking it up. Whose is it? Is it in an album? Is it the one from Tuesday? The number carries no structure, so every question about it is a lookup. And when Mira’s server grows to the point where photos live on one disk and profiles on another, which is a matter for Part III but is certainly coming, a flat identifier gives the server no clue where to look. This is the problem of Chapter 5 over again. A name with no location in it means that everything has to be remembered individually.

So you reach for hierarchy, for the second time in this book.

/users/mira
/users/mira/photos
/users/mira/photos/42
/users/mira/photos/42/likes

Each segment of the path narrows the one before it. The name tells you that the photo belongs to Mira, that it is a photo, that it is the forty-second, and that this particular request concerns its likes. The server can route on the prefix, so that everything under /users/mira/ lives wherever Mira’s data lives, in the same way that a router routes on a network prefix. And a human can read it, which for the feed matters more than it did for addresses. Chapter 18 gave you the shape of a request; this gives you what goes in its path, and it is enough naming for now. The real problem is waiting.

You type a message to Mira and tap send. Your client builds a request, which says in effect post this message to /conversations/mira/messages, and writes it to the socket. Then it waits for the response, the status that says the message has been stored. Nothing comes back. A second passes, then two. Your client has a timeout, because Chapter 11 taught you that waiting forever is not an option, and the timeout expires. Now your client has to decide what to do. It has two choices, and both of them are wrong.

The first choice is to send the request again. Perhaps the request was lost on the way and never reached Mira’s server. In that case resending is right, and the message gets through. But perhaps the request arrived, the server stored it, and it was the response that was lost on the way back. In that case resending means Mira’s server stores the message a second time, and Mira sees your message twice. You have seen this happen. Everyone has: the doubled message in a chat, the duplicate email, the order confirmation that arrives twice. All of them come from this moment.

The second choice is not to send it again. Then, if the request really was lost, your message is gone, and you do not know it, and Mira never sees it. Your client shows the message as sent, because you tapped send, and nothing follows.

The timeout told you one thing, that no reply came in time. It did not tell you whether the request was processed, and it cannot. From your side of the wire, a lost request and a lost response look identical. It is the lost acknowledgement of Chapter 11 once more, for the same reason as before: the only evidence that something happened is a message, and messages get lost. The like button makes matters worse. You tap the heart on Mira’s photo; the request says add one to the like count; the response is lost; your client retries; and the count goes up by two, though you liked the photo once.

You have solved a problem like this once already. In Chapter 11 you numbered the pieces, and the receiver discarded any repeat. The transport does that still. So it is fair to ask why Mira’s server is storing the message twice, and the answer is that the transport’s numbers belong to the transport. When your client retried, it did not resend the same piece. It sent a new request, made of new bytes, possibly over a new connection with new sequence numbers. To the transport this is a fresh stream of data, and it delivered that stream perfectly, once. The transport kept its promise, which was that every byte your client wrote would arrive once. Your client wrote the message twice.

There is a general truth in this, and it is worth stating plainly. Every layer that retries needs its own detection of duplicates. The transport’s protects the transport’s retries. Your client’s retries happen above the transport, where the transport cannot see them, and they need protection of their own. Protection against repeats does not propagate upward. It has to be rebuilt on every floor of the tower where something might be sent twice.

Rather than agonise over each retry, look at what the requests do. Take fetch photo 42 and do it twice: you get the photo twice, and nothing on the server has changed. It is entirely safe to repeat; the second time is merely wasteful. Take set my display name to “Sam” and do it twice: your display name is “Sam”, and then it is “Sam”. The end state is the same, and the request is safe to repeat. Take delete photo 42 and do it twice: the first deletes it, the second finds nothing to delete and says so. The end state, that photo 42 does not exist, is the same either way, and so this is safe to repeat in the sense that matters. Take post this message and do it twice: two messages. Not safe. Take add one to the like count and do it twice: plus two. Not safe.

The dividing line is not between reads and writes. Setting your display name is a write, and it is safe. Deleting is a write, and it is safe. What matters is a particular property, which is whether doing the thing twice leaves the world the same as doing it once. Requests with that property can be retried freely; send them ten times if you like, and the world ends up where it would have with one. Requests without it cannot be retried without care. The property is called idempotence, and a request that has it is idempotent. You have now met the idea three times over: in Chapter 11 as duplicate pieces, here as duplicate requests, and it will return in Part III as duplicate jobs. It is always the same question. If this happens again, is anything different?

A property becomes an agreement when it is written down where others can read it. Your client can reason about whether post this message is safe to retry, because your client knows what the request means. But your client is not the only thing that might retry. A helper library that manages connections might. In Part III, a box in front of Mira’s server that spreads the load might, and that box has never heard of Courtyard and has no idea what your requests mean. If anything in the path is ever going to retry safely, the request has to say whether it is safe. So you put the declaration in the request, up front, where anything can read it without understanding the body. Chapter 18 already left a slot for it, and that slot is the verb.

The verb GET means give me this. It has no effect on the server, so it is safe to repeat, and because nothing changes, the answer can also be kept. If you asked for photo 42 a minute ago and got it, a second ask can be answered from the copy. That second property matters a great deal, and it is the whole of Part III’s chapters on caching, so it is worth remembering from here. The verb PUT means make this be exactly this: set the display name, replace the photo’s caption. It is idempotent, since the same request gives the same end state, and so it is safe to repeat. The verb DELETE means make this not exist, and it is idempotent for the reason given above. The verb POST means do this: post a message, add a like, charge a card. It is the catch-all for things that are not idempotent, and you repeat it at your peril.

This is the reason the verbs exist. People have built philosophies of service design on top of them, but underneath the philosophy they are a declaration of retry safety and cacheability that anything between client and server can act on without understanding the request. A retrying library sees GET and retries freely. It sees POST and stops to ask. A cache sees GET and keeps the answer; it sees POST and never does. Every intermediary you will meet in Part III relies on the verb meaning what it says, which is why using the wrong one, a GET that deletes something or a POST that merely fetches, is not a matter of style. It is a lie told to the infrastructure.

But you do want your message to get through. You cannot decline to retry a POST and let Mira never see it. So you make the POST idempotent, not by changing what it does but by giving the server a way to recognise a repeat. Your client generates a unique key before the first attempt, a random identifier or a counter, something no other request will ever carry, and puts it in the request. Mira’s server, before storing the message, checks whether it has seen this key. If it has not, it stores the message, stores the key alongside the result, and responds. If the key is already there, it stores nothing. It looks up the result from the first time and sends that same response again.

Now your client can retry as often as it likes. The first request to arrive is processed. Every later one carrying the same key is answered from memory. Mira sees one message, the like count goes up by one, and your client never had to know whether it was the request or the response that was lost, because it no longer matters.

Consider what you have built. It is a unique number per request, remembered by the receiver, used to discard repeats. It is the sequence number of Chapter 11, rebuilt one floor from the top, for a different kind of repeat. The field calls it an idempotency key, and it comes with the costs you would expect. The server has to store the keys, and cannot store them forever, so each key has a lifetime, and there is the question of how long that lifetime should be, and there is no right answer. A retry that arrives after its key has expired will be processed twice. Timeouts and staleness have turned up again, because they are always there.

The status number from Chapter 18 has a part to play here as well. It does not merely say that the request worked or did not; it tells the client what kind of failure it was, and therefore what to do next. A status in the 200s means the request worked and you are done. A status in the 400s means your request was wrong, malformed or unauthorised or asking for something that does not exist, and sending it again unchanged will get the same answer, so you should not. A status in the 500s means the server failed, because it crashed or something behind it did, and the request might well succeed if tried again in a moment. One particular code, 429, means that you are asking too often and should slow down, which is the backoff of Chapter 16 arriving at the application layer.

So the status classifies the failure as the verb classifies the request. A client that retries 500s but not 400s, backs off on 429, and only ever retries a non-idempotent request with a key attached, is a client that behaves well under every kind of failure. You will build that client in Part III, and most of what it does you have now derived.

One more matter remains, and it is a decision rather than a derivation. Everything in a request, who you are, what you want, the idempotency key, is in the request. Mira’s server, between one request and the next, remembers nothing about you. It holds no notion that you are logged in or that you were looking at page three. Each request arrives complete, is answered, and is forgotten. If you want the server to know who you are, you tell it, every time. That costs something. Every request carries a little more than it strictly needs to, repeating an identity that a remembering server could have kept. It is fair to ask why you would pay that.

You pay it because a server that remembers nothing can be replaced. If Mira’s server crashes between your requests, nothing is lost, since there was nothing on it to lose. If there are ten servers instead of one, any of them can answer any request, because none of them needs to have seen you before. If a request fails and you retry against a different machine, it works, because the retry carries everything the first attempt did. Forgetting is what makes the server interchangeable. The field calls such a server stateless. It is a choice, with a bill that comes due in Part III, where it turns out to be the single most important property a server can have, and where you will watch a team that broke it without noticing spend a chapter recovering. The chat connection, it should be said, is the opposite case. A long-lived socket is state, and Part III will have to deal with that separately.

Courtyard now names what it holds with paths, declares what each request does with a verb, and survives lost replies without duplicating anything, because the dangerous requests carry a key and the server remembers the key so the client need not. You have also seen the idempotency question in its second form. The first was pieces, and this was requests. There will be a third, and by then you will know it on sight.