Idempotent REST API

A REST API is called idempotent when making multiple identical requests to an API has the same effect as making a single request.

In the realm of RESTful web services, idempotency relates to the concept that making the same API request multiple times should yield the same result as making it just once. This means that regardless of how many times you repeat an idempotent request, the outcome remains consistent.

1. What is an Idempotent API?

Idempotency essentially means that the effect of a successfully performed request on a server resource is independent of the number of times it is executed.

For example, in arithmetic, adding zero to a number is an idempotent operation.

When we design the REST APIs, we must realize that API consumers can make mistakes. Consumers can write the client code in such a way that there can be duplicate requests coming to the APIs.

These duplicate requests may be unintentional as well as intentional sometimes (e.g. due to timeout or network issues). We have to make our APIs fault-tolerant so that duplicate requests do not leave the system unstable.

Also, idempotency is a key enabler for efficient caching and optimization strategies. Caches (and CDNs) can store and serve the results of idempotent requests to reduce the load on servers and improve response times.

2. Idempotency with HTTP Methods

An idempotent HTTP method is a method that can be invoked many times without different outcomes. It should not matter if the method has been called only once, or ten times over. The result should always be the same.

If we follow the REST principles in designing our APIs, we will have automatically idempotent REST APIs for GET, PUT, DELETE, HEAD, OPTIONS, and TRACE methods. Only POST and PATCH APIs will not be idempotent.

  • POST and PATCH are NOT idempotent.
  • GET, PUT, DELETE, HEAD, OPTIONS and TRACE are idempotent.

Let’s analyze how the above HTTP methods end up being idempotent – and why POST and PATCH are not.

2.1. HTTP POST and PATCH

Generally – not necessarily – POST APIs are used to create a new resource on the server. So when we execute the same POST request N times, we will have N new resources on the server. So, POST is not idempotent.

POST requests can trigger various side effects beyond just resource creation. These side effects may include sending notifications, making changes to the server’s state, or performing actions that are not idempotent.

HTTP PATCH is used for making partial updates to an existing resource without replacing the entire resource. The result of a PATCH request depends on the initial state of the resource and the specific changes provided in the request. Repeating the same PATCH request may lead to different resource states if the resource has been modified in the meantime.

For instance, if you use a PATCH request to increment the number of items in a shopping cart, repeating the same PATCH request multiple times will increase the quantity with each request, which is a non-idempotent behavior.

2.2. HTTP GET, HEAD, OPTIONS and TRACE

GET, HEAD, OPTIONS and TRACE methods NEVER change the resource state on the server. They are purely for retrieving the resource representation or metadata at that point in time.

So invoking multiple requests will not have any write operation on the server, so GET, HEAD, OPTIONS, and TRACE are idempotent.

2.3. HTTP PUT

Generally – not necessarily – PUT APIs are used to update the resource state. If you execute a PUT API N times, the very first request will update the resource; the other N-1 requests will just overwrite the same resource state again and again – effectively not changing anything.

Hence, PUT is idempotent.

2.4 HTTP DELETE

2.4.1. Delete with the resource identifier

When you execute N similar DELETE requests, the first request will delete the resource and the response will be 200 (OK) or 204 (No Content).

Other N-1 requests will return 404 (Not Found).

Clearly, the response is different from the first request, but there is no change of state for any resource on the server-side because the original resource is already deleted.

So, DELETE is idempotent.

2.4.1. Delete without the resource identifier

Please keep in mind that some systems may have DELETE APIs like this:

DELETE /item/last

In the above case, calling operation N times will delete N resources – hence DELETE is not idempotent in this case. In this case, a good suggestion might be to change the above API to POST – because POST is not idempotent.

POST /item/last

Now, this is closer to HTTP spec – hence more REST compliant.

3. Handling Non-Idempotent Operations

As discussed above, not all HTTP methods are inherently idempotent. POST and PATCH, for instance, are non-idempotent methods. These methods can have side effects, and issuing the same request multiple times can result in different outcomes.

For example, when a user signs up for an account, a POST request is made to create a new user. If this operation is repeated with the same data, it can lead to multiple accounts being created for the same user.

To make non-idempotent operations safer, developers often implement strategies such as using unique request identifiers, transactional mechanisms, or idempotent request headers to ensure that repeating the request doesn’t lead to unintended consequences.

4. Summary

The idempotent REST APIs help in enabling predictable behavior, even in the face of network failures and retries. By understanding idempotency and adhering to its principles, we can create robust web services that offer a consistent experience to the users.

References:

Rfc 2616
SO Thread

Leave a Comment

  1. In your diagram in section 2, the table has two colums “idempotency” & “safe”, but don’t really explain what you mean by that term, how put / delete are in the first column but not the other. I assume it is to do with the response 1 & 1 + n cases?

    If so, it would be good to tie that term into each case explanation is “safe” or “not”

    The small change would improve clarity to an already great article.

    Reply
    • Safe means the operation doesn’t alter the state of the server at all. So all READ ONLY operations are considered SAFE. All SAFE HTTP methods are Idempotent, but not all Idempotent methods are SAFE. Example: deleting a record from the server with the DELETE HTTP method changes the state of the server (the record is now gone) so it is not SAFE but it is considered Idempotent.

      Reply
  2. I think your definition contradicts itself. First it says: “Idempotency essentially means that the result of a successfully performed request is independent of the number of times it is executed.”

    I want to point the word ‘result’ in there, as it sounds like later result is confused with ‘side-effect’.

    If I’m a client and call a DELETE endpoint, with a way to identify requests, I would expect the same result on both requests, not a 204 and later a 404. To do this I would need to pass a way to identify each request (a request id or idempotency UUID) so the server can know if a request has been processed before and what result it returned.

    It’s common for many developers to mix idempotency with resubmisstion / replay prevention. But both techniques are very different. A good idempotency solution makes the life of a client a lot simpler as they need to take less error cases into consideration.

    Reply
  3. PATCH is missing. It’s ike POST not idempotent since it’s usually used to partially update a resource (for example just sending and setting the first name of a user profile, ignoring the state of the remaining data).

    AFAIK PUT can also create a new entity (but always the same, so calling it twice wouldn’t create two entities – basically you’re providing also the ID of the element to be created and if it already exists it’s being overwritten, but it doesn’t have to exist when using PUT the first time). Of course this depends on the implementation. But it would be valid.

    Reply
  4. When we send a POST request, it will create a new resource. Now if we resend identical POST request N times, conflict error would be returned as an identical record already exists. So, in this case will POST be idempotent (like the DELETE case you mentioned) as the resource on server remains same?

    Reply
    • I think you are more like a business/implementation side of question while this post is merely providing concept/theory of REST idempotence in general.

      Reply
    • In a POST, the server determines the new ID, not the client, so that would be an improper implementation. PUT is appropriate for creation when the client is allowed to determine the new resource ID, so that scenario should use a PUT, which would be idempotent.

      Reply
      • It is a good practice to have a candidate keys that by definition are also unique. That would prevent creating duplicate resources. It is obviously not possible for all types of resources.

        Reply
    • Resending identical POST requests N times will create N resources, each with a new ID. In the case an ID is involved, PUT would update the document with the same ID N number of times and produce same result.

      Reply
    • Wrong. A put is NOT the same as a create command in a database. It is more like writing out a file out of a text editor. If it does not exist, it creates the resource (return 201), it it does exist it updates it (return 200).

      Reply
    • You can achieve idempotency in POST method for building a fault-tolerant API.

      The standard approach that you will notice is that of passing the unique Idempotency-key in every request made by the client. we can pass an Idempotency-key with a unique value which is in the UUIDV4 format in the headers. Now in case of network failure or no response from the server , client can send the same request again and again without worrying about a duplicate request.

      Let me explain the server side implementation:

      On the server side, we have created a middleware. This middleware will check each of the requests coming into the application, and check if the Idempotency-key is present in the headers.

      If the Idempotency-key is present in the request, we will query the database, and check if there’s a record corresponding to that Idempotency-key. If there is, the middleware will stop any further execution in the middleware layer itself, and will immediately respond with the saved data as a response.
      Otherwise, it will continue to the application layer and will create a new record in the database with the Idempotency-key and the generated response, and finally return the same response to the client.

      Therefore, any request with the same Idempotency-key already seen earlier by the server will not be treated as new and server will return the cached response.

      Reply

Leave a Comment