Idempotency: The Art of Doing It Once

1. The Scary Reality of Networks

In a distributed system, network calls are unreliable. When you send a request (e.g., “Charge User $100”), 3 things can happen:

1. Success: The server did the work and told you.
2. Failure: The server failed to do the work.
3. Unknown (The Ghost): The server did the work, but the response was lost (Network Timeout).

In Case 3, if the client retries blindly, the server executes the work twice.

• Result: User charged $200 instead of $100. This is unacceptable.

2. What is Idempotency?

Mathematically: f(x) = f(f(x)). Applying an operation multiple times has the same effect as applying it once.

• Idempotent: x = 5 (Assignment).
• Not Idempotent: x = x + 1 (Increment).
• Not Idempotent: INSERT INTO users... (Creates duplicates).
• Idempotent: DELETE FROM users WHERE id=1 (Deleting a deleted user does nothing).

Messaging Guarantee: “At-Least-Once”

Most message queues (Kafka, SQS) guarantee At-Least-Once delivery. They promise the message will arrive 1 or more times. They never promise exactly once (unless you use heavy transactions). Therefore, your consumers MUST be idempotent.

3. Solution: The Idempotency Key

To make a non-idempotent operation (like “Charge Card”) idempotent, we use a unique ID.

1. Client generates a unique idempotency_key (UUID v4) for the button press.
2. Server checks a database/cache: “Have I seen this Key?”
   • Yes: Return the previous result. Do nothing.
   • No: Execute operation, save Key + Result, return Result.

---

4. Interactive Demo: The Double Charge Simulator

Simulate a “Network Flaky” environment where the response is often lost.

• Scenario: You are sending a payment. The network is terrible (50% packet loss on response).
• Goal: Ensure the balance only drops ONCE ($100), no matter how many times you click due to timeouts.
• Action: Try sending payments with and without Idempotency enabled.

Wallet Balance: $1000

Request ID: uuid-101

Enable Idempotency Key Simulate Network Timeout

Tip: If "Simulate Network Timeout" is ON, click PAY multiple times like a frustrated user!

> Server Ready... Waiting for requests.

5. Implementation Patterns

A. The “Unique Constraint” (Database)

The simplest way. Rely on the database’s ACID properties.

INSERT INTO payments (idempotency_key, amount, user_id)
VALUES ('uuid-101', 100, 50);
-- If run twice, DB throws "Duplicate Key Violation"

The app catches this error and returns “Success” (since the work is already done).

B. Redis + Lua (The Race Condition)

A common mistake is the “Check-Then-Act” pattern, which causes a Race Condition:

# BAD CODE - Race Condition!
if not redis.exists(key):
    # <-- Another thread could insert here!
    redis.set(key, "processing")
    process_payment()

Solution: Use Redis SETNX (Set if Not Exists) or a Lua Script to make the Check+Set operation Atomic.

-- ATOMIC LUA SCRIPT
if redis.call("EXISTS", KEYS[1]) == 1 then
    return 0 -- Already exists
else
    redis.call("SET", KEYS[1], "processing")
    return 1 -- Success, lock acquired
end

6. Soft Delete vs Hard Delete

Idempotency often requires “Soft Deletes” to handle re-execution of “Delete” logic safely, although DELETE is naturally idempotent.

• Hard Delete: DELETE FROM users WHERE id=1.
  • First call: Returns “OK”.
  • Second call: Returns “0 rows affected” (or 404). This might confuse the client.
• Soft Delete: UPDATE users SET deleted_at=NOW() WHERE id=1.
  • First call: Sets deleted_at. Returns “OK”.
  • Second call: Updates deleted_at again (or ignores). Returns “OK”.
  • The client always gets a consistent “Success” response, which is better for retry logic.

7. Idempotency in REST APIs

Not all HTTP methods are equal.

Method	Idempotent?	Description
GET	Yes	Reading data doesn’t change state. Safe to retry.
PUT	Yes	“Replace this resource”. Calling PUT /users/1 {name: "Bob"} 10 times results in the same state (Name is Bob).
DELETE	Yes	“Delete this resource”. Calling it 10 times results in the same state (Resource is gone).
POST	NO	“Create a resource”. Calling POST /payments 10 times creates 10 payments.

How to make POST idempotent?

Use a custom header like Idempotency-Key (Stripe does this).

1. Client generates UUID.
2. Client sends POST /payments with header Idempotency-Key: uuid-123.
3. Server checks if uuid-123 exists in DB.

8. Summary

• Network failures (Timeouts) create ambiguity. You never know if the server did the work.
• At-Least-Once delivery means you WILL get duplicates.
• Use Idempotency Keys (UUIDs) to distinguish retries from new requests.
• Use Atomic Operations (DB Constraints) to prevent Race Conditions.
• Consider Soft Deletes for cleaner retry handling.