Idempotency — Algorithmics

A payment request times out. Did it go through?

The client does not know. The request may never have arrived, or it may have succeeded and the response was lost. Those two cases are indistinguishable from outside, and they need opposite actions.

Second attempt

client POST /charge $40

server charged $40 ✓

network response lost

client timeout — retry

server charged $40 again ✗

customer charged

$80

Both requests were valid, both arrived, both were processed. Nothing failed and the customer paid twice. The client had no way to tell the first attempt had worked.

The definition

An operation is idempotent if doing it twice has the same effect as doing it once.

account.balance = 100;      // idempotent — same result every time
account.balance += 100;     // not — twice is 200

Note what it does not require: the responses need not be identical, and the operation need not be read-only. Only the observable state has to converge.

HTTP already tells you which is which

Method	Idempotent	Why
`GET`, `HEAD`	yes	reads nothing changes
`PUT`	yes	sets a value to an absolute state
`DELETE`	yes	already-deleted stays deleted
`POST`	no	creates something new each time
`PATCH`	depends	`{"status": "paid"}` yes; `{"increment": 1}` no

This is why a proxy may retry a GET on your behalf and will not retry a POST. It is also a design lever: expressing an update as “set to X” rather than “add X” makes it idempotent for free.

The idempotency key

When the operation genuinely creates something, the client supplies a unique key and the server remembers it.

async function charge(request: ChargeRequest, key: string) {
  const existing = await db.idempotency.find(key);
  if (existing) return existing.response;          // replay the original answer

  const result = await processPayment(request);
  await db.idempotency.insert({key, response: result, expiresAt: in24Hours()});
  return result;
}

This is what Stripe’s Idempotency-Key header does, and it is the standard pattern.

The check and the insert must be one transaction

// Two concurrent retries both find nothing, both charge.
const existing = await db.idempotency.find(key);
if (existing) return existing.response;
const result = await processPayment(request);

// One transaction, and let the unique constraint arbitrate.
await db.transaction(async (tx) => {
  await tx.idempotency.insert({key});   // throws on duplicate — that IS the lock
  result = await processPayment(request);
  await tx.idempotency.update(key, {response: result});
});

Read-then-write across two statements is a race, and the window is exactly when retries are most likely — during the timeout that caused them.

Also decide what to do while the first attempt is still running: returning 409 and asking the client to poll is usually better than making it wait.

Making things idempotent

Natural keys. If a record’s identity is derivable from the request — order_id — a unique constraint does the whole job. INSERT … ON CONFLICT DO NOTHING is idempotency in one statement.

Absolute, not relative. SET status = 'shipped' over INCREMENT attempts.

Conditional updates. UPDATE … WHERE status = 'pending' runs once, because the second attempt matches nothing.

Sequence numbers. Reject anything older than the last applied version. This also gives you ordering, and it is how most replication protocols work.

Where it earns its keep

Message consumers. At-least-once delivery is the only guarantee you actually get, so idempotent consumers are what turn it into exactly-once effects.

Webhooks. Providers retry aggressively, and a duplicate payment.succeeded must not ship the order twice.

Deployment and provisioning. Terraform, Kubernetes and Ansible are built entirely on this: declare the desired state, apply repeatedly, converge. Being able to re-run a half-finished deploy is the whole value proposition.