What Are Idempotency Keys and How to Implement Them Safely for Payments

An idempotency key is a unique identifier attached to an API request to ensure that repeating the request produces only one effect, preventing issues like duplicate charges in payment transactions.

Understanding Idempotency and Idempotency Keys

Idempotency is the property of an operation that yields the same result even if executed multiple times.

In simple terms, an idempotent operation doesn’t create side effects or duplicate data when repeated.

For example, reading a record or deleting a specific item can be done over and over with the same outcome, but creating a new record (or processing a payment) without precautions could create duplicates.

According to HTTP standards, methods like GET, PUT, and DELETE are typically idempotent, but POST (often used for creating resources or charging payments) is not idempotent by default.

This is where idempotency keys come into play when designing APIs, especially for sensitive operations like payments.

An idempotency key is essentially a unique token (often a random string or UUID) that the client generates for a specific operation and sends along with the request (commonly via an Idempotency-Key HTTP header).

The server uses this key to recognize and track the state of that request.

If something goes wrong, say the client gets a network error or no response, the client can retry the request using the same key, and the server will know it’s a retry of the same operation, not a brand new request.

This mechanism allows the server to ensure the operation’s side effects occur only once.

In practice, once the server finishes processing a request (successfully or with a definitive failure), it stores the outcome (e.g. in a database or cache) associated with that idempotency key.

Any subsequent request with the same key will be intercepted: the server can “short-circuit” the processing and return the previous result instead of performing the action again.

This guarantees that even if a client calls a payment API multiple times (intentionally or due to retries), the customer will be charged only once.

Why “Idempotency” Matters

In distributed systems and real-world APIs, network failures, timeouts, or client-side retries are common.

Idempotency ensures that reprocessing a request doesn’t cause unintended side effects like duplicate charges or records.

In other words, it makes unsafe operations (like creating a payment) behave safely as if they were naturally idempotent.

The term "idempotency key" itself was popularized by payment APIs such as Stripe, underlining its importance in financial transactions.

Why Idempotency Keys Matter (Especially for Payments)

In payment systems, the consequences of duplicate processing can be severe. No one wants to charge a customer twice for the same transaction!

Idempotency keys are crucial for avoiding double charges and other side effects in scenarios where an API call might be repeated.

Consider these real-world examples of what can go wrong without idempotency:

• Duplicate Charges on Retry: A user submits a payment and the network times out. Their app isn’t sure if the payment went through, so it retries the payment API call. Without an idempotency key, the user could be charged multiple times for the same transaction.

• Repeated Orders: A customer clicks the “Place Order” button several times due to a slow response. Without idempotency, each click might create a new order, leading to duplicate orders, multiple shipments, and inventory errors.

• Double Account Creation: A user sign-up form is submitted, but the confirmation page hangs, prompting the user to submit again. Without idempotent handling, the system might create two identical user accounts for the same person.

As these scenarios show, idempotency is critical for reliability and consistency. It prevents data duplication, financial discrepancies, and user frustration.

In the context of payments, idempotency keys let us guarantee “exactly-once” execution of a charge operation: even if a client or network hiccup causes multiple attempts, only one actual charge will occur.

Stripe’s API, for instance, allows a unique Idempotency-Key on payment requests specifically so that clients can safely retry a charge call knowing that the customer will not be double-billed.

In summary, idempotency keys are a simple addition that makes APIs more robust by eliminating duplicate side effects in failure/retry scenarios – an essential safeguard for any payment system.

How to Implement Idempotency Keys Safely

Implementing idempotency keys involves changes on both the client and server sides to coordinate request tracking.

Below is a platform-agnostic, step-by-step approach to using idempotency keys safely for a payment (or any similar operation):

1. Generate a Unique Key for Each Operation: When the client is about to perform a payment request (or another non-idempotent action), it should generate a unique idempotency key for that specific operation. This is often a random UUID or similarly unique string. The key must be unique per distinct action. For example, each payment attempt gets a new key to avoid collisions. (Conversely, if the same operation is retried, you must reuse the same key; do not generate a new key on a retry, or the server will treat it as a different transaction.)

2. Attach the Key to the API Request: Include this idempotency key with the request sent to the server. Typically, APIs expect it in a header like Idempotency-Key: <unique-value> (though some designs might use a request parameter). This header doesn’t change the core request data; it’s a tag for the server to recognize retried requests.

3. Server-Side: Check for the Key: When the server (or payment service) receives the request, it looks at the idempotency key:

   • If the key has never been seen before, this is the first attempt. In this case, the server will lock or record this key (often by inserting a record in a database or cache) to mark that it's processing that operation. This can be as simple as creating an entry in an “idempotency keys” table with the key and request details, or using a unique constraint so that no two identical keys can be inserted. This step helps prevent two servers or threads from processing the same key at the same time.

   • If the key has been seen before, the server knows this request is a retry of an earlier operation. It should not perform the action again immediately. Instead, it will retrieve the result associated with that key (which was saved after the first attempt) and prepare to return that stored result.

4. Process the Request (First Attempt): For a first-seen idempotency key, proceed to execute the payment operation normally. Charge the credit card, create the order, or perform whatever actions are needed as if it’s a regular request. Important: Do this operation only once. If your backend consists of multiple steps (e.g., create database records, then call an external payment API, then send confirmation email), you may want to wrap this in a transaction or carefully orchestrated steps. Many implementations will record a “status” of the idempotency key as “in-progress” during processing, to handle concurrency safely.

5. Store the Result of the Operation: Once the operation completes (either successfully or with a final failure), save the outcome and associate it with the idempotency key. This typically means storing:

   • The final status code or result (e.g., “payment succeeded” with transaction ID, or “payment failed” with error code).

   • The response data that should be returned to the client (e.g., payment confirmation details, or error message).

   • Possibly a timestamp. (Idempotency records shouldn’t live forever; more on expiration shortly.)

6. From now on, this key is considered “used” or completed. The server might update the idempotency key record to mark it as finished and attach the response.

7. Return the Response to the Client: For the initial request, the server returns whatever result came of processing the operation. The client receives, say, a “Payment successful” message with a charge ID (or an error if something went wrong). At this point, the client can safely assume the operation was handled. If the client never has to retry, the idempotency key has done its job by being there just in case.

8. On a Retry Request (Duplicate Key): If the client retries the request (for example, after a timeout, not knowing the result) and sends the same idempotency key, the server will recognize the key from before:

   • The server fetches the stored outcome tied to that key.

   • It compares the new request’s parameters to the original request to ensure they are identical (to prevent accidental reuse of a key for a different operation). If the details differ (e.g. someone mistakenly reused the key for a different payment or amount), the server should reject the request with an error because it’s likely a bug on the client side. This preserves the guarantee that a key maps to one specific action.

   • Assuming the request is truly identical, the server will return the cached result from step 6 instead of reprocessing the payment. In other words, the client gets the same response as the first time, and no duplicate charge is made because the server did not execute the operation again. From the client’s perspective, the retry now cleanly resolves with the original success (or failure) result.

9. Handle Concurrent Requests (Edge Case): A well-designed implementation also considers if two identical requests with the same idempotency key arrive at the same time (perhaps due to a network glitch causing duplicate submissions). The service should ensure that only one of them actually runs the operation. For instance, when the first request with a new key is being processed, a second request with the same key could be blocked or told to wait/abort. Stripe’s system, for example, will not save a second response if another request with that key is still in progress. It might return an error indicating the request is currently processing, and the client can retry. The key point is to avoid two parallel executions of what should be a single operation. Using a lock or unique database transaction on the idempotency key is a common strategy to serialize these. This ensures exactly-once execution even under high concurrency.

10. Expire/Recycle Old Keys: Idempotency keys aren’t meant to live forever in your database. They are a temporary bookkeeping mechanism to handle retries within a reasonable timeframe. In practice, you can purge or expire stored idempotency records after some period (for example, 24 hours is a common policy). This horizon is chosen to be long enough that a client retry will still find the record, but not so long that your database grows unbounded. If a key is seen after it’s expired/purged, servers typically treat it as a completely new request (since they no longer have the record of the first attempt). Make sure this expiration policy is communicated or documented so clients know not to retry too late.

Best Practices and Tips for Safe Idempotent Payments

Implementing idempotency keys involves some careful considerations.

Here are some best practices to ensure safety and effectiveness:

Use Idempotency for Unsafe Operations

Reserve idempotency keys for actions that are not inherently idempotent (e.g. POST requests that create charges, orders, etc.).

There’s no need to use them on GET or DELETE requests, since those are already defined as idempotent (repeating them doesn’t change state).

Focus on payment APIs, order creation, account creation and similar scenarios where retries could otherwise duplicate something.

Generate Truly Unique Keys

The client should generate keys with very low chance of collision – using a UUID v4 or other high-entropy random string is recommended.

Keys don’t need to carry secret information; they just have to be unique.

Ensure the key is included in each retry attempt without change.

Avoid patterns that might reuse keys incorrectly (for example, don’t use a fixed string or an easily repeated value that might collide across different operations).

It can be helpful to incorporate something like a unique order ID or user ID in the key to tie it to a specific context (e.g. payment_<order123>_attempt1. Just keep it under any length limits (Stripe allows up to 255 characters, other systems may vary).

Idempotency Scope

Design your keys such that one key corresponds to exactly one logical operation.

For example, if a user is making a payment for order #123, you might use a key that includes the order number so you don’t accidentally reuse it for a different order.

If the user tries the same payment again (after a failure), reuse the same key.

But do not use the same key for two different payments or requests that would cause the second operation to be mistaken for a duplicate of the first.

Similarly, if you generate a random UUID for a key, do not generate a new UUID on each retry. Doing so defeats the purpose of idempotency (the server would see different keys and process each as a new payment).

The client application should be coded to retain and reuse the original key during retry loops.

Store and Check Request Data

On the server side, consider storing not just the result but also a hash or snapshot of the request parameters/body along with the idempotency key. This allows the server to verify that any subsequent request with the same key has the same intent as the original.

If there’s a mismatch (e.g. same key, but one request was for “charge $10” and another was “charge $15”), the safest action is to reject the request as a bad request.

This protects against scenarios where a client accidentally reuses an old key for a different operation, which could otherwise lead to confusing behavior or suppressed operations.

By validating that Key X always maps to Operation X, you maintain consistency.

Provide Proper Responses

When a duplicate request is detected via the idempotency key, the server should respond in a clear manner.

In most cases, this will be a normal success response with the same data that was returned originally (so the client gets the expected result without any new action occurring).

If an operation is still in progress (concurrent duplicate), you might return an HTTP 409 Conflict or a 202 Accepted with a “still processing” message, prompting the client to try again after a delay.

If a key misuse is detected (same key, different payload), respond with an HTTP 400 Bad Request or similar error indicating the idempotency key conflict. Clear signals help the client handle these cases correctly.

Clean Up and Logging

Implement a background job or mechanism to delete or archive idempotency records after their expiration time (24 hours or whatever window you choose). This keeps the storage manageable.

Also, log idempotency key events. It’s useful for debugging to know when a request was treated as a duplicate or when a key was rejected.

Monitoring duplicate request counts can even help detect if clients are misusing keys or if certain operations are flaky (causing frequent retries).

Test Thoroughly

Idempotent logic can be tricky to get perfect. Write tests for scenarios like “first request succeeds”, “first request times out and second succeeds”, “simultaneous requests with same key”, and “misused key with different payload”.

Ensuring your implementation covers all these will make your payment API much more robust.

Real-world testing (as Resend did with internal usage before full rollout) can validate that no duplicates occur even under error conditions.

By following these practices, you can confidently implement idempotency keys in a platform and language agnostic way.

The result is a payment system (or any critical API) that is resilient to network glitches and user retries, providing a better experience for users and preventing costly mistakes like double charges.

Idempotency keys might seem like a small addition, but they are a key ingredient for building reliable, fault-tolerant services especially in financial applications where consistency and trust are paramount.

Making the "check-then-act" atomic: the one race that still double-charges

The dangerous gap is between step 3 (check the key) and step 4 (process). If two retries race, both can see "key not found" and both charge. The fix is to make claiming the key and detecting a duplicate a single atomic operation, not a read followed by a write. In SQL that is one insert against a unique constraint:

INSERT INTO idempotency_keys (key, status, request_hash)
VALUES ($1, 'pending', $2)
ON CONFLICT (key) DO NOTHING;      -- returns 0 rows if the key already existed

If the insert wins (1 row), this caller owns the operation: it charges, then flips status to done and stores the response. If it loses (0 rows), the key already exists, so the caller reads the stored row and either returns the saved response (done) or tells the client to retry shortly (pending). A plain "SELECT then INSERT" has a window where both callers see nothing and both insert — the unique index closes that window for you, which is why the claim must be one statement, not two.

The third state nobody handles: pending-and-dead

There is a failure the state machine above quietly ignores: the owner inserts pending, calls the gateway, and dies before flipping the row to done. Now the row is pending with no living owner. Every retry hits the 0-rows branch and is told "retry shortly" — forever. Worse, if you rely on the 24 h TTL purge to eventually clear it, the next retry after the purge is treated as brand-new and can double-charge a payment that did reach the gateway the first time.

The fix is to give pending rows a lease. Store a locked_at timestamp with the claim, and let a retry that finds a pending row older than the lease window (a safe rule of thumb: at least 2× the gateway timeout) take ownership atomically:

UPDATE idempotency_keys
SET locked_at = now()
WHERE key = $1
  AND status = 'pending'
  AND locked_at < now() - interval '60 seconds';   -- 1 row = you now own recovery

The winner must not blindly re-drive the charge: it first queries the gateway — using the gateway's own idempotency key from the original attempt — to learn whether the charge actually happened (the recovery point), then either completes the record or compensates. This is why lease-based recovery has a precondition: the downstream must be queryable, or itself idempotent, so re-driving is safe.

There is a cleaner alternative when it applies: if all the operation's effects are rows in the same database, run the insert-claim and the business write in one transaction. A crash then rolls back the claim atomically with the work — no orphaned pending state can exist. The lease machinery is the price you pay precisely when the effect lives outside your transaction (an external gateway), which is the payments case.

Why the stored response body is not optional

Returning a duplicate request's result by re-running the business logic is a trap: the second run can compute a different answer (a new timestamp, a different fraud score, a now-expired card). Persist the actual HTTP status and body alongside the key on first completion, and replay that. The guarantee you are selling — "the retry gets exactly what the first call got" — only holds if you stored what the first call returned.

TTL is a correctness parameter, not a cleanup detail

The retention window must be longer than the client's maximum retry horizon. If you purge keys after 5 minutes but a mobile client retries a timed-out payment after 10 minutes, the second attempt finds no record, is treated as brand-new, and double-charges. Pick a TTL that dominates every retry path (24 h is a common floor) and index the timestamp so a background job can purge in ranges. This is the single most common idempotency bug in production, and it hides because it only fires on the slow-retry tail.

Sizing the key store

The table grows with payment write volume, so size it. At a peak of 10,000 charges/s sustained: 10,000 × 86,400 s ≈ 864 million keys/day; at ~300 bytes per row that is ≈ 260 GB/day of raw rows before purge (864e6 × 300 B ≈ 2.6e11 B). That is enough that an unsharded, never-purged key table becomes the checkout bottleneck — partition by time so the TTL purge can drop whole partitions cheaply instead of deleting row-by-row.

The distributed-systems catch: "exactly-once" is narrower than it sounds

A key gives you exactly-once for the effect guarded by the same transaction. Side effects outside that transaction — a call to an external gateway, a confirmation email, a downstream event — are not automatically deduplicated by your key; each needs its own idempotency (the gateway's own key, an outbox with dedup). And across regions, a key uniquely claimed in one region is not visible in another during a partition: two regions can each insert pending and both charge (split-brain). Keep the key in the payment writer's home region, or make it globally unique through consensus — do not assume a locally-unique insert is globally safe.

When a dedicated idempotency key is the wrong tool

Do not reach for a key table when a cheaper natural guard already exists. Read-only GET/list endpoints need none. Internal batch jobs usually already carry a natural batch_id you can put a unique constraint on — that is your idempotency, with no separate key infrastructure. Reserve the client-supplied key for externally-triggered, non-idempotent writes (charges, order creation) where only the caller knows "this retry is the same intent as that first attempt." A good production alarm here is a spike in same-key/different-payload rejections: that means either a client bug reusing keys or an attacker probing.