What does an idempotency key do?
An idempotency key identifies one intended action across repeated API attempts. The client keeps the same key when it retries that action. A supporting server uses the key to recognize the retry and avoid repeating the action. The protection depends on the endpoint’s contract, its retention window, and the handling of concurrent requests and downstream systems. Adding a header alone does not provide it.
For an engineering lead connecting an internal tool to another system, the important question is what happens after the first request stops giving a clear answer. A timeout can mean that nothing happened, that work is still running, or that work succeeded but the response was lost. Creating a new key after every timeout can turn one intended action into several.
This guide proposes a retry contract and failure-test matrix for a hypothetical fulfillment integration. It is a design example, not a tested Hapy implementation. The broader custom web app integration guide covers API access, ownership, data mapping, and sync choices. This page focuses on the identity and recovery of one action.
Give the action a stable identity
Create the key before the first attempt and retain it with the operation record. A retry must recover the same key after a process restart. Generate a new key when the user intends a genuinely new action, not merely because a network attempt failed.
AWS explains why a caller-provided request identifier expresses intent better than matching request parameters alone. Two requests with identical inputs can still represent two legitimate actions. An input hash can help detect misuse of a key, but it cannot always decide whether two actions were intended.
In the fulfillment example, a user requests one shipment for an approved order. The application records the shipment request and its key before sending it. A retry belongs to that shipment request. A later, separately approved replacement shipment needs a different operation identity, even when it refers to the same order.
Keep these identifiers separate:
- Business ID: identifies the order or other business record. One order can require several different actions.
- Idempotency key: identifies this intended action across its attempts.
- Request or trace ID: helps find one attempt in logs. Several attempts can share one idempotency key.
Scope key lookup to the authenticated customer or tenant and the operation. Do not let one tenant retrieve another tenant’s stored response by supplying its key. A key is not a password or permission to perform an action; apply authentication and authorization on retries too. Avoid putting personal data in keys or unrestricted logs.
Define what the server does with each state
The server needs more than a lookup that distinguishes an absent key from a saved response. Agree on the behavior for an active attempt, a completed attempt, changed inputs, and an uncertain outcome.
MDN describes client and server responsibilities for the Idempotency-Key header, including key reuse, request fingerprints, and documented expiry. MDN currently marks the header experimental and non-standard. Check the actual provider’s supported endpoints and contract rather than assuming universal behavior.
Use the following as a proposed contract for the fulfillment example. Adapt it to the API and storage system; these are not universal HTTP response rules.
| State or condition | Proposed behavior | Evidence the operator needs |
|---|---|---|
| Key is absent and input is valid | Atomically reserve the scoped key before starting the action | Operation ID, accepted input fingerprint, and start time |
| Same key and same input, still running | Return a documented pending or conflict result; do not start another action | Current state and a supported way to check or retry |
| Same key and same input, completed | Return the recorded outcome or a documented equivalent response | Stable shipment reference and completion state |
| Same key, different meaningful input | Reject reuse; do not overwrite the first operation | A clear mismatch error without exposing another request’s data |
| Downstream outcome is unknown | Keep the operation unresolved and reconcile it | Provider reference, attempt history, and recovery owner |
| Key is outside the retention window | Apply the documented expiry rule; require reconciliation before a risky replay | Expiry time and the business record used to check the old action |
Claim the key before concurrent work can start
Two workers can both read “key not found” before either saves a result. A later cache write does not prevent both workers from starting the shipment.
Use an atomic reservation, such as a unique scoped operation record, so only one worker obtains permission to start. PostgreSQL’s INSERT documentation explains how unique constraints and conflict handling arbitrate competing inserts. The application must check whether it actually acquired the operation before starting work. Where the business change and idempotency record share a transactional database, design their commit boundary together. A separate cache check followed by an unrelated database write leaves a failure gap.
The implementation also needs a recovery rule for a worker that stops after reservation. Do not treat an expired worker lease as proof that no shipment exists. First determine which effects can already have occurred.
Separate replayable outcomes from unknown outcomes
A saved completed result can answer a retry without starting new work. An empty result after a timeout cannot.
Document which failures are recorded as final outcomes and which permit another execution attempt. Validation rejection before work starts differs from a failure after an external action may have succeeded. Choose response codes and client behavior together. Do not infer that every server error is safe to repeat with a new key.
Carry the protection across the downstream boundary
An idempotency record in your application does not automatically prevent duplicates in another service. The fulfillment provider may accept a shipment and then lose the response before your worker saves its result. Your local record now says “unknown,” while the provider may already have a shipment.
Before enabling automatic retries, check whether the provider accepts a stable operation key or supports lookup by a unique external reference. If it supports a key, keep the same downstream key for retries of that action. If it supports lookup, reconcile the existing shipment before issuing another create request.
When neither mechanism gives a reliable answer, route the uncertain operation to a person or a defined reconciliation process. A local lock cannot prove what a remote system did. Do not advertise end-to-end exactly-once execution on that basis.
For asynchronous work, the transactional outbox pattern can keep a committed business change connected to work that still needs delivery. That record alone does not remove duplicates at the receiver. Define duplicate handling for each effect separately: shipment creation, stock movement, and notification delivery need their own recoverable identities.
Choose expiry from the recovery window
Keep idempotency records for a period that covers the retries your system permits, including delayed jobs and operator replays. There is no universal retention period that makes every integration safe.
Once the provider forgets a key, sending it again may create new work. A long-lived business reference can help an operator reconcile an old action, but only if the receiving system enforces or reliably exposes that reference. Do not assume an order number is unique per shipment.
Specify the longest automatic retry age, the provider’s retention rule, the handling of late messages, and the approval needed for manual replay. Set a client cutoff that respects the provider’s guarantee. Keep a useful audit reference without retaining sensitive response bodies longer than needed.
Test the contract before enabling automatic retries
Test the business effect as well as the HTTP response. Two successful responses are acceptable when both identify the same intended shipment. One successful response is not enough if the provider created two shipments.
This proposed acceptance matrix makes the integration’s failure boundaries explicit:
| Injected condition | Check at the application | Check at the downstream system |
|---|---|---|
| Response disappears after a successful create | Retry retains the original operation key and resolves to the original result | No additional shipment is created |
| Two workers send the same operation together | Only one worker wins the atomic reservation | The same operation does not produce two shipments |
| Same key arrives with a changed destination | The mismatch is rejected and the original record stays unchanged | No shipment uses the altered destination |
| Worker stops after the provider accepts the action | State remains unknown until lookup or controlled reconciliation resolves it | Existing shipment is found before another create |
| Job resumes after the key’s retention window | Automatic replay stops under the documented cutoff | Operator checks the prior action rather than assuming the key still protects it |
| Another tenant supplies the same key | Lookup and authorization remain tenant-scoped | No cross-tenant result or action is exposed |
Use test accounts and controlled failure injection. Record the expected outcome, actual operation references, and the person responsible for unresolved cases. The matrix is a proposed test plan, not evidence that a particular API has passed it.
Before approving retries, ask the implementation team to demonstrate the lost-response case and the uncertain-downstream case. Those tests show whether the contract extends beyond the application’s happy path.
If you need help defining that contract, Hapy offers business systems automation. Hapy has a commercial interest in that service. Bring one real operation, its provider documentation, and its current failure path to the review.
