The same webhook arrives twice and runs twice? Acknowledge fast and make a repeat do nothing

Why one event can reach you more than once

A webhook is a promise to tell you something happened, and the provider cannot know whether you received it unless you answer. GitHub's documentation says your server should return a 2XX status within 10 seconds; if it takes longer the connection is closed and the delivery is treated as failed. A failed delivery can then be delivered again, and GitHub notes that a redelivered request keeps the same X-GitHub-Delivery header as the original. Other providers have their own retry rules, so read yours: how many attempts, how long between them and how long they keep trying.

An endpoint that does its real work (sending an email, creating a record, charging a card) before it returns, and is slow, invites exactly this: the provider gives up, redelivers, and the work runs again while the first run is still going.

• Find the provider's documented retry and timeout rules.
• Look in its delivery log for the same event identifier appearing more than once.
• Compare your processing time with its timeout.

Acknowledge first, work afterwards

GitHub recommends acknowledging a delivery immediately and processing the payload in the background, so a slow job does not hold up later deliveries. In practice this means: verify the signature, record the event, return success, and let a separate step perform the action. If recording the event itself fails, return an error so the provider tries again. Do not return success for an event you have not safely recorded.

• Verify first, then store, then answer, then act.
• Keep the stored record small: the event identifier, the type and the raw body.
• A background step needs its own failure handling and its own record of what it has finished.

Make a repeat harmless without losing the event

Even with fast answers, assume a repeat can arrive. The standard defence is an idempotency key: a stable identifier for the event, stored with a uniqueness rule so the same event cannot be claimed twice. GitHub suggests its X-GitHub-Delivery header, which stays the same on redelivery, as a way to check that each delivery is unique. Its signature, though, is a hash of the request body, so that header is not covered by it: someone holding one genuine signed request could resend the same body with a different header value. Our recommendation is to key on an identifier inside the signed body where the provider gives one, or on a hash of the exact received body, so the key cannot be changed without breaking the signature. The cost is that two different events with byte-for-byte identical bodies would look like one, so check that your provider's bodies carry a unique id or timestamp.

Recording and acting must also be coordinated, or an event can be lost. If you insert the identifier first and act only if the insert succeeded, a crash between the two leaves a row but no effect: the provider's redelivery finds the row, is skipped, and the work never happens. The safer order is claim, then complete. Insert the identifier with the status received, perform the effect (in the same database transaction when the effect is a change to your own data), then mark it processed. A redelivery that finds processed is acknowledged and ignored. One that finds received for longer than the work should take is retried, because the first attempt probably died, taking over the claim with a single conditional update so that two retries cannot both do it; one that finds a recent received row is still in progress and should not start a second copy. Because the claim is a single insert with a uniqueness rule, two simultaneous copies cannot both claim it. For an effect outside your database, such as an email, a crash after the effect and before processed means a retry repeats it, so check whether that service accepts an idempotency key. Do not assume deliveries arrive in order; the cited documentation makes no promise about order, so an older event should not overwrite newer state.

• One identifier, one claim, enforced by the database rather than by a code check alone.
• Status received, then processed: a processed event is acknowledged and ignored; a stale received one is retried.
• Return success for a repeat of a processed event, so the provider stops retrying.
• Where order matters, compare the event's own timestamp or version with what you hold.

A safe first investigation

Start with the provider's delivery log, not your code. Look for repeated identifiers, the status each attempt received and how long your endpoint took. Then check your own records for two effects from one identifier. Use a test endpoint and the provider's sample or test delivery tool, and never replay real customer events to experiment. Replaying or repairing past events is a separate decision with its own risks.

• Repeats with long response times: the endpoint is too slow to acknowledge.
• Repeats with error statuses: the endpoint is failing, perhaps at the signature check.
• Two effects with different identifiers: not a repeat, a different fault.

What fixes it and how the paid job is accepted

The fix is to record each event under a stable key with a uniqueness rule, acknowledge quickly, and let a repeat of a finished event do nothing, inside your existing data model. It does not repair history or change provider accounts. That is a build job, not part of our webhook signature repair: our signed, repeat-safe webhook receiver (posted test price from £495, untested, paid only after sign-off) is accepted when the same delivery sent twice, including at the same moment, causes one action and a success response for the repeat. The signature repair (from £295, untested) is the narrower job for an endpoint that refuses genuine events. Send the provider, the event type and what ran twice in your first enquiry, not real event bodies or secrets.