Webhook Idempotency: The Producer Side No One Talks About

Most articles on webhook idempotency cover the receiver side: how should the receiver dedup so it doesn't process the same event twice? That's a real problem, but it's the second-order one. The first-order question is how the sender prevents duplicate sends in the first place.

This is about idempotency on the producer side, what your service emits to your webhook platform, and what your platform emits to the receiver.

The producer's problem

Your service is about to emit a webhook for "order completed." Something in the path from your transaction commit to the webhook queue is between-state: maybe your message broker had a retry, maybe a worker died after sending but before marking the event handled, maybe a queue consumer ran twice. Whatever the cause, you can end up enqueueing the same event twice.

If the webhook platform processes both copies, the receiver gets two webhooks for one business event. Even if the receiver is perfectly idempotent, the receiver is now responsible for figuring out that the duplicate is a duplicate, which they shouldn't have to.

The right place to dedup is at the producer side, in the webhook platform, before the receiver sees anything.

Idempotency keys

The mechanism is the idempotency key. On every ingest, the producer attaches a key that uniquely identifies the business event. The platform stores recent keys and discards repeat sends.

What makes a good key:

• Stable. The same business event must produce the same key every time, even across producer retries. "Random per-send UUID" is the wrong choice, that defeats the entire mechanism.
• Specific. The key should change when the business event changes. "order.completed" alone isn't specific; "order.completed:ord_38a91f" is.
• Bounded. You can't dedup against the entire history of time. Keys are usually unique within a window (24 hours is typical; 7 days is generous; longer than 30 days is rarely useful).

The pattern that works almost always: <eventType>:<resourceId>:<state>. For an order completion, that's order.completed:ord_38a91f:completed. Sending the same event twice from the same producer yields the same key. The platform discards the second send.

Where it goes in the request

In Pushrail's ingest API, the key is a top-level field:

{
  "eventType": "order.completed",
  "occurredAt": "2026-06-17T14:21:08.493Z",
  "customerExternalId": "acct_8K2zRq",
  "idempotencyKey": "order.completed:ord_38a91f:completed",
  "payload": { ... }
}

Two ingest calls with the same key, within the dedup window, produce one accepted event. The second call returns a "duplicate" status to the producer so the producer knows it was deduped rather than dropped.

The same key propagates to the receiver as a header (X-Pushrail-Idempotency-Key) so the receiver can do its own second-line dedup if it wants to.

Producer dedup vs receiver dedup

Both matter, for different reasons.

Producer dedup catches duplicates from the producer's infrastructure: queue retries, worker double-processing, network blips. It keeps the receiver from ever seeing the duplicate.

Receiver dedup catches duplicates that come through other paths: replays, manual re-sends, the platform's own retries that the producer doesn't know about. The receiver should still check the idempotency key on every event it processes, defense in depth.

The combination means a business event is processed exactly once by the receiver, regardless of what happens between.

What goes wrong without it

The failure modes when idempotency isn't enforced:

• Order webhook fires twice; charge processor runs twice; customer is double-charged.
• Subscription-renewed webhook fires twice; receiver creates two renewal records; analytics show 2x renewal rate.
• Deployment-completed webhook fires twice; CI marks the deploy as "completed" then "completed" again, triggering downstream automations twice.

These are all real outage post-mortems from teams that built webhook senders without producer-side idempotency. The fix is always the same and is always retroactive: add an idempotency key, propagate it through the platform, and have the receiver check it.

Building it vs using it

The platform-side implementation is two pieces: a key store (Redis or Postgres works, the data is small and short-lived) and a wrapper around the ingest path that checks the key before enqueueing. A senior engineer can build this in a day; making it actually correct under load takes a week of edge-case work.

Pushrail's ingest API enforces idempotency by default, see the canonical event payload to see where the key lives. The receiver-side header is documented per destination type.

Next in this series: dead-letter queues, what they look like when they fill up, and how to recover from a long-running misconfiguration.