Never apply a payment twice: idempotent Stripe webhook handling for ACH and card payments
As lead engineer on a small team, I build the payment side of a platform for a pre-need insurance and burial trust provider. Customers use Stripe to make an ACH or credit card payment into their burial trust, and a separate Mercury integration confirms paper-check deposits.
Behind the Stripe side is a webhook that acts on Stripe's events: ACH micro-deposit verification, a payment going pending, succeeding, or failing, an NSF return, a dispute, and more. Each event updates the customer's contract based on its internally tracked status and the business rules around it. Because every one of those updates touches someone's trust funds, the rule that matters most is simple to say and easy to get wrong: a duplicate event must never be applied twice.
Webhooks are "at least once," not "exactly once"
Stripe delivers events by webhook, and it retries any delivery that doesn't get a successful response in time. That's good for reliability, but it means your endpoint will see the same event more than once: after a timeout, after a deploy, after a network blip. Events can also arrive in a different order than they happened. ACH makes this sharper than cards: a bank account may first need micro-deposit verification, a payment can sit in pending for days, and a payment that looked successful can still come back later as an NSF return. Code that assumes each event arrives once, in order, will eventually double-count a payment or reverse the wrong one.
The rules that keep it correct
- Verify, then record, then act. Check the webhook signature first. Then record the event's unique ID before doing any work, so a second delivery of the same event can be recognized and skipped.
- Make "already processed" a normal outcome, not an error. When a duplicate arrives, acknowledge it with a success response and do nothing else. Returning an error just invites more retries.
- Apply each event in one transaction. Recording the event as processed and changing the payment record should succeed or fail together, so a crash halfway through can't leave a payment applied but unrecorded, or recorded but unapplied.
- Treat events as signals about state, not commands. Because order isn't guaranteed, decide what to do from the contract's current, internally tracked status, and don't let an older event overwrite a newer one. A failure, NSF return, or dispute moves a payment forward in its lifecycle; it never "undoes" history.
- Respond fast, work safely. Acknowledge quickly so Stripe doesn't time out and retry, and keep the processing idempotent so a retry is harmless if it happens anyway.
Reconcile anyway
Idempotent handling keeps you from making mistakes; reconciliation catches the ones that slip through from somewhere else. I spent years before this on systems where money and data had to agree: building payment-processing solutions with Citibank and BNY Mellon settlement on Clearingworks, and replacing a legacy messaging layer at JPMorgan Chase with direct integration plus reconciliation. The lesson carries over: even with careful event handling, periodically compare your records against the processor's and surface anything that doesn't match, before a customer finds it.
The short version
Assume every event can arrive twice and out of order. Record it before you act on it, apply it atomically, decide from current state, and reconcile on a schedule. None of it is complicated, and all of it is the difference between payment code that works in a demo and payment code a business can trust.
Building something similar? Email me, connect on LinkedIn, or ask the AI assistant on brandoneilers.com.