Engineering
Webhooks that do not flake — three rules and a replay button
What we built into our webhook system from day one to make sure it does not become someone else’s problem.
Most webhook systems are an afterthought. The auth provider you use probably has one — does it sign payloads? Does it retry? Can you replay a delivery from a dashboard? In my experience the answer is "one of the three, and the docs do not specify which."
When we sat down to build SNDR’s webhook layer, we started with three rules. They are not novel — they are the bare minimum.
Three rules
- Every event is signed with HMAC-SHA256 over the timestamp and body
- Retries follow exponential backoff with full jitter, up to 12 attempts
- Every delivery is replayable from the admin dashboard
Sounds obvious. Almost no provider does all three.
Verifying the signature
We send X-Sndr-Signature: t=<timestamp>,v1=<hex_hmac> on every webhook. The receiver computes HMAC-SHA256(secret, "<timestamp>.<body>") and compares it with constant-time equality. If it does not match, drop the request — someone is replaying or forging.
The replay button
During an incident, the most valuable thing you can give an engineer is a button that re-fires the exact event with the exact payload. The admin dashboard shows every delivery, the response code we got, the headers we sent. One click and we replay it against the customer’s endpoint.
What we still want to add
A receiver-side test endpoint. Per-event-type filtering on the receiver. A "deliver as soon as the next change happens" mode for incident debugging. Coming soon.