Case study · Software team
Genuine payment messages were being rejected as fake. The check stayed on, and it now works.
Paid customers still showed as unpaid. The company's code was checking the wrong thing. We fixed it without switching the check off.
- Who
- A software company with a small engineering team. It sells subscriptions through a payment provider.
- The problem
- Customers who had paid still showed as unpaid, because every message from the payment provider was being turned away.
- What we did
- Found why the genuine messages failed the check, fixed it with the check left on, and added tests for forged messages.
- The result
- Genuine messages are accepted and altered or forged ones are rejected. The company's developers reviewed and released the change.
- Time
- Three working days.
What was going on.
When a customer pays, renews or cancels, the payment provider sends a message to the company's system so the account can be updated. The company had recently put a new security layer in front of its application. After that, accounts stopped updating: customers who had paid still looked unpaid.
The provider's dashboard showed every message failing with an 'unauthorised' error. One developer suggested switching the check off to get things moving. That would have let anyone on the internet send a fake 'payment received' message.
What we found.
Every message from the provider carries a signature: a tamper-proof stamp worked out from the exact content of the message. The company's code checked the stamp against a slightly different version of the content, one that had been tidied up on the way in. A stamp checked against tidied-up content never matches.
The check was working properly. It was looking at the wrong thing. So the fix was to make it look at the right thing, and to leave it switched on.
The technical detail
The signature is an HMAC of the raw request body. The web framework parsed and re-serialised the JSON before the handler calculated the HMAC, so spacing and key order differed from what the provider signed. We moved verification ahead of parsing, calculated it over the exact bytes received and compared the result in constant time.
What we did.
How we checked it.
These had to pass before we asked for sign-off.
- A test message signed with the test secret, using the provider's documented method, is accepted.
- The same message is rejected when one byte of the body changes, when the signature changes, when the wrong secret is used and when the signature is missing.
- The change contains no secret, and the check is not switched off in any setting, including in production.
- The company's existing tests still pass.
What they have now.
The company received the proposed change with its tests, a short note naming the cause and the provider's method the code now follows, redacted evidence of a genuine test message accepted and altered ones rejected, and steps to reverse the change. Its developers reviewed and released it.
Accounts updated from new messages straight away.
What we did not do.
- This was one provider and one endpoint.
- We did not replay the messages that failed while the problem lasted. The provider can resend them, and the company chose to do that itself.
- Making sure a repeated message only ever acts once is a separate job. We pointed it out and did not change it.
- Releasing the change to production stayed with the company's team.
The service behind this.
If genuine messages from a provider are being refused, the answer is almost never to switch the check off. It is to find what the check is looking at.
One-off job
Fix webhook signature checks
One named provider's test events pass signature verification at your endpoint, and forged or altered events are rejected. You review and release the change.
From $385. The agreed fixed price after the agreed checks pass and you sign off. No payment before sign-off.
More case studies.
One-off job
Checkout stopped taking orders after an update
Customers could fill a basket but not pay. We found the cause on a copy of the shop, fixed it and proved it with test orders.
Online store · Two days
Project
Hundreds of products flagged in Google's shopping feed
Several problems at once, and every earlier fix had only revealed the next one. We grouped them by cause and worked through them in order.
Online store · Three weeks, mostly waiting for Google to reprocess
Monthly service
Supplier payment runs checked before they go out
A second pair of eyes on every payment run, with the evidence for each flag, so a duplicate or a changed bank detail is found before the money leaves.
Finance and back office · Ongoing, monthly
Get a price for something like this
You are asking about: Fix webhook signature checks.
Sending this is not an order. We check we can help, then agree what is included, safe access and the price in writing before any work. Prefer email? hello@syntheticindustry.ai.