Insights · 27 July 2026 · 4 min read

What lives in the joins.

The hardest bugs in a live system rarely live inside your code. They live in the gap between what your system believes happened and what a third-party API actually did.

The booking went through. The payment gateway returned 200. The confirmation email was in the customer's inbox before they'd closed the browser tab. Two hours later, the charge had been reversed at the gateway, but the booking was still marked paid and the seat was still allocated. Nobody raised an alert, because every individual system had done exactly what it was supposed to do.

That gap — between what one system believes and what another system actually did — is where real engineering work happens. Not in the algorithm, not in the UI, not in the code-quality debates that fill most technical conversations. In the join.

The lie of the 200

HTTP success codes are optimistic by design. They mean: I received your request and I will attempt to process it. Sometimes — usually — they mean: I processed it. The difference is invisible until the moment it matters, which in a live booking system handling real money is a question of when, not if.

Payment gateways are particularly good at this ambiguity. Webhooks arrive late, or twice, or not at all. A charge can be authorised and then reversed. A refund can fail silently while returning a success response. The documentation describes the happy path in detail. It will not tell you what happens when their infrastructure has a problem mid-transaction, when your webhook endpoint was down for thirty seconds, or when a customer hits pay twice on a slow connection.

State that disagrees

The practical consequence is that any system touching money needs to maintain its own authoritative record of what it believes happened — and reconcile that against what external systems report actually happened, regularly, automatically, in a way that surfaces disagreements before a human discovers them through a complaint.

On an events platform we run, the booking record and the payment record are held with deliberately limited coupling between them. A failed webhook does not corrupt the booking. A duplicate event delivery does not trigger a double charge. The join between the two states is managed by a reconciliation process that runs on a schedule and flags anything that does not match. This is not over-engineering. It is the minimum defensible structure for a system where getting it wrong means real money going missing without anyone noticing.

Where the seam appears next

The same structural problem appears anywhere you are writing to one system and expecting to read truthfully from another. An inventory count that does not update quickly enough for a concurrent booking flow. A calendar integration that silently drops recurring events outside a certain date range. A CRM sync that duplicates contacts because your definition of a unique record and theirs diverge in an edge case that only emerges at volume.

These seams multiply as a system grows. Every new integration is another opportunity for two systems to hold contradictory beliefs about the truth. The joins do not get simpler over time. They accumulate.

What to check before you inherit

When Pivot takes on a system already in production, the joins are the first place to audit. Not the code style, not the test coverage — those tell you about the interior of the system. The seams tell you about the risk profile.

The questions that matter: what happens if this outbound call fails? Does the system retry, and if so, does it know the difference between a call that failed and a call that succeeded without acknowledgement? If a webhook fires twice, does anything break? If a third-party API starts returning a new field or stops returning an existing one, how does the system behave? If we cannot answer these without reading source code, that is the finding.

What to ask an engineer before you bring them in

None of this surfaces in a demo. A system can behave perfectly under observation and fail quietly under the right combination of load, timing, and external unreliability. The useful questions to ask an engineer before you hand them a project where real transactions flow — bookings, payments, subscriptions, event registrations — are not about their preferred stack or their opinions on architecture patterns. They are about what they build when things go wrong. How they handle the failure case. What they put in place before they consider something production-ready.

The answer will tell you more than the CV will. If that kind of work is relevant to something you are currently building or inheriting, a brief is a good place to start.

Written by Alex O'Neill— founder & lead product engineer, Pivot. About Alex →