When the Systems Don't Talk
- Belinda Anderton
- 2 days ago
- 7 min read
Updated: 1 day ago
Hayek's 1945 paper in the American Economic Review was not, technically, about ecommerce (Hayek, 1945). But his central argument that the economic problem is fundamentally one of dispersed knowledge rather than scarce resources, has aged better than most things written in 1945, and applies to a remarkable number of situations that Hayek could not have anticipated. Including, as it turns out, why your retention sequence is firing for customers who are mid-return.
The knowledge problem, as it came to be called, is this: no single institution can possess all the information relevant to a decision. Knowledge is local, contextual, and held in fragments by different actors who each have a coherent picture of their own piece and no visibility into the rest. What makes systems fail is not ignorance. It is the absence of any mechanism for reconciling what the fragments collectively know.
This problem does not care about scale. It has produced some of the most expensive coordination failures in modern economic history. It is also, quietly, responsible for a meaningful portion of ecommerce retention revenue that does not exist.
The securitization chain and the information it lost
The 2008 crisis had no shortage of villains, and the subsequent decade produced enough congressional reports to keep historians occupied for generations. But underneath the conflict-of-interest structures and the regulatory failures, there was a more fundamental problem: a securitization chain in which each layer was operating on a coherent model of its own position and no model at all of what the other layers knew.
Mortgage originators had information about individual loan quality that did not travel upstream. Investment banks packaging those loans into mortgage-backed securities were modeling default correlation using assumptions calibrated to historical data that predated the origination standards producing the loans being packaged.
Rating agencies were applying AAA designations to instruments built from tranches that had never performed through a housing downturn at national scale. The Financial Crisis Inquiry Commission documented that Moody's alone rated nearly 45,000 mortgage-related securities triple-A between 2000 and 2007. Six private-sector US companies held that rating in early 2010 (FCIC, 2011).
The NBER's post-mortem on the credit rating collapse is instructive (Benmelech & Dlugosz, 2009). CDO issuance grew from $157 billion in 2004 to $551 billion in 2006. By 2007-2008, 95% of all rating downgrades were tied to residential mortgage-backed securities and CDO products. Write-downs exceeded half a trillion dollars from ABS CDO exposure alone. What is interesting about these numbers is not their size but their distribution: institutions taking write-downs in the tens of billions from exposure to instruments they had been rating or holding based on information models that bore no relationship to what the originators on the other end of the chain actually knew.
"The mortgage-related securities at the heart of the crisis could not have been marketed and sold without their seal of approval. Investors relied on them, often blindly." — Financial Crisis Inquiry Commission, Final Report (2011)
Akerlof had described this mechanism in 1970: information asymmetry between parties to a transaction produces market failure (Akerlof, 1970). What 2008 demonstrated was that you can distribute that asymmetry across an entire institutional chain, and each institution can be acting entirely rationally on the information it has, and the system as a whole can still produce an outcome that nobody chose and everybody was surprised by.
The problem was not that the models were wrong, exactly. It was that the models were never asked to talk to each other.
Monetary union, fiscal opacity
The Eurozone crisis is worth examining alongside 2008 precisely because it is a different failure mode. In 2008, private institutions shared an asset pool but modeled it differently. In the Eurozone, member states shared a monetary framework while running entirely separate fiscal regimes with no real-time mechanism for the ECB or partner states to see what any individual member was actually doing.
The Maastricht criteria set limits (deficits below 3% of GDP, debt below 60%) but created no continuous monitoring infrastructure capable of producing reliable real-time fiscal data across members. National governments reported their own statistics. The ECB set rates for the union as a whole. The gap between those two facts turned out to matter (CFR, 2015).
When the Papandreou government revised Greece's 2009 deficit from the previously reported 6% of GDP to something closer to 12.7%, with debt projections moving from 99% to 117% of GDP, the European Parliament's subsequent inquiry found what it delicately called "statistical fraud in the years preceding the setup of the programme" (European Parliament, 2014).
The CEPR's analysis noted that national central banks in both surplus and deficit countries, then still responsible for local banking supervision, failed to understand what intra-Eurozone credit flows were financing not because they were negligent, but because the system had not been designed to produce that understanding (CEPR, 2015).
"A lack of fiscal policy coordination among eurozone member states contributed to imbalanced capital flows in the eurozone, while a lack of financial regulatory centralization or harmonization... incentivized risky financial transactions by banks." — CEPR, The Eurozone Crisis: A Consensus View
A monetary union had been constructed without the information infrastructure that coordination requires. The Stability and Growth Pact was a mechanism without a model. Which is a reasonable description of most integration projects.
The structural signature
What these crises share, beneath the surface details, is a pattern. Institutions operating on the same underlying reality. Each with a coherent picture of its own fragment. No mechanism for reconciling those fragments. And a discovery process that is catastrophic rather than routine, because the inconsistencies only become visible when the system is under stress.
Hayek's argument was that you cannot solve this by centralizing information, because the knowledge is local and constantly changing and cannot survive the translation. What you can do is build communication protocols that allow local knowledge to propagate to where it is needed. Markets do this through prices. Institutions do it less elegantly, through shared data standards, interoperable systems, and people who have been asked to compare notes.
The failure mode, in every case, is not malice or incompetence. It is the absence of any designed protocol for reconciliation. Everyone is doing their job. The system is producing the wrong output because its components have never been required to maintain a shared model of the reality they collectively describe.
The ecommerce version
When an order is placed, a sequence of events occurs across systems that were not designed to speak to each other. Customer records update. Inventory adjusts. Lifecycle segments shift. RFM scores recalculate. In a separate system, a retention sequence either fires correctly or it does not, depending on whether the data it needs arrived in the format it expected. Usually it did not, because the team that built the checkout was solving a different problem than the team that built the retention flow, and neither team was in the same room when either decision was made.
The OMS knows things the retention platform does not. Which products have three-week lead times. Which SKUs generate returns at above-average rates. Whether a customer has an open return on their last order. None of this travels. So win-back sequences fire for customers mid-return. Replenishment flows trigger for customers who have already repurchased. Post-purchase sequences run for orders that have not yet shipped. Each system is behaving rationally on the information it has.
This is not a technology problem. The technology is doing exactly what it was configured to do. It is a coordination failure. The same structural signature as 2008 and the Eurozone crisis, at a scale where the cost is measured in repeat purchase rate rather than sovereign default risk, but the mechanism is identical.
Adding middleware is not the same as having the conversation
The standard response is a Customer Data Platform. The CDP will aggregate customer data from multiple sources into a single profile. Every downstream system draws from the same model. In theory.
In practice, the CDP reflects the assumptions of whoever configured the integrations. If the OMS and the retention platform have different definitions of what constitutes a completed order (which they usually do, because the concept means something different depending on whether you are tracking inventory state or customer journey stage) the CDP will produce a unified definition that is wrong in subtly different ways for both systems. The Eurozone built a Stability and Growth Pact and discovered that a coordination mechanism without a shared model of the underlying reality is not, in the end, a coordination mechanism.
What actually needs to happen is earlier and less technical. Someone who built the back end needs to sit with someone building retention flows and walk through what happens to customer data after checkout. Not at the level of API documentation. At the level of: here is the sequence, here is what each system records, here is where the data stops traveling, here is why. Most developers have a precise mental model of this. It has almost never been shared with the people who depend on it, because the organizational structure does not produce the meeting where it would happen.
The checkout is not a transaction. It is the first data point in a customer relationship, and most ecommerce operations are architected as though it were the last. The back-end systems that govern everything that happens after that moment are, in Hayek's terms, holding local knowledge that the rest of the organization needs and has no protocol for accessing.
Everything downstream is making decisions on fragments. And as the historical record on this particular failure mode suggests, that tends to end badly. Just usually not all at once, and usually not in a way that makes it obvious what went wrong.
Sources
Comments