Falsifiability for Engineers
In 1919, Arthur Eddington photographed a total eclipse from an island off West Africa and measured how far the light of distant stars appeared to shift as it grazed the sun. Einstein had committed to a number in advance, and had the photographs come back matching Newton instead, general relativity was finished. Einstein said as much himself. The theory was worth believing afterward because it had been standing somewhere it could have been knocked down.
Karl Popper built a philosophy of science on that asymmetry. No pile of confirming observations can establish a general claim, because the next observation is always outstanding. One contrary observation retires it. So the interesting question about any claim is not what evidence supports it. It is what evidence would force you to abandon it, named in advance, before you know how things turn out. Popper called a claim falsifiable when that evidence exists and can be stated. A claim with no such evidence, one that fits every possible outcome equally well, is not a strong claim. It is a claim that has stopped saying anything about the world while continuing to sound like it does.
Our field is full of both kinds, and we do not always sort them.
“This architecture is flexible” is the one I have watched survive every possible future. When a later change lands easily, the architecture gets the credit. When a change turns out to be brutal, the reading becomes that nobody could have predicted that particular axis of change. Both outcomes leave the claim standing, which means the claim was never doing any work. Compare it to “adding a payment provider will touch one module and no others.” That one can die. Somebody adds Klarna, the diff spans nine packages, and the claim is gone. The second version is riskier to say out loud, and that risk is precisely what makes it worth something.
The asymmetry is already in our bones#
Dijkstra put the same point in our own vocabulary decades ago when he observed that testing can reveal the presence of defects but never their absence. That is Popper, applied to software, and it is the reason a green suite feels less reassuring the more carefully you think about it.
A passing test is a failed attempt to refute the code. That is all it is, and it is not nothing. Popper called this corroboration, and he was careful never to call it proof. Corroboration is a report on the past, a record of the beatings the claim has taken and survived, and it licenses no promise about tomorrow.
What follows from that is a change in how you count. Confidence should track the severity of what your code has survived, not the number of survivals. Four hundred assertions that exercise the same happy path through the same seam corroborate almost nothing, because they were never in a position to fail separately. One test that drives a real serialization boundary with a genuinely hostile input tells you more than all of them, because it went somewhere the code could have died. Coverage counts attempts. It has no way to measure whether the attempts were dangerous.
Sharp claims and the machinery that holds them#
Much of the tooling we are proudest of exists to take a vague intention and grind it into a claim sharp enough to break.
A type signature is a statement about what the program will refuse to do. A database constraint forbids a row. An invariant asserted at a boundary forbids a state. A schema forbids a message shape. Each of these narrows the set of worlds in which the system is behaving correctly, and the narrower that set, the more you learn the moment reality steps outside it. This is why a well placed constraint feels good in a way that a comment expressing the same wish never does. The constraint has teeth.
The craft, then, is partly a matter of taking intentions that live in your head as feelings and giving them somewhere to fail. “The cache should help” becomes a p99 target you will read on a date you picked beforehand. “These services are independent” becomes a dependency rule the build enforces. Nothing about the underlying belief changed. What changed is that the belief is now exposed, and you will find out.
The quiet way claims die#
Popper’s other lasting contribution was noticing what people do when a claim they like runs into trouble. They rescue it. Not by lying, but by amending the claim just enough that the inconvenient observation no longer counts, and each amendment feels reasonable on its own.
I have done every version of this. The test that fails one run in thirty gets a retry wrapper instead of an investigation. The latency target gets revised upward after the quarter where we missed it, using the reasoning that the original number was always arbitrary. The incident gets reclassified as degradation rather than an outage because the health check was still returning two hundred. Every one of those moves is cheap in the moment and defensible in isolation. What they cost is the thing the claim was for. A target that moves to wherever the system happens to be is no longer telling you about the system.
The discipline worth building is not never adjusting a claim. Sometimes the number really was wrong. It is noticing the difference between adjusting a claim because you learned something and adjusting it because it was about to embarrass you, and being willing to say which one you are doing in front of other people.
Not everything is a hypothesis#
Falsifiability is a property of empirical claims, and plenty of what we argue about is not empirical. Whether the tests read well, whether the module names carry their weight, whether a particular abstraction is pleasant to sit behind. These are matters of taste, and taste is a real thing that a craftsman is entitled to have and defend.
Nothing goes wrong until taste puts on the clothing of an empirical claim. “I find this hard to read” is honest and hard to argue with. “This will slow the team down” is a prediction, and a prediction that arrives without a number, a date, or any interest in being checked is doing rhetorical work rather than engineering work. Knowing which kind of claim you are making, and saying so, costs nothing and settles more arguments than winning them does.
So here is the habit worth stealing. The next time you write a claim down anywhere somebody else will read it, in a design doc, a pull request, a ticket, finish it with a sentence that begins “I would know this was wrong if.” Some of the time you will name something concrete, and you have just made your claim worth more than it was a minute ago. Some of the time you will find you cannot finish the sentence at all, and that is the more valuable outcome, because you have discovered what kind of claim you were actually making before it went into the world dressed as something else.