Code Entropy and LLMs

Introduction#

Take the same LLM, the same prompt pattern, and the same developer. Point the model at a clean module with consistent naming, explicit dependencies, and clear control flow. Then point it at a legacy module in the same repository: inconsistent conventions, dead code paths, implicit coupling to three other files, duplicated logic that has diverged over time.

The results will be radically different. Not because the model got dumber between requests. Because the code got harder to predict.

Most discussions about AI-assisted development focus on the tool: which model, which context window, which prompting strategy. This framing misses the variable that matters most. The code itself determines whether AI assistance helps or hallucinates.

Software Entropy Is a Law, Not a Metaphor#

Lehman’s second law of software evolution states that the complexity of an E-type system increases with every change unless deliberate work is done to reduce it. This is not a suggestion. It is an empirically observed tendency confirmed across decades of longitudinal studies on production systems.

Software entropy is the concrete manifestation of this law. It shows up as:

  • Inconsistent naming conventions across modules written by different people at different times
  • Dead code paths that nobody deletes because nobody is confident they’re truly dead
  • Duplicated logic that started identical and has since diverged in subtle ways
  • Implicit coupling where file A depends on behavior in file B that isn’t expressed in any interface
  • Tangled control flow where conditional logic is scattered across multiple layers

Each of these is a form of disorder. Each makes the next token in a sequence harder to predict.

Constantine’s Equivalence quantifies the economic cost: total effort grows with the square of changes made. Entropy is the mechanism that drives the M term (effort per change) upward. As the codebase becomes more disordered, every change becomes harder, and the squared relationship amplifies the damage.

Why LLMs Depend on Predictable Code#

LLMs are next-token prediction machines. The entire architecture bets that statistical patterns in training data can predict what comes next in a sequence. The more predictable the next token, the higher the model’s confidence, the better the output.

This bet pays off on code because of a property discovered by Hindle et al. in “On the Naturalness of Software” (ICSE 2012). Source code is significantly more repetitive and predictable than English prose. Software has higher statistical regularity than natural language. This is the naturalness hypothesis, and it is the reason LLMs work so well on code in the first place.

Allamanis et al. confirmed and extended this finding in “A Survey of Machine Learning for Big Code and Naturalness” (ACM Computing Surveys, 2018). The survey established that software’s predictability comes from conventions, idioms, API usage patterns, and the constrained vocabulary of programming languages. When developers follow consistent patterns, statistical models can exploit that consistency to make accurate predictions.

The inverse follows directly. When code violates its own patterns, when the same concept is expressed three different ways, when naming conventions conflict, when dead code introduces misleading signal, the statistical regularity that LLMs depend on degrades. The model’s predictions become less confident and less accurate in proportion to the entropy in the input.

What Entropy Does to a Prediction System#

Hindle et al. did not just observe that code is predictable. They measured it. Their cross-entropy calculations showed that code has lower entropy per token than English prose. The statistical models they tested (n-gram models, the predecessors to modern LLMs) achieved their performance specifically because developers write code that follows local conventions. The regularity is the substrate.

Allamanis et al. (2018) identified what creates that regularity: “developers prefer to write code using a small number of patterns and idioms.” API usage clusters around common call sequences. Error handling follows project-local templates. State management repeats established patterns. This is not an accident. It is how teams coordinate without constant communication. Conventions are compressed knowledge about how this codebase works.

Entropy is the degradation of that regularity. Not the absence of patterns, but the presence of conflicting ones. A codebase with three incompatible error handling strategies, two diverged copies of the same validation logic, and implicit state shared across module boundaries does not lack patterns. It has too many. The model receives conflicting statistical evidence about what comes next and must assign probability mass across mutually exclusive continuations. Confidence drops. Hallucinations rise.

The empirical evidence that this matters at scale comes from SWE-bench (Jimenez et al., ICLR 2024). The benchmark draws from real GitHub issues across 12 production Python repositories. Resolving those issues “frequently requires understanding and coordinating changes across multiple functions, classes, and even files simultaneously.” The best-performing model at publication solved 1.96% of issues.

What makes SWE-bench hard is not any single mechanism. It is accumulated entropy in aggregate. Implicit coupling between files. Conventions that have drifted over time. State managed in ways that no single file reveals. The issues with the lowest solve rates require the model to hold a mental model of how disordered systems actually behave. That is precisely what entropy prevents.

Liu et al. showed in “Lost in the Middle” (TACL, 2023) that even when the answer exists in the context, surrounding it with noise degrades retrieval. Accuracy dropped from 75.8% to 53.8% by changing where relevant information sat relative to irrelevant material. In a high-entropy codebase, the correct pattern exists somewhere in the context window. It is surrounded by conflicting patterns. The model loses it.

This is not a list of separable failure modes. Statistical regularity degrades, and prediction quality degrades with it. The specific form of the entropy (naming drift, dead code, implicit coupling, scattered control flow) matters less than the aggregate effect on the model’s confidence distribution.

The Entropy Ratchet#

AI-assisted development tools generate code fast. Speed is their primary value proposition. But speed without entropy reduction means faster accumulation of the very disorder that degrades the tool’s effectiveness.

This is a feedback loop:

  1. AI generates code quickly
  2. Fast generation skips the deliberate work that prevents entropy accumulation (consistent naming, removing dead paths, making coupling explicit)
  3. Entropy increases
  4. The AI becomes less effective on subsequent requests
  5. The developer compensates by providing more context and correcting more outputs
  6. Velocity drops toward the pre-AI baseline or below it

The hidden cost of agentic development documents this pattern through the lens of Constantine’s Equivalence. AI tools optimize for initial code generation speed while ignoring the maintenance economics that dominate total cost.

Architectures that emphasize locality resist this ratchet. Vertical slices contain entropy within feature boundaries. When each slice is self-contained, entropy in one slice does not degrade the model’s effectiveness on another. Shared abstractions, by contrast, are entropy magnifiers: one inconsistency in a shared module ripples into every context that includes it.

Lehman’s sixth law compounds the problem. Functional content must continually increase to maintain user satisfaction. The system must grow. Growth increases entropy surface area. Without proportional investment in entropy reduction, the ratchet tightens with every release.

The Stall Point#

At some threshold of accumulated entropy, the LLM crosses from net-positive to net-negative. It generates more problems than it solves per interaction. Every suggested function conflicts with an existing convention. Every fix introduces a new inconsistency. The developer spends more time correcting the model than they would have spent writing the code directly.

SWE-bench provides empirical evidence that this threshold exists and that the degradation is cliff-like, not gradual. The benchmark stratifies issues by complexity. For simple, self-contained changes, modern models achieve reasonable solve rates. For issues requiring coordination across files through implicit dependencies, solve rates collapse toward zero. The curve does not decline linearly. It drops.

This pattern is observable without benchmarks. Teams already experience it on specific modules: the legacy service where Copilot suggestions are wrong more often than they’re right. The module where every AI-generated PR requires extensive manual correction. The file where developers disable AI assistance entirely because the suggestions introduce more bugs than they prevent.

The stall point is not a fixed complexity number. It varies by model capability, context window size, and codebase characteristics. But the direction is fixed by the architecture of next-token prediction. As entropy increases, prediction accuracy decreases. At some point, the decrease crosses the threshold where AI assistance becomes AI hindrance.

The Belief That Won’t Save You#

Give it a year. Two years. Context windows will expand. Reasoning will improve. The model will handle any complexity we throw at it. We won’t need to care about code quality because the machines will figure it out regardless.

This misunderstands what entropy does to a prediction system.

More parameters allow a model to memorize more patterns. But code entropy means the patterns themselves conflict. When the same concept is expressed multiple incompatible ways in the same codebase, no amount of memorization resolves the ambiguity. The signal is genuinely ambiguous. The model must guess. More parameters do not convert a guess into a certainty.

Larger context windows allow more code in the input. But Lost in the Middle showed that more context with more noise produces worse results, not better. Stuffing more high-entropy code into the window does not help. It gives the model more conflicting evidence to navigate.

Better reasoning helps with logical deduction. But implicit coupling is not a reasoning problem. It is an information availability problem. The dependency between file A and file B cannot be deduced from reading file A alone, no matter how capable the reasoning system. The information is physically absent from the input.

The fundamental constraint is architectural. A system that predicts the next token from statistical patterns in its input is vulnerable to any degradation in those patterns. Code entropy is precisely that degradation. This vulnerability is not a capability gap that scales away with compute. It is a structural property of how the system works.

The code still has to make sense. Not just to you. To the thing predicting what comes next.