Thinking too much can make AI dumber
There is a story from the early days of cartography, when mapmakers first began adding mountains to their charts. The problem was not that they lacked the tools to measure elevation, but that every hour spent sketching a ridge was an hour not spent tracing a coastline. Some of the most detailed maps of the era were also the most useless, because their creators had spent so long perfecting the interior that they never finished the borders. We are watching the same error repeat itself in artificial intelligence, only now the mountains are made of planning steps and the coastlines are the answers we actually need.
The new research, available as a preprint, documents a strange inversion of common sense. When you give an AI system more room to think—more tokens, more structure, more chances to verify its work—it does not always get smarter. Sometimes it gets dumber, and the reason is embarrassingly simple: thinking costs tokens, and tokens are a finite resource. The paper, available at arxiv.org/abs/2608.27506, tracks two competing architectures across fourteen different budget tiers, from a meager 250 tokens to a generous 42,000. The preprint has not yet been peer-reviewed, and its findings should be read with that caveat in mind. The results are not a smooth curve of improvement but a jagged landscape of trade-offs, with a crossover point that reveals something uncomfortable about how we measure machine intelligence.
The Empty Scoreboard
Start with the lowest budgets, because that is where the deception begins. At 250 and 500 tokens, both systems score exactly 0%. [1] They are not failing because they are stupid; they are failing because they cannot even fit the question into their working memory. The prompt alone consumes the entire budget, leaving nothing for computation. This is the first gap between what AI appears to do and what it actually does: a system that scores zero is not making an error, it is simply not running. Yet the scoreboard does not distinguish between these states, and so we read failure where there is only absence.
At 1,000 tokens, the gap widens into something more troubling. The monolith—a single, uninterrupted call to the language model—reaches 18% accuracy. The verified search architecture, which adds planning, checking, and repair steps, scores near 0%. The structured system is not worse because it reasons poorly; it is worse because it spends its entire budget on the scaffolding and never reaches the answer. This is the second deception: the appearance of rigor, the elaborate machinery of verification, produces nothing at all when the resources are thin. The system looks like it is thinking, but the thinking is happening in a vacuum, with no room left for the actual response.
The crossover arrives between 1,000 and 1,500 tokens, confirmed by a strict statistical test at both endpoints. [1] From that point onward, the verified search architecture pulls ahead and never looks back, reaching roughly 44% at the highest budgets while the monolith plateaus near 40%. The paper is careful with its numbers, and the difference is not dramatic—four percentage points is not a revolution. But the shape of the curve is the story. There is a threshold, a point of diminishing returns where structure stops being overhead and starts being an asset. Below that line, the more elaborate system is a liability. Above it, the same system becomes an advantage.
The Measure That Lies
This is where the deception becomes structural rather than incidental. The standard way to evaluate an AI system is to feed it a task, count its correct answers, and call that number its intelligence. But the token budget is a hidden variable, and it changes the meaning of every score. A system that scores 18% at 1,000 tokens and 40% at 42,000 tokens is not the same system; it is a different creature at each point. The budget is not a setting, it is a dimension, and flattening it into a single number is like measuring a building by its shadow.
The deeper issue is that the token budget itself is an artificial constraint, a proxy for cost that has become a proxy for cognition. In the real world, we do not pay per thought. A human expert does not run out of reasoning tokens halfway through a financial analysis. But the AI does, and so the architecture must decide: spend the budget on planning or spend it on answering. The verified search system makes the rational choice at high budgets, but at low budgets it makes a catastrophic one, pouring its resources into a process that never produces output. The monolith, by contrast, is dumb but efficient, skipping the planning entirely and going straight for the answer, even if that answer is often wrong.
The financial reasoning tasks in the study—FinQA and TAT-QA—make the stakes concrete. [1] [2] These are not abstract puzzles; they are the kind of questions an analyst might ask about a company’s earnings or a balance sheet. When the budget is tight, the monolith at least attempts an answer, and sometimes it is right. The verified search system does not even try, because it is too busy verifying a plan that never materializes. This is the third deception: the appearance of carefulness, the promise that more structure means more reliability. In practice, structure can be a way of avoiding the answer, a bureaucratic delay in a system that was supposed to be instant.
The Clarity at the End
What becomes clear, reading the full results, is that the crossover point is not a fixed law but a moving target. The paper tests one model, GPT-5.4 mini, and two tasks, and the threshold could shift with different architectures or different domains. The authors themselves caution against generalizing from a single model and two benchmarks. The principle, however, is likely to hold: there is always a budget below which thinking is a luxury you cannot afford. The implication is not that we should abandon structured reasoning, but that we should stop pretending that more thinking is always better. The token is the unit of both cost and cognition, and the two are inseparable.
The historical parallel is exact. The mapmakers who drew mountains instead of coastlines were not lazy; they were following a logic that made sense in isolation. A map with detailed mountains is a better map, if you ignore the missing coastline. An AI that plans and verifies is a better AI, if you ignore the token budget. But the budget is never ignorable, and the coastline is never optional. The deception is not in the systems themselves but in our willingness to read their outputs as if they were independent of their constraints.
The moment of clarity, then, is not a solution but an understanding. We are not measuring intelligence when we score these systems; we are measuring a trade-off between depth and breadth, between structure and output, between thinking and doing. The AI that appears to reason carefully may simply be spending its budget on theater, while the AI that appears to rush may be the only one actually answering. The token is the truth, and everything else is a story we tell ourselves about what the machine is doing. The map is not the territory, and the score is not the mind. The next time a vendor boasts about a new reasoning model, ask not how smart it is, but how much it costs to let it think.
Sources
1. TAT-QA