AI Auditors Lose Authority as Machines Self Evaluate
When the Auditor Becomes the Audited
The promise was straightforward: put a second set of eyes on the most consequential technology in human history. Independent evaluators would test frontier models, probe their limits, and report back to a public that has no other window into what these systems can do. The arrangement sounded like governance. It functioned as theater.
In practice, the evaluators arrived with less access than the companies’ own safety teams, less compute than the models they were supposed to scrutinize, and less leverage than a mid-level product manager. They asked questions the labs had already answered internally. They ran tests the labs had already run. The judgment that mattered — whether a model was safe enough to deploy, whether its capabilities crossed a line, whether the risks justified a pause — remained where it had always been: inside the organizations building the thing.
What disappeared was not the evaluation. It was the evaluator’s authority. The role survived as a title while the function migrated back to the people with the GPUs.
The Number That Says Everything
This month Anthropic disclosed that its own AI systems now perform 26 percent of the company’s AI research. [1] At the start of 2025, that figure was zero. The same disclosure noted that 6 percent of the company’s compute budget goes toward making its AI safer. [1]
Read those two numbers together and a structural fact emerges. The institution charged with understanding whether AI is dangerous is increasingly staffed by AI. The research that would tell us what these systems are capable of is itself being conducted by systems we do not fully understand.
This is not a scandal. It is a business model. Research is expensive, slow, and hard to scale. Models are cheap, fast, and parallelize. The substitution is rational. It is also the precise mechanism by which human judgment becomes optional.
The Question That Has No Reader
The RSI Index, a new benchmark, is one of several efforts to measure recursive self-improvement — the possibility that AI systems will soon be able to improve themselves faster than humans can understand or intervene. The metric is useful. The audience is the problem.
Who is supposed to look at the RSI Index and decide what to do? The companies that are racing to build the thing it measures? The regulators who cannot hire enough technical staff to interpret it? The public, which has no mechanism for acting on the information even if it understood it?
We have more measurement than ever. The puzzle is that the role of the person who interprets those measurements, weighs risks, and makes a binding decision has been quietly eliminated.
The Solved Problem That Isn’t
Every element of an AI slowdown has been technically specified. Evaluations exist. Compute tracking exists. Chip-level controls exist. Treaties have been drafted. Destruction scenarios have been imagined. The engineering is done.
What remains unsolved is the human part: who decides, who enforces, who bears the cost of being wrong. That part cannot be automated, because the whole point of the exercise is to preserve the capacity for human judgment in the face of a technology that is dissolving it.
The 26 percent is not a statistic about efficiency. It is a description of a vacancy. The role that was supposed to belong to a person — the person who looks at the machine and says no — has been filled by the machine. The 6 percent is the budget for confirming that this arrangement is safe, and the confirmation comes from the same source as the arrangement.
We built the instruments of control and then handed them to the thing they were meant to control.