AI Code Looks Right But Deceives Engineers
The promise was straightforward: AI writes code, engineers ship faster. What no one predicted was that the speed of generation would outpace the speed of understanding — and that the gap between the two would become the defining problem of software engineering.
The Illusion of Clean Code
Large language models have become remarkably good at producing code that reads well. Variables are named sensibly. Functions are organized logically. Comments explain what each block does. The surface is polished. And that polish is precisely what makes the deception so effective.
A survey of more than 1,100 developers by Sonar, an AI code-verification startup, found that respondents estimated AI contributed 42 percent of the code they added to shared codebases. Yet 96 percent did not fully trust its output to work correctly. This is not a contradiction born of paranoia. It is a rational response to a tool that presents confidence it has not earned.
The code compiles. It passes the tests that were written for it. It looks like something a competent engineer would produce. But beneath that appearance, large language models embed faulty assumptions, security vulnerabilities, and subtle errors that surface only after deployment. The cost of fixing them can erase every hour the AI supposedly saved.
The Review Bottleneck Nobody Planned For
When code generation accelerates, review becomes the constraint. According to the Sonar study, 38 percent of developers said more effort is required to review AI-generated code than code written by colleagues, and 61 percent reported that AI often produced code that looked correct but was unreliable
At Synthesia, an AI video-generation platform, the numbers tell the story. As of November 2025, pull requests — proposed changes to the codebase submitted for review — had risen 120 percent year over year, according to Peter Hill, the company’s chief technology officer. Ninety-five percent of those requests contained AI-generated code The volume is not the only problem. The nature of the errors is.
Hill describes a recurring pattern: AI tools fail to recognize that code for a task already exists. Because they have limited context, they write another version. Synthesia has found as many as 10 versions of the same function, leaving engineers to identify and remove redundant code, then retrain the agent so it does not repeat the mistake. At the company’s scale, Hill calls getting the AI to produce the intended output an enormous amount of work.
“I don’t know if we ever get to the point where you can truly trust the agentic generation of code,” he says.
The Specification as Preemptive Correction
Some teams are trying to intercept the deception before it begins. McLaren Stanley, a senior principal engineer at Amazon Stores, leads a 70-person team supporting more than 1,000 developers who maintain the architectural backbone of Amazon’s mobile shopping app. With AI writing code, Stanley says engineers now spend more time deciding what the code should do before generation starts.
Much of that work involves writing a specification — a detailed plan for what the AI agent should build and how. The specification is not documentation. It is a constraint. It narrows the space in which the model can make assumptions, and assumptions are where AI deception lives.
Stanley recalls a missing instruction that caused an agent to generate 25,000 lines in the wrong version of the programming language Swift. Switching versions produced 600 errors the agent could not fix. Stanley discarded the code, updated the specification, and restarted the agent. Fifteen minutes later, it regenerated the code correctly. The first attempt looked like progress. It was not.
Agents That Read So Humans Can Judge
Once code exists, specialized AI agents can perform a first pass before a person sees it. David Yanacek, a senior principal engineer at Amazon Web Services, says the company uses agents to test whether code works, check it against the original plan, and look for security flaws before human review
That first pass becomes more critical as volume increases. At Bonterra, a nonprofit software provider with about 290 engineers, proposed changes tripled within three months of adopting AI, according to CTO Tanuja Korlepra. Code entering review rose tenfold and review times tripled, making it impractical for engineers to inspect every line
Bonterra’s agents compare code against the approved design, security rules, coding standards, and accessibility requirements, then report their confidence in the result. A low score or flagged problem sends the change to a person. Code involving payments, personal data, or other sensitive systems always receives human review.
“Agents do the reading and humans do the judging,” Korlepra says.
Synthesia also uses AI agents to decide where human review is necessary. Criteria set by engineers direct more scrutiny toward higher-risk changes. Altering an error message carries less risk than code that handles customer data or core business rules. Even so, fewer than 5 percent of changes bypass human review entirely
The Theater of Approval
Automated review does not change who is responsible for the code. But when machines produce more code than engineers can closely read, human approval can become what JD Raimondi, chief AI architect at the software consultancy Making Sense, calls “theater approval.” An engineer confirms that the feature works, skims the code, and approves it — all without understanding the choices underneath.
This is the deepest form of AI deception: not that the code is wrong, but that the review process itself becomes a performance. The signature is there. The judgment is not.
Temporal, an open-source developer platform, puts the burden back on the person submitting the code. CEO Samar Abbas says code volume and review time have increased with AI. Under its “Send Back” policy, Temporal’s engineers must explain in their own words the agent’s design choices and how the code handles unusual conditions. Otherwise, the reviewer rejects it.
“We refuse to let code review become a dumping ground for unchecked model outputs,” Abbas says.
What Junior Engineers No Longer Learn
As AI shifts engineering work from writing code toward judging it, companies are reconsidering how entry-level engineers gain experience. Junior engineers at Making Sense have seen some of the largest productivity gains from AI, Raimondi says, raising concerns about what they no longer learn by doing. The consultancy keeps juniors involved in deciding why a customer needs a feature and how it should work, rather than limiting them to checking AI output.
IBM is using AI to give new engineers harder assignments sooner. Neel Sundaresan, IBM’s general manager of automation and AI, says recent graduates now work on product features and projects once reserved for senior-level engineers. AI helps implement and test the code, but if it fails, juniors assess what went wrong and fix the issues before the work passes to senior developers for final approval. Sundaresan estimates that AI can help junior engineers perform 70 to 80 percent of some tasks that once required a senior engineer
The risk is not that juniors will be replaced. It is that they will never develop the instinct for what AI code hides — because they never wrote enough code themselves to recognize when something is subtly wrong.
The Gap Between Measurement and Reality
Investors see opportunity in closing the gap between what AI claims to do and what it actually does. In August, AI code-review startup CodeRabbit raised US $143 million at a $1.5 billion valuation, claiming it performs more than 2 million reviews a week for 17,000 customers, including Nvidia, Indeed, and BMW Group. [9]
The numbers are impressive. They are also a measurement of volume, not of reliability. The question is not how many reviews an AI can perform. It is whether those reviews catch what matters — the faulty assumption buried in a function that passes every test, the security flaw that only manifests under load, the subtle error that surfaces three weeks after deployment when the engineer who approved it has moved on to the next feature.
The new era of code review will determine whether AI can provide code that is both faster and more reliable. That determination will not be made by benchmarks or funding rounds. It will be made in the quiet moments when an engineer reads a line of code and asks: do I understand what this does, or do I only understand what it appears to do?
The comparison that matters is not between AI and human code. It is between a workflow that treats review as a checkpoint and one that treats it as a form of understanding. The first produces theater. The second produces software. The choice between them is still ours to make.
Sources
1. Sonar
2. Synthesia
5. Bonterra
6. Making Sense
7. Temporal
8. IBM
9. CodeRabbit
10. Nvidia