AI Language Models Fail at Real Decisions
Diogo Almeida helped build the machine that made chatbots fluent, then watched it fail at the one thing that mattered. RLHF — reinforcement learning from human feedback — taught models to produce text that reads like a person wrote it. That turned out to be a magnificent trick and a trap. “We have lightning in a bottle, and yet it is not useful,” he told TechCrunch. [1] The system was correct in every way a human reader could judge, and wrong in the only way a computer cares about: it could not be trusted to decide anything on its own.
The Confidence Problem
Ask a language model whether an email is a sales pitch or a support request and it will answer. Ask it how sure it is and the trouble starts. The number it hands back is generated the same way the answer was — as plausible-sounding text. A model that says “95% confident” has not measured anything. It has written a sentence that looks like a measurement. This is the deception at the center of the current AI boom, and it is not a bug anyone can patch, because the entire architecture is built to produce language, not to produce truth. The output is calibrated to sound right, not to be right, and those two things diverge exactly when a decision has consequences.
Almeida spent years inside that divergence before naming it. The problem, he concluded, was that the industry had optimized for human language. Computers do not speak human language. They speak probabilities, thresholds, yes-and-no. Four years of spectacular progress in making machines talk had produced almost nothing that could be wired into software without a human standing by to catch the mistakes.
What Jev Does Differently
His company, TypeSafe AI, released a model called Jev that does not write. It returns probabilities — what the company calls calibrated decisions. The user defines the possible outputs in advance, which means the model cannot invent a category that was not on the list. It cannot hallucinate because there is nothing to hallucinate into. The output is a number between zero and one, and that number means what it says.
The economics follow from the architecture. Output tokens are free. Input tokens are metered by the billion rather than the million. When Vercel’s Pranit Sharma swapped out OpenAI’s Luna for Jev on a safety classifier, results came back five to 18 times faster and more accurate. [3] [4] Nikhil Mudholkar, CTO of Bryo AI, ran Jev against Gemini on business email sorting. [5] Gemini edged it on accuracy and cost 10 to 20 times more. [5] But the number that interested him was the confidence score. “It is the only one that hands back a real probability which makes it ideal for automating workflows,” he said. [5]
The Delegation of Doubt
Here is where the deception shifts rather than disappears. Armin Ronacher, CTO of Earendil, put it plainly: the model delegates the hallucination problem to the user. [6] If Jev returns 50%, the user has to decide that this is a coin toss and disregard it. If it returns 95%, the user can act. The system is honest about its uncertainty, but honesty about uncertainty is not the same as being right. It moves the burden from the machine to the person reading the number, and that person now has to know what a 50% threshold means for their specific workflow, their error tolerance, their liability.
This is a real improvement over a chatbot that states a falsehood with perfect grammar. It is not a solution to the underlying problem. A calibrated probability is only as good as the calibration, and calibration is a statistical property that holds across a population of decisions, not for any single one. A model that says 95% is right 95 times out of a hundred. The other five times, it is confidently wrong, and the user who trusted the number has no way to know which case they are in until the damage is done.
Why This Took So Long
Ronacher’s explanation for the delay is blunt. “We should have seen this earlier in many ways, but presumably because the LLMs are so cheap and subsidized, you often don’t have to be creative yet.” [6] The subsidies masked the inefficiency. As long as someone else was paying for the tokens, there was no pressure to ask whether language was the right output format for a decision that had nothing to do with language. The industry built cathedrals of text because text was what the models produced, and then spent years trying to make text behave like a decision.
Almeida’s bet was on synthetic data. Jev is trained exclusively on data the company generates, using a technique he calls reinforcement learning from calibrated decisions. “We made an early bet that we will be making all of our data, and that has been one of the best bets I’ve ever made in my life — better than our launch, in my opinion, better than RLHF,” he said. [2] Half the company, in his description, is a lab that owns a subfield of statistically well-understood synthetic data. The claim is that because the training data is generated rather than scraped, its statistical properties are known, and a model trained on known data can be calibrated in a way a model trained on the open internet cannot.
The Uses That Matter
Jev is not only a replacement for language models. It can watch them. Almeida sees users deploying it to track agent traces and catch jailbreak attempts — using a cheap, fast, non-linguistic model to monitor the expensive, slow, linguistic ones. Agents monitoring agents gets costly quickly, but a model that returns a probability at a fraction of the cost changes the arithmetic. Model routing is another application Ronacher points to: predicting whether a given task needs a specific model, in real time, without spending an LLM call to make the prediction.
The name comes from William Stanley Jevons, the 19th-century economist who noticed that making a commodity cheaper tends to increase how much of it gets used rather than decrease it. Almeida’s hope is that the same logic applies to intelligence. “We think that there’s just going to be smart software all over the place in a way that’s emergent and distributed… much more like the early internet than you know like the mega apps that people are trying to build right now,” he said. [2] Cheap decisions, embedded everywhere, doing small things well.
The Question Left Standing
TypeSafe calls Jev a “System One model” — focused on intuition rather than reasoning. Almeida is tight-lipped about the architecture, and outside observers suspect it sits on top of an open-weight language model. He rejects the label of frontier lab. “The main product of frontier labs is fear or hype,” he said. [2] “I would like our main product to be intelligence… [but we are] not a lab in the sense of, you know, like bet on infinite wealth, or a religion, or building God in a data center, or whatever is the thing of today.”
The honest version of the Jev story is not that it solves AI deception. It is that it hands the doubt back to the person who has to act on it. A language model hides its uncertainty inside fluent prose, and the user never sees the doubt because there is no doubt to see — only text. Jev surfaces the uncertainty as a number, and the user is left holding it. When the 95% case turns out to be one of the five, the model did not lie. It said what it believed. The person who acted on it has to answer for the outcome, and the system that produced the number has already moved on to the next decision. That is the arrangement Almeida built, and it is a better one than what came before. It is not an arrangement where anyone escapes responsibility. It just makes clear who was always holding it — and that was never the machine.
Sources
1. TechCrunch
2. TypeSafe AI
3. Vercel
4. OpenAI
5. Bryo AI
6. Earendil