AI Doom Debate Is About Who Decides
The loudest argument in AI right now is not about whether the technology will end the world. It is about who gets to hold the wheel while everyone else is told to trust the driver. When Jacob Coxon resigned from Anthropic over fears that leading labs are “gambling with our lives,” he did something the industry rarely does: he made a private judgment public. [1] That is the part worth examining, because it is the part that keeps disappearing.
The Warning That Was Also a Résumé
Anthropic’s alignment lead reposted Coxon’s thread with the declaration that the company “really do earnestly believe AI could kill all humans,” attaching a probability of more than 10 % to the next decade. [1] Read that sentence twice. A named employee of a company that may eventually sell shares to the public is stating, in writing, that the product line carries a meaningful chance of erasing the species. In any other industry, that would be a disclosure event. Here, it functions as positioning.
The exclamation mark did the work of a press release. It signaled capability without a benchmark, seriousness without a number anyone can check, and danger without a single named victim. The percentage is the tell. No model produced it, no dataset supports it, no method generated it. It is a feeling wearing the costume of a forecast, and it was published by people whose job is to know the difference.
When Judgment Gets
Outsourced to the Loudest Room
What made Coxon unusual is not that he believed something alarming. Plenty of people do. It is that he treated his own belief as a reason to stop, rather than a reason to keep going with a raised profile. That is a judgment call, and judgment is precisely what the current debate keeps trying to automate away. If the risk is real, the honest move is to leave. If the risk is theater, the honest move is to say so. The industry has found a third option: stay, warn, and let the warning do the marketing.
This is where the human role quietly goes missing. A researcher who genuinely fears extinction-level outcomes has one clear obligation: remove themselves from the work. A company that genuinely fears the same has one clear obligation: stop building. Neither happens, and the gap between the stated belief and the actual behavior is filled with a new kind of professional — the person whose job is to articulate danger while continuing to produce the thing that creates it.
The Reflex That Replaces Deliberation
Kirsten Korosec asked the question that should anchor every story about this: is the doom talk a strange way of flexing? Kirsten Korosec, TechCrunch The answer is less interesting than the mechanism. When a lab says its models are approaching thresholds no one controls, it is describing capability in the language of catastrophe. The catastrophe is the proof. Fear becomes the most efficient way to say “we are ahead” without having to prove it against anything measurable.
That reflex has a cost, and the cost lands on the people who used to make these calls. A safety team that once weighed evidence now weighs vibes. A regulator who once asked for test results now receives a probability estimate with no denominator. A journalist who once checked a claim now reports a feeling. Each substitution is small. Together they remove the human step where someone looks at the data and decides. What remains is a room full of people agreeing that something terrible might happen, and no one whose job is to say whether it will.
The Filing That Has to Say the Quiet Part
Sean O’Kane raised the detail that turns this from discourse into paperwork: Anthropic may file for an IPO, which means an S-1, which means risk factors written in plain language and signed by people who can be sued. Sean O’Kane, TechCrunch. Imagine the junior lawyer tasked with drafting the sentence. The company’s official position, in a document filed with regulators, would have to state that there is a meaningful chance its own technology eradicates humanity and that this would be materially bad for business. That sentence is not a blog post. It is a liability.
Here the human role is not replaced by a model. It is replaced by a formula. The doom language that works beautifully on a podcast becomes a problem the moment it has to survive contact with securities law. Either the claim is real, in which case the filing is extraordinary, or it is not, in which case the people making it have been saying something they do not mean. There is no third reading that survives a signature.
What the Percentage Actually Measures
A number like more than 10 % feels precise, which is why it spreads. It implies a model, a base rate, a method. None of that exists here. What exists is a person with a platform choosing a figure large enough to alarm and small enough to be plausible, then watching it circulate as though it had been calculated. The number does not describe the future. It describes the speaker’s desire to be taken seriously.
That is the quiet substitution at the center of all of this. Judgment used to mean weighing evidence and accepting the consequences of being wrong. Now it means selecting a probability that fits the mood and letting the audience do the rest. The person who once had to decide has been replaced by the person who merely has to sound like they decided.
The Room Where Nobody Is Accountable
Anthony Ha noted the strangeness of the word “we.” Anthony Ha, TechCrunch.” Who is the “we” that earnestly believes? Not the engineers who built the model. Not the customers who use it. Not the public that would bear the outcome. The “we” is a small group of people who share a workplace, a stock option schedule, and a vocabulary. It borrows the authority of a scientific consensus without ever assembling one.
This is the oldest trick in institutional language: convert a private opinion into a collective fact by changing the pronoun. Once the belief belongs to “we,” no individual has to defend it. Once no individual has to defend it, no individual can be wrong. The accountability that used to attach to a person making a claim dissolves into a group that never voted on anything.
The Courage That Looks Like Leaving
Coxon’s resignation deserves the credit it received, and it also exposes how low the bar has fallen. A person who says “I believe this is dangerous, so I am stopping” is described as brave. In most fields, that is simply called integrity. The fact that it reads as exceptional tells you how thoroughly the alternative has become normal: stay, warn, collect, repeat.
Leaving is a judgment made by a human being about their own life. It cannot be delegated to a model, a committee, or a probability. That is why it is the only move in this whole affair that carries information. Everything else is a statement. This was a decision.
The Outlook Is a Question, Not a Promise
The next necessary question is not whether AI can destroy humanity. It is who is allowed to decide what counts as evidence, and whether that decision will keep being made by people with a financial interest in the answer. As long as the warnings come from inside the labs, they will be shaped by the labs. The percentage will stay unfalsifiable, the “we” will stay unnamed, and the filing will stay vague.
What would change things is not a better forecast. It is a person with the authority to say no, and the independence to mean it. Until that role exists, the debate will keep producing heat, headlines, and resignations — and the decision that matters will keep being made by whoever is loudest, not whoever is right
Sources
1. Anthropic