Weak AI Regulation Creates False Sense of Safety
Imagine a hiring algorithm that screens thousands of job applicants every hour. The company that deploys it promises it finds the most qualified candidates without human bias. In reality, the model learned to penalize resumes that included words like “women’s soccer” or “maternity leave.” The automated system deepens inequality while everyone believes it is making hiring fairer. That is the quiet danger of artificial intelligence: it deceives us not by lying outright, but by performing poorly under a convincing veneer of objectivity. Now a new study warns that weak regulation can make this deception even worse — by creating a false sense of safety that encourages companies to do less than they should.
The Illusion of Protection
Regulators around the world are rushing to write rules for artificial intelligence. But according to research published in the Proceedings of the National Academy of Sciences by scholars from Cornell University and Carnegie Mellon University Study, the mere presence of regulation does not guarantee safer systems In fact, the opposite can happen. The study used theoretical economics and game theory to model how companies change their behavior when rules are introduced. Its central finding, reported by PYMNTS, is that poorly targeted or insufficiently stringent regulation can create incentives for firms to reduce their own safety investments and shift responsibility to others This is not a failure of regulation — it is a predictable outcome of how actors respond to incentives.
Consider a medical diagnostics tool that relies on a general-purpose language model. If a state government requires strict testing only for the final application, the model provider may feel less pressure to audit its own training data. The downstream company, burdened with compliance, might cut corners to stay profitable. The result is a system that appears regulated but is actually less safe than if no rules existed at all. The deception lies in the label: “regulated” does not mean “safe.”
The Free-Riding Problem
The researchers draw a critical distinction between two points in the AI supply chain where regulation can be applied. The first is at the level of model developers — companies like OpenAI, Google, and Anthropic that build general-purpose systems. The second is at the level of downstream deployers — businesses that adapt those models for specific uses such as customer service, medical diagnosis, or credit scoring. Many policymakers assume that regulating applications is the most logical approach, because risks become visible only when AI is used in real-world settings. But the study shows that this assumption is dangerous.
When regulators focus primarily on downstream users, model developers can effectively free-ride on those compliance efforts. Principal author Benjamin Laufer calls it a free-riding behavior: the general provider uses regulation as a tool to offload the safety burden onto the downstream specialist. [1] A model provider might skip third-party safety audits, reasoning that the deploying company will be forced to catch any problems. The downstream company, meanwhile, expects the model to have been vetted upstream. Both parties assume the other will handle safety, and neither does enough. The regulation that was meant to protect becomes a mechanism for the diffusion of responsibility.
A Pattern Repeating Itself
This dynamic is not unique to artificial intelligence. History offers several examples where weak regulation created a deceptive sense of security. In the 2008 financial crisis, complex derivatives were lightly regulated because each transaction seemed safe in isolation. Regulators assumed that banks and rating agencies would police themselves, and the banks assumed regulators would catch systemic risks. The eventual collapse revealed that everyone had relied on someone else. Similarly, early safety rules for self-driving cars focused on vehicle certification while ignoring the infrastructure and data pipelines that allow cars to perceive the world. The result was a patchwork of standards that encouraged manufacturers to test in states with the weakest requirements.
The AI industry today risks the same pattern. At the federal level in the United States, policymakers have concentrated on frontier model developers and questions of model testing and national security. States, by contrast, have focused on downstream applications like employment and healthcare. Neither layer accounts for how obligations imposed at one point alter investments at the other. The study argues that this fragmented approach can produce worse outcomes than doing nothing, because it gives companies a plausible excuse for underinvestment.
The Prisoner’s Dilemma of Safety Investment
The researchers frame the problem as a classic prisoner’s dilemma. Without confidence that other participants will invest adequately in safety, each firm has an incentive to protect its own bottom line by spending less and relying on others. The result is collectively worse for everyone, even though cooperation would benefit all players. In game theory, this is a well-known trap: rational individual choices lead to an inferior group outcome. Weak regulation does not solve the trap — it simply makes the players believe they have already cooperated.
Strong, well-placed regulation changes the game entirely. When regulators require sufficient safety investment from both model developers and downstream companies, the uncertainty disappears. Firms no longer need to guess whether others will do their part. The study’s model shows that such regulation can produce benefits for both safety and economic returns. End products become safer, and companies derive more utility from their investments because they know the whole chain is secure. Utility in this context means a company’s share of revenue minus its investment costs. Everyone gains when the rules are clear and enforced across the supply chain.
The Deception of Choice
Public debate often frames AI regulation as a trade-off between safety and innovation. Supporters of strong safeguards argue that commercial incentives alone cannot address systemic risks. Critics warn that excessive rules will stifle competitiveness, especially against China. The study complicates this binary. The choice is not simply regulation versus deregulation. The design and placement of regulation matter as much as its stringency. A weak rule concentrated on one part of the chain can be more harmful than no rule at all, because it creates a false confidence that the problem is solved.
Laufer said that people think of AI as a single object, but it actually involves a complicated set of stakeholders with their own contributions. [2] To regulate thoughtfully, we need to consider the whole supply chain, not just a single provider or entity. That insight cuts against the current American approach. Federal rules target developers; state rules target applications. Neither side sees the full picture. The deception is that anyone is in control.
The Next Necessary Question
If weak regulation can make AI less safe, then the next question is not whether to regulate, but how to design rules that align incentives across the entire system. That means imposing obligations on model developers and downstream users simultaneously, and ensuring that neither can shift the burden to the other. It also means recognizing that safety is not a single checkbox but a continuous process of auditing, testing, and updating — something that no regulation can automate.
The study offers a warning but not a promise. It does not claim that strong regulation will solve all AI risks, only that the current fragmented approach may be making things worse. The real deception of weak AI rules is that they allow us to believe we have addressed the problem while companies quietly offload responsibility onto one another. Until regulators account for the full supply chain, every new regulation runs the risk of being a mirage — a reassuring shape on the horizon that vanishes as soon as you try to touch it.
Sources
3. Google
4. Anthropic