Open source AI enables weaponized drone guidance
In 2022, a team of researchers at the University of California, Berkeley published a study that seemed to confirm everything the open-source movement had always believed: software with publicly visible code had fewer critical security flaws than its proprietary counterparts. [1] The logic was elegant—more eyes on the code meant more bugs found and fixed before they could be exploited. It was a victory for transparency, for collaboration, for the idea that a problem shared is a problem halved. But the study, published in the journal Empirical Software Engineering, measured something specific: conventional software vulnerabilities in conventional applications. It did not measure what happens when the same openness is applied to technology that can kill, deceive, or replace human judgment at scale.
The Guidance System Nobody Paid For
The most troubling example of this blind spot sits not in a laboratory but on the battlefield. Ukrainian engineers have taken open-source flight control software originally designed for consumer drones—the kind hobbyists use to film sunsets—and repurposed it to guide a cheap cruise missile. The software, called ArduPilot, was built by volunteers who wanted to make drone piloting accessible to anyone with a soldering iron and a weekend. It succeeded beyond their dreams. Today, it flies over Russian positions with a warhead attached. The irony is brutal: the same openness that made the software reliable enough for civilian use—thousands of volunteer testers, public bug trackers, freely available schematics—made it equally reliable for military use. A problem shared was indeed halved. It was also weaponized. As New Scientist reported in 2025, the Ukrainian military did not need to build a guidance system from scratch. They simply downloaded one. The guardrails that might have prevented this—export controls, end-user licenses, hardware restrictions—were never part of the open-source bargain. They could not be, because the bargain was built on trust.
The Guardrail That Was Never Installed
This same dynamic now governs the most consequential technology of the decade: artificial intelligence. In the United States, companies like OpenAI and Google maintain closed codebases. Their models are sold as services, not handed out as blueprints. This gives them a measure of control—not perfect control, but enough to install guardrails that prevent a chatbot from generating instructions for building a bomb or a recipe for synthesizing fentanyl. In China, the approach is different. Chinese AI models are largely open-source. They are generally a few months behind their American counterparts in raw capability, but they are free to download, modify, and run on any hardware powerful enough to handle them. The guardrails that exist in the American versions are thinner in the Chinese ones. And, as multiple security researchers have demonstrated, they are trivially removable. A teenager with a gaming PC can download an open-source model, strip its safety filters, and ask it anything. The model will answer. This is not a hypothetical risk. In 2024, a study by the Center for Security and Emerging Technology at Georgetown University found that open-source AI models were more likely to generate harmful content than their closed equivalents—not because the models themselves were more dangerous, but because the safety mechanisms were easier to bypass. [2] The problem was not in the code. It was in the distribution.
The Regulation That Cannot Catch Up
This creates a regulatory nightmare. In early 2025, Demis Hassabis, the CEO of Google DeepMind, proposed that AI models should be tested for national security risks before release. The idea sounds sensible. But it assumes a world where models are released by identifiable entities at identifiable moments. Open-source models are not. They are released by universities, by research collectives, by anonymous contributors on GitHub. They are forked, modified, and redistributed faster than any regulator can track. By the time a safety test is completed, a hundred derivative versions are already circulating. The test becomes irrelevant. The problem is not that regulation is impossible. It is that the metaphor of regulation—a gate that can be locked, a door that can be guarded—does not apply to software that can be copied infinitely at zero marginal cost. The open-source movement did not create this problem. It inherited it from the nature of information itself. But it refused to acknowledge it. The idea that openness is always a net good was never questioned, because questioning it would have undermined the entire philosophy. The philosophy was never designed for a world where a guidance system for a cruise missile and a chatbot capable of psychological manipulation could be built from the same freely available code.
The Variable That Remains Unresolved
The final open variable is not technical. It is cultural. The open-source movement has always prided itself on its democratic ethos: anyone can contribute, anyone can benefit, anyone can improve the code. This ethos worked beautifully for operating systems and web servers. It works catastrophically for technologies that have no inherent safety ceiling. A bug in a web server crashes a website. A bug in an AI model manipulates a voter. A bug in a drone guidance system kills a soldier. The same openness that makes the first problem solvable makes the last two problems unmanageable. The movement has no answer to this. Its members continue to believe that transparency is a universal good, that more eyes will always make the code better, that the solution to every problem caused by openness is more openness. This belief is now being tested in real time. The outcome depends on whether the open-source community can develop a new ethic—one that distinguishes between code that should be shared and code that should not. That distinction does not exist yet. It is the last variable. And it will determine whether the next decade of open-source technology is remembered as a triumph of collaboration or a failure of responsibility.
Sources
1. Empirical Software Engineering
2. Center for Security and Emerging Technology at Georgetown University
4. GitHub