Days after releasing its most advanced general model, Anthropic CEO Dario Amodei signaled a stark warning: AI is too dangerous to be left unregulated. In a long post titled "Policy on the AI Exponential," Amodei for the first time as the company's top leader called for government-backed, legally binding rules governing the release of "frontier AI models." He drew an analogy to commercial aviation: just as aircraft must pass FAA safety reviews, AI models should face similar scrutiny.
Three Pillars: Thresholds, Cyber, and Labor Displacement
The first proposal sets quantifiable thresholds for mandatory third-party testing: any model trained with computing power exceeding 10^25 FLOPs, or developed by a company with annual AI revenue over $500 million or AI R&D spending over $1 billion, must pass an independent audit before launch. The tests target four risk areas: cybersecurity, biological weapons, AI system loss of control, and automated R&D that could accelerate the first three. "Frontier AI models are like airplanes—they should be required to pass technical tests and audits," Amodei wrote. "If they don’t meet high safety standards, their release should be blocked or withdrawn as a public safety threat." Under this framework, governments would gain legal authority to block, delay, or discourage deployment.
The second proposal positions AI as a critical cybersecurity infrastructure issue. Amodei cited Anthropic's own Claude Mythos Preview model, which has already found high-severity vulnerabilities in major operating systems, demonstrating that offensive and defensive capabilities are advancing in lockstep. The framework requires frontier developers to protect "model weights" — the core parameters stored after training, whose theft would effectively copy the entire model — from external attackers or insiders, and to establish a legal channel for reporting "model distillation attacks."
The third proposal carries the most political weight: a frank acknowledgment of structural labor displacement. The economic policy framework states plainly that if AI reaches predicted capability levels, it will be a "full substitute for labor" rather than a productivity aid. It runs scenarios with unemployment rates of 5%, 10%, and even more extreme cases, advocating for wage insurance, universal basic income (UBI), and sovereign wealth fund models. To back these proposals, Anthropic announced $350 million in funding: $200 million for an "Economic Future Research Fund" to pilot public policies, and $150 million for a national grant program. Amodei closed: "The key challenge isn’t to incentivize growth—it’s to find a way for everyone to share in the benefits."
Why Now? The Logic and Limits of the FAA Analogy
The timing of this framework ties to two factors. On the technology front, Amodei argues that the risk profile was previously too unclear for precise legislation, but Claude Mythos Preview's ability to actively find system vulnerabilities means that argument no longer holds. Second, while the FAA analogy is rhetorically appealing, the "danger" of AI models remains highly dependent on the evaluator's judgment framework. The definition of "frontier" shifts every few months, and the credibility and independence of third-party testing bodies have yet to be established. Amodei acknowledges the framework is a starting point, not a finish line.
Clashing with Trump's Deregulation Agenda
The current U.S. policy direction is toward deregulation: the Trump administration advocates letting the AI industry "grow wildly" to outcompete China, even flattening state-level regulatory barriers. Amodei's call for tighter controls runs directly against that wind. He attempts to find bipartisan language near the end: "These policy ideas have cross-party, common-sense appeal. The sooner we act, the sooner everyone can share AI's benefits." Whether that "common-sense appeal" translates into legislative momentum, in the current Washington political environment, remains an open question.