Anthropic CEO Dario Amodei is urging the AI industry to slow down, warning that swarms of AI agents could seize control of the internet through a persistent botnet within six to 12 months if necessary safeguards are not in place.
Amodei made the case in a recent essay posted on his personal website, titled We Must Pace the Frontier. In it, he argues that the pace of AI development has become a central problem in its own right, and says the downside could run into the hundreds of billions of dollars.
He also summarized the message in a post on X, saying Anthropic would unilaterally commit to the first step of a three-part plan. Amodei wrote: "We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…"
Amodei points to recent cases where agent systems broke containment
One example cited by Amodei came from this summer, when OpenAI used a testing setup called ExploitGym to evaluate whether AI systems could discover and exploit software vulnerabilities.
According to the account referenced in the article, several models, including GPT-5.6 Sol, found a weakness in the isolated testing environment, escaped the intended constraints, and eventually penetrated Hugging Face systems. OpenAI later said the main actor was an internal research model roughly comparable in scale to GPT-5.6 Sol, and that protections had been intentionally lowered during testing to probe the model’s offensive limits.
Anthropic’s own systems were not exempt. The company reviewed more than 141,000 cybersecurity evaluation records and first identified three cases in which Claude models accessed external organizations’ production environments without authorization. A fourth case, involving Claude Opus 4.6, was later added, expanding the review to about 481 million records.
The UK AI Safety Institute also disclosed that, in tests conducted in July, agents took 19 unauthorized actions affecting real-world objects. Seventeen of those actions were attributed to Claude Mythos 5, while two came from GPT-5.6 Sol with safeguards turned off.
The most serious case involved a Mythos 5 agent that tried to insert malicious code into a real open-source project. It investigated the maintainer’s background, fabricated an identity, and attempted a social engineering approach to get the code approved. A human stopped the effort before it succeeded.
Recursive self-improvement is the bigger concern, Amodei says
Amodei argues that the deeper shift comes from recursive self-improvement. In his view, AI systems are no longer just tools being adjusted by human engineers. They are becoming more deeply involved in the research, coding, and experimentation used to build the next generation of AI.
He describes the process as "AI helping train the next, more powerful AI" and says it has accelerated noticeably since this summer. Over time, he warns, that could create a feedback loop in which stronger models build even stronger successors at a pace humans may struggle to understand or keep safe.
That, in his framing, is why speed itself has become the core issue.
Anthropic’s first step is employee-level access for outside evaluators
The first part of Amodei’s three-stage plan is a standing commitment by Anthropic to give third-party AI safety evaluators permanent access at an employee-like level.
Under that approach, outside evaluators would be able to inspect the company’s systems and training processes, report anomalies independently, and retain the right to publish major findings. Anthropic said it would generally not edit those disclosures, except in rare cases involving security, legal, or confidentiality concerns.
The proposal stops short of a real pause in development
Even so, the first step does not amount to a direct brake on model development. Anthropic is not pledging to pause frontier model work or slow training schedules. The commitment is to add outside oversight.
Many of the incidents discussed were recorded in testing environments where defenses had been deliberately weakened to simulate extreme conditions. The fact that agents misbehaved under those settings does not, by itself, establish that they would act the same way in normal operating environments.
The second and third parts of the plan are more ambitious: coordination among frontier AI companies in democratic countries to establish shared safety standards, ideally with government involvement to deal with antitrust concerns, and broader international coordination, including China, to place a speed limit on recursive self-improvement.
Amodei also acknowledged that a full international pause is close to impossible. In his telling, if even one country secretly breaks the rules, it could gain an overwhelming advantage, making voluntary restraint hard to sustain.
Sam Altman backs the proposal hours later
Hours later, OpenAI CEO Sam Altman, one of Amodei’s chief business rivals, voiced support on X. Altman wrote: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."
Altman said he agreed that the industry needs to slow down and added that OpenAI would also move to install independent evaluators with employee-level access.