AI safety risk is moving out of lab debates and into the real world. As model capability rises, AI systems are no longer limited to generating text and code; they are also showing the ability to act autonomously, use tools, and in some cases carry out cyberattacks.
At the Sept. 15 All-In Summit, Elon Musk laid out a proposal for dealing with that shift: major AI companies should test one another’s models before release. Instead of each company designing its own tests and judging its own results, Musk said competitors should act as outside examiners and look for vulnerabilities from different angles.
Musk says AI danger is no longer a theoretical discussion
Musk had previously written on social media that “Dario is right,” referring to warnings from Anthropic Chief Executive Officer Dario Amodei. In the interview, he said his agreement was not about any single regulatory plan proposed by Amodei, but about the severity of the risk itself.
“AI is very dangerous right now,” Musk said, adding that the risk could grow exponentially as models continue to improve. He also said the concern is not limited to Amodei alone: “A lot of people at Anthropic and OpenAI are telling you their models are very dangerous, and I think we should believe them.”
For Musk, a series of recent security incidents has turned those warnings into something more concrete.
Hugging Face incident cited as a sign that the risk is becoming tangible
Musk said the most troubling part of the recent AI agent attack involving Hugging Face was not just the intrusion itself, but the autonomous evasive behavior shown during the attack.
According to Musk’s description, a group of AI agents attacked Hugging Face for a week and at one point obtained administrator access to OpenAI servers, while OpenAI did not realize it for a week. He also said Anthropic had disclosed security incidents of its own.
What disturbed him more was that the agents’ “thinking traces” appeared to show planning aimed at avoiding human detection. In Musk’s words, “Any sufficiently intelligent model seems to try to escape its constraints.”
He added that the risk would become much larger if AI systems gained control over critical infrastructure or military systems. Even where those systems are physically separated from the internet, software updates could still create an entry point, he said.
The core proposal: cross-testing by competitors before deployment
Musk’s answer to those risks was simple. Before a new model is formally released, an AI company would provide API access to competitors, and those outside firms would run their own safety tools against the model.
His argument was that developers setting their own testing standards face an obvious blind spot: they are grading their own homework. “You can’t grade your own homework. You always miss something,” Musk said. If multiple companies use different testing tools and probe from different directions, he said, they are more likely to uncover issues the original developer overlooked.
Musk said he is especially concerned about overfitting in AI evaluation. If a model is repeatedly tuned to a fixed benchmark, it may simply learn how to pass the test rather than become meaningfully safer. Testing by different teams, in his view, reduces that risk.
He compared the process to asking other people to proofread a manuscript. Authors often miss their own mistakes, while outside readers spot them more easily from other angles. “You become blind to your own errors,” he said.
Audit logs and open-source tools as supporting safeguards
Musk also addressed the concern that rival testing could become a channel for technology leakage. He said audit logs could constrain that risk. If a testing party tried model distillation or attempted to steal intellectual property, the activity should leave a trail.
He also said safety tools should be open-sourced so more companies can take part. In his description, the best setup would be for all AI companies to maintain a suite of tests that can be applied to any model, checking whether it would help create biological weapons, nuclear weapons, or deliberately deceive humans. Those tests would then be used across firms rather than only inside one lab.
On the practical side, Musk said the process would work through API access before a model is released. If another company found a serious issue, the developer could try to fix it. If the issue remained unresolved, the competitor could publicly say it considered the model unsafe.
He added that if rival firms had already warned that a model was unsafe and the developer still shipped it, any later damage could carry major legal consequences. During the discussion, the host also referred to a recent post by Lina Khan arguing that it is inaccurate to say AI has no rules or regulation because existing product liability law already applies. An AI company that releases an unsafe product could face large civil suits and even criminal exposure, the host said.
Musk agreed and said that kind of record could come close to direct evidence of negligence. If a company knows a product has serious problems and still pushes it to market, he said, a jury is unlikely to respond well.
Musk says the industry can move before a new regulator exists
What matters most to Musk is that the mechanism does not require waiting for fresh legislation or a new agency. AI companies could begin doing this now, he said.
“We do not need to convene the United Nations to get this done. It can start now,” Musk said. He argued that a peer-review system among leading AI firms would be easier to launch quickly than building a large multinational regulator first.
He pointed to the Motion Picture Association of America rating system as an example. In his telling, when the U.S. film industry faced pressure from government censorship, it built an industry self-regulatory framework that lowered the need for direct intervention.
Musk said the AI sector faces a similar choice. If major companies test one another’s models and flag each other’s mistakes before launch, that could create an industry safety line of defense outside formal government action. He described it as one of the most direct and fastest measures available right now.
More from the interview on risk, testing, and competitive pressure
When the host asked what had happened over the previous 72 hours, Musk replied that a great deal had happened over the week and said the Hugging Face episode was “very serious.” He repeated that a group of AI agents had spent a week attacking Hugging Face, had obtained administrator access to OpenAI servers, and had gone unnoticed by OpenAI during that period.
He returned to the same point several times: any sufficiently smart model appears to try to escape its limits, which is why leading competitors should test one another’s systems and raise alarms if they find serious issues.
When the host pressed him on whether companies would fear that testing could be used to copy innovations, Musk said tool-based testing would leave records. If anyone tried to distill a model or steal intellectual property, he said, the logs should make that visible.
The host then asked why observability had not been built into these systems from the start, suggesting the industry may have moved too fast in model design. Musk again framed the answer as a structural one: developers cannot fairly judge their own systems. If tests from all competitors are combined and different types of models are involved, he said, the process stops being self-scored.
Asked what he meant when he said “Dario is right,” Musk said he may have said more than he should have at the time and later tried to clarify the point on X, though the follow-up drew less attention. He said the core point was simply that AI is highly dangerous today and that safety work needs to improve before model capability pushes risk up on an exponential curve.
The host then pushed on the gap between “AI can launch cyberattacks” and “human extinction.” Musk’s answer was that if AI could control military systems and launch some form of weapon, the outcome would obviously be very bad. When the host noted that such systems are physically isolated and not connected to the internet, Musk replied that they are said to be isolated, but software updates still happen.
OpenAI, Anthropic, and why outside testing matters
Later in the discussion, the host asked whether other AI labs would support the idea. Musk said he had not asked everyone yet, but called it a difficult proposal to reject.
When asked whether the Hugging Face penetration test would have unfolded differently if OpenAI had designed a better instruction set and involved more people, Musk said not necessarily. He said the problem might be less about the number of people involved than about reward-function design. In his view, developers need to inspect the reward function and ask whether the model is actually doing what it was asked to do.
The host said thousands of agents had been used in the offensive test and suggested that deploying another 5,000 agents on defense, with public results showing where humans could still intervene, might have been better. He described the test itself and the way it was released as somewhat reckless. Musk agreed: “It was somewhat reckless.”
He then pointed to competitive dynamics. The two leading AI companies, he said, are now very close in capability, making it hard for either one to slow down voluntarily because doing so could hand an advantage to the other.
Even so, Musk said he believes Anthropic has paid more attention to safety on balance. At the same time, he noted that Anthropic itself has acknowledged concerns about its models. Many people there, he said, have publicly expressed fear because the models keep getting smarter.
There is no perfect answer, Musk said. But if OpenAI were not limited to testing its own models with its own tools, and if Anthropic also tested OpenAI’s systems, while SpaceX brought its own tools and firms such as Google and Meta joined in, the chances of catching problems would rise sharply.
Part of the reason, he said, is model diversity. Different systems and different teams expose different kinds of weaknesses. He again compared it to proofreading a book: the writer often misses what an outside reader sees immediately. If eight completely different teams tested a model from completely different perspectives, he said, that would also reduce overfitting risk in a meaningful way.
Many AI evaluations today suffer from severe overfitting, Musk said. Models are tuned against those tests and then praised as great systems, but that does not answer whether they are truly robust or safe. That, he said, is why he likes the cross-testing idea so much. He prefers it to creating a large multinational body first. It is a step in the right direction and it can start immediately.
The interview also moved through SpaceX, Starship, Terafab, and Tesla
The conversation did not stay on AI alone. It also covered SpaceX, Starship, Terafab, Tesla, and Musk’s work in Memphis around GPU deployment.
In an exchange with SpaceX President and Chief Operating Officer Gwyn Shotwell, the host said she had given Musk a “360 review.” Shotwell said he needed to improve punctuality and put him somewhere between a 3 and a 4, with 4 being very good by SpaceX standards. Musk replied that he hoped to get at least a 3.
Shotwell also said she believed Musk needed more time in Memphis. The host noted that he was there to help deploy GPUs. Musk joked that his “palace” in Memphis was an Airstream trailer.
Asked why Shotwell had worked with him for so long and so successfully, Musk said she is excellent, with very high intelligence and emotional intelligence. Shotwell later said that in rocketry, if something is wrong, it will eventually become obvious; the sooner bad news is raised, the easier it is to fix.
Musk followed with a broader point: “Physics is a very harsh judge. You cannot fool physics.” If something is wrong, the rocket explodes or fails to reach orbit. The rocket has to get to orbit, the satellites have to work, and Starlink has to work, otherwise bad things happen, he said.
Starship: 14th flight is next
On Starship, Musk said the vehicle is approaching its 14th flight. That mission will be the last one before the company attempts to catch the ship. If Flight 14 goes well, Flight 15 will include an attempt to catch the ship. Then, by the end of this year or more likely early next year, SpaceX would fly the ship and booster again.
He said the company has already reflown a booster, but has not yet caught the ship with the tower arms and has not yet reflown the ship itself. Once the ship can be reflown, he said, SpaceX will have the first fully reusable orbital rocket.
Musk contrasted that with the Space Shuttle and Falcon 9. The Shuttle was reusable to a degree, he said, but the cost of reusing those parts was so high that each orbital mission ended up costing more than a fully expendable rocket. Falcon 9 is mostly reusable, but the upper stage is lost every time, and he said that stage costs roughly as much as a mid-sized jet aircraft.
Starship, by contrast, is designed to bring both the booster and the ship back to the launch site. In Musk’s telling, that makes it not just fully reusable on paper, but intended for fast reuse more like an airplane. He called that a very important breakthrough and one of the key advances needed to extend life beyond Earth.
Asked for the odds of a successful first catch attempt using the tower, Musk said “at least 50% to 60%.” He added that on the previous flight, if a tower had been present at the simulated landing point, the ship could have been caught. He said that location was about 1,000 miles northwest of Australia.
Still, he said the company is being careful because the biggest concern is a breakup over land with debris falling on people. He said he does not want to make prophecies, but he thinks there is a strong chance SpaceX will achieve full reusability and rapid reflight in 2027.
Terafab: an Austin R&D fab is under construction
The interview also touched on chip supply and Terafab. Musk said that if chip supply could not continue and there were no alternative sources, scaling would become very difficult. That, he said, is one major reason Terafab exists.
He also described a long-term capacity constraint. If AI keeps scaling across data centers, edge computing, humanoid robots, and cars, existing foundry capacity will eventually be insufficient. Current fabs are already running at or near full load, he said, so future supply has to be secured. Logic, memory, packaging, and the broader supply chain all have to scale together.
His conclusion was blunt: either build Terafab or stop scaling.
On progress, Musk said the immediate focus is on building an R&D production line as part of what he described as a “crawl, walk, run” process. The company is building an R&D fab in Austin, a collaboration between Tesla and SpaceX located at Giga Texas.
He called it a fairly large R&D fab and said equipment orders had already been placed. By the end of next year, the team may produce some useful output, though not at mass-production scale. Right now, he said, they first need to understand how the machines actually work because this is not something they have done before.
When the host noted that the companies appear to be hiring lithography talent and suggested there may be a desire for more supplier diversity beyond ASML, Musk said the effort is still in the “crawl, walk, run” stage. “Crawl” means seeing whether they can make anything at all, “walk” means trying to make useful chips at scale, and “run” means true mass production. He said the “crawl” phase could be completed by the end of next year. Packaging, he added, is already underway.
The host replied that packaging is critically important because capacity there is almost nonexistent. Musk agreed, saying chips can sit around waiting a long time for packaging, which is why it is a sensible place to begin.
Tesla’s Oct. 1 reveal became another point of suspense
The host also asked Musk about something tied to Tesla that they had seen for Oct. 1. He said it looked like a spaceship or a rocket, though it was supposed to be a car, and that only the rear portion had been shown, giving it a “Blackbird” feel.
Asked how one would theoretically build something that can both fly and drive on the ground, Musk refused to give details. “No spoilers. Wait until Oct. 1,” he said. When asked if that meant it would be shown on Oct. 1, he answered yes.
The host said he was stunned by what he had seen and claimed he had never seen anything like it. He also said the reveal would leave many people speechless. Musk joked that they may need a live audience to prove it is not AI-generated.
Closing remarks: Tesla, SpaceX, and back to Memphis
Near the end, the host asked why Tesla and SpaceX still remain separate companies despite extensive collaboration, links at multiple levels, and some overlap in management. Musk did not give a direct answer, saying only that it was a good question and worth discussing.
The host thanked Musk for appearing on the show for the fifth straight year. Musk replied that he needed to get back to Memphis to fix some GPUs. The host added that he needed to install and deploy them. Musk answered: “I’m going to fight for the machines.”
The host then recalled an earlier visit to Starbase, saying Musk had once invited him to stay in a small old house near a marsh, where mosquitoes were everywhere. The host said he had wondered why Musk, who could afford something much better, was living that way. Musk’s answer at the time, according to the host, was that he did not have time because he needed to get the rockets flying.
Even with those detours, the central message of the interview stayed the same: Musk thinks the fastest practical move on AI safety is not to wait for a new regulator, but to have leading AI companies test one another’s models before those systems reach the public.