Anthropic said it has chosen Accenture as its first embedded evaluator, taking a first concrete step toward the AI slowdown proposal released last week by CEO Dario Amodei. The partnership is non-exclusive, and Anthropic said it expects to announce other evaluators in the coming weeks that it will also work with.
Amodei published a three-step proposal on Sept. 12 calling for AI development to slow so safeguards can be put in place, after warnings that rapid and unchecked progress could lead to catastrophic harm.
The proposal drew a positive response from OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Nvidia CEO Jensen Huang took a different view, arguing that this kind of regulation was not necessary.
First step focuses on independent evaluators
In the proposal, Amodei wrote, 「AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement,」 and warned that 「left unchecked, it could outrun our ability to understand and control these systems.」
He said the first step is to use independent evaluators with employee-like access, something Anthropic had already unilaterally committed to.
On Friday, Anthropic announced a move toward that commitment by partnering with Accenture and its AI business Faculty in 「evaluating and red-teaming models, conducting alignment assessments and testing model safeguards.」
The company said the details of how embedded evaluation will work are still being developed, because the practice is new.
Each side expects to put in at least $1 billion over five years
According to the announcement, Anthropic and Accenture each expect to invest at least $1 billion in the project over the next five years.
Anthropic said there is currently no existing system for funding independent evaluation. It added that long-term funding should come from pooled or government sources, but because of the urgency, it will directly fund Accenture’s work for now.
Anthropic points to Faculty’s model testing background
Anthropic said Friday that Accenture’s Faculty has expertise in testing and evaluating models for some of the world’s leading AI labs and in building complex AI systems that are safe and ethical by design.
Julie Sweet, chair and CEO of Accenture, said: 「Embedded evaluation is an emerging area, and we look forward to partnering with Anthropic to help accelerate the development of embedded evaluators, which we see as an important part of the safety landscape going forward.」