Anthropic has formally started rolling out what it calls "embedded evaluations," with Faculty, an AI company under Accenture, becoming the first outside group to enter the company under the new setup, according to BlockBeats.
Faculty staff will work inside Anthropic to evaluate models, run red-team tests, conduct alignment reviews, and check safety measures. The arrangement goes well beyond a standard external review.
Evaluators will receive access close to that of employees
Under the model, evaluators will be given permissions close to employee level. That includes the ability to view the model training process, understand how development and deployment decisions are made, and communicate directly with internal staff.
That makes the program materially different from a typical outside assessment, where reviewers usually test systems from the perimeter rather than from inside the organization.
Anthropic and Accenture each plan to commit at least $1 billion
The two sides plan to invest at least $1 billion each over the next five years to build this evaluation capability.
Anthropic said the sector still does not have a unified set of rules for how such evaluations should work. Questions including what evaluators can see, how results should be made public, and who should cover the cost are still being worked out.
Commercial ties raise independence questions
For now, Anthropic is directly paying for Accenture's evaluation work. At the same time, Accenture is already a long-term strategic partner of Anthropic. It has also set up an Anthropic Business Group that has trained about 30,000 people to use Claude.
That point stood out because, on the same day, the AI Evaluator Forum said one of its minimum conditions is that an evaluation organization should not have major commercial dealings with the company it is assessing.
More evaluators may be added later
Anthropic plans to bring in other evaluation organizations after Faculty. It is also discussing pilot programs with nonprofit groups including METR, which BlockBeats described as another longtime Anthropic partner.