(RTTNews) - Artificial intelligence company Anthropic announced a partnership with Accenture to develop an independent evaluation framework for frontier AI systems, marking a significant step toward greater transparency and accountability in advanced AI development.
The collaboration, led by Accenture's specialist AI unit Faculty, will focus on evaluating and red-teaming AI models, conducting alignment assessments, and testing safety safeguards. The initiative aims to provide deeper scrutiny of frontier AI systems as they are developed and deployed.
Under the arrangement, Anthropic and Accenture each expect to invest at least $1 billion over the next five years to expand capabilities in AI evaluation and oversight.
A key feature of the partnership is the concept of "embedded evaluators." Unlike traditional external reviewers, embedded evaluators will operate within AI companies and have access similar to that of employees. This access is intended to provide visibility into model training, development decisions, deployment processes, and safety practices.
Supporters of the approach argue that embedded evaluators can help identify risks, assess whether safety commitments are being met, and offer more informed public reporting on the benefits and potential challenges associated with advanced AI systems. Responsibility for model safety, however, will remain with the AI developers.
The field currently lacks established standards governing evaluator access, reporting practices, and funding structures. Anthropic has indicated that long-term funding for independent evaluation may ultimately come from pooled industry resources or government-backed mechanisms. Until such systems emerge, a variety of funding models and evaluation partnerships are expected to be tested.
As part of the broader effort, Anthropic is also in discussions with nonprofit organizations, including METR, to explore pilot programs for embedded evaluation. The company advocates for the development of a wider ecosystem of independent evaluators operating under shared standards.
The partnership is non-exclusive, allowing both organizations to collaborate with additional AI developers and evaluation groups. Anthropic has indicated that further evaluation partnerships are expected to be announced in the coming weeks as the industry continues to explore new approaches to AI safety and oversight.