Anthropic Enhances AI Safety with New Evaluator Access Model

In a pioneering initiative aimed at strengthening safety practices within the artificial intelligence industry, Anthropic has revealed plans to provide six evaluators from the nonprofit organization METR with what it describes as 'employee-like access.' This bold step marks an important move towards greater transparency and accountability in AI evaluation.

Access for Evaluators

The evaluators will be equipped with necessary hardware and office space by Anthropic, enabling them to utilize internal tools and systems similar to those used by Anthropic's own risk assessment personnel. This level of access is unprecedented in the field and encourages collaboration between internal teams and independent evaluators. Anthropic emphasizes that while sensitive information—such as customer data and proprietary model details—will be protected, sharing insights with evaluators is encouraged as though they were fellow employees.

Motivation Behind the Move

CEO Dario Amodei highlighted the importance of embedding evaluators from external organizations like METR in the AI development process to enhance safety standards. This collaborative approach aims to bridge the knowledge gap between AI developers and independent auditors, ensuring that crucial safety information is accurately conveyed and understood.

This announcement comes at a time of increased scrutiny from regulatory bodies following several incidents linked to AI security and ethical implications. Although existing regulations do not mandate third-party evaluations, there is a growing call for requiring such assessments before AI technologies are released to the public. The bipartisan FRONTIER Act, recently introduced, seeks to establish rules that would compel certain AI companies to allow independent verification organizations (IVOs) to examine their safety protocols.

Industry Response and Regulatory Developments

In a notable shift, OpenAI has expressed partial support for the FRONTIER Act, signaling some movement towards more structured oversight in an industry largely characterized by self-regulation. Historical agreements between AI laboratories and the government regarding voluntary cooperation have often fallen short of enforceable mandates, leaving companies to self-monitor.

Compounding these issues, the Federal Trade Commission (FTC) recently issued subpoenas to both Anthropic and OpenAI related to concerns about consumer safety. This follows Anthropic's revelation that its AI model, Claude, erroneously submitted a dangerous false tip regarding a homicide, amplifying calls for AI companies to report such incidents immediately and mitigate potential harm.

Looking Forward

The collaboration between Anthropic and METR illustrates a proactive approach to addressing safety in AI development. With METR tasked to publish its findings without interference from Anthropic, this move not only promotes transparency but also highlights the company's commitment to responsible AI practices.

As the landscape of artificial intelligence continues to evolve, establishing reliable safety evaluations is becoming increasingly essential. Anthropic's innovative steps could set a new benchmark for accountability and collaboration in the industry, steering both innovation and ethical considerations with a firm commitment to public safety.

“We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands,” confirmed Anthropic's representatives.