Anthropic Grants METR Employee-Like Access for Independent Model Evaluation

Anthropic has granted METR, a nonprofit evaluating frontier AI models, employee-like access to its internal operations. This move follows the company's commitment to stronger third-party assessment and its signing of the White House Accord on Super Intelligence. Six METR staff members have been onboarded with company hardware, office space, and access to internal tools roughly on par with Anthropic’s own risk assessment teams.

Access and Publication Rights

The agreement allows METR to publish its analysis of risk levels, incidents, and practices without Anthropic’s veto. While Anthropic can require redactions for narrow categories like customer data, attorney-client privilege, or specific model architecture details, METR retains the right to publicly characterize these redactions and state whether they believe them justified. This arrangement aims to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself knows.

Tiered Evaluation Framework

Anthropic is establishing three tiers of access for independent evaluators to balance transparency with security:

• Tier 1: Evaluators conduct narrowly scoped investigations with access decided by Anthropic; publication requires Anthropic’s approval. SecureBio is currently in this tier, reviewing the chemical and biological weapons chapter of the August Risk Report.
• Tier 2: Evaluators can publish without approval and have access to more information. Anthropic can still require removal of specific information categories, but evaluators can comment on the justification of redactions.
• Tier 3: The current METR arrangement, featuring employee-like access and publication rights similar to Tier 2.

Anthropic is also working with Accenture and Faculty on an ambitious partnership involving model red-teaming and alignment assessments, though these engagements are still in early stages with more limited access. The company emphasizes that embedded evaluation is voluntary and believes every frontier AI company should adopt similar practices.