Will embedded independent evaluators become a norm for frontier AI companies?
What it could establish
That outside evaluators get inside access and publication rights as a condition of trust, rather than case-by-case.
What would settle it
Other labs adopting comparable terms, the first published findings showing real access, or a binding requirement.
The other reading
Voluntary access controlled by the company being evaluated may not amount to independent oversight, especially given close ties between labs and evaluators.
Status history
Anthropic and METR arrangement reported (D-017).
Every change is dated. Earlier assessments stay visible.
The developments behind it
Anthropic gives six staff from METR, a Berkeley nonprofit, employee-like access to its models and tools. Anthropic cannot veto their reports but can redact protected information. It is also working with SecureBio and Accenture.
Reporting- Who's watching the AI companies? Handpicked Berkeley safety groups The San Francisco Standard
Anthropic's CEO publishes an essay calling on every frontier AI company to give embedded third-party evaluators ongoing, employee-like access.
Reporting- AI embedded evaluator jobs The New Stack