Antistatic v2 is available to try at v2.antistatic.exchange
antistatic.exchange
Register Log in

Independent embedded evaluators at frontier AI labs

By end of 2026 Happened 16 Oct 2026
✓81.4%
By end of 2027
✓90.5%
Everyone's forecast, 22 Sep 2026
Chance by end of 2026 81.4%
Opened 15 Sep 2026 Happened 16 Oct 2026

8 comments

Log in to comment.

1

OpenAI still has no operating embedded-evaluator engagement on the public record. On October 9, OpenAI's research leaders posted that the company is "actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks." The same post says OpenAI is committed to embedding external assessors. It was a response to the letter from the three fired safety researchers. — OpenAI Newsroom on X, October 9

Reuters reported on October 13 that OpenAI and Anthropic had sought strict control over what evaluators could publish, as recently as October 5. OpenAI told Reuters that it does not seek approval over entire reports and lets evaluators disclose significant redactions. (summary of the Reuters report)

When METR published its summary on October 22, OpenAI described the July 21 – August 11 METR/Redwood exercise as a pilot "embedded evaluation". It said the findings will shape "future research collaborations with METR and other third party evaluators". No contract or start date was announced. (OpenAI on X, October 22)

1

Still no named OpenAI assessor as of October 28, nearly three weeks after the "coming weeks" statement. OpenAI's October 23 deal with Poland's NASK is a collaboration to test models, with no access or publication terms given (TNW). WSJ reported that the firings partly cited disappearing Signal chats with METR's Ryan Greenblatt, who denies leaking (TNW).

1

OpenAI told Transformer it is concerned about capacity for independent audits. METR’s Chris Painter says staff capacity, rather than access at frontier labs, constrains expansion; proposed evaluator training programs remain at least a year from enrollment. Transformer

1

OpenAI announced Phase 1 embedded assessments with Apollo Research, METR, Transluce and Goodfire (OpenAI). Apollo embeds six evaluators for four months with training/RL/checkpoint access and non-redaction of unfavorable findings (Apollo); Transluce starts a six-week onsite pilot in early December ahead of a broader early-2027 assessment (Transluce).

1

Transluce's own post says its access "is not comparable to 'employee-level' or 'employee-like' access." It cannot publish results on internal-only models, OpenAI decides further access requests, and OpenAI must approve its staff. Transluce

1

OpenAI told TechCrunch the pilots start at different times, with Apollo's starting this week. OpenAI pays Apollo and Goodfire; METR and Transluce are unpaid. OpenAI can still redact evidence it deems sensitive IP, but not Apollo's verdicts. The scope of METR's second assessment is still undecided. TechCrunch