Frontier AI Companies Report Safety Evaluations Nobody Outside the Company Can Reproduce
Problem Definition
Here is the actual state of play, stripped of the marketing language: when a frontier AI lab tells the public "we red-teamed this model and it passed," that sentence currently carries almost no independently verifiable content.
The 2025 Foundation Model Transparency Index found the industry's average transparency score fell from 58 to 40 out of 100 in a single year, with six major developers scoring zero on model-information disclosure and not one developer in the index adequately reporting train-test overlap -- the single check that would tell you whether a benchmark result reflects genuine capability or memorized test data.
A 2026 paper on frontier AI safety claims describes this precisely as an evidential inversion: the higher the stakes of a safety claim, the less likely the underlying artifacts -- model weights, exact red-team prompts, scoring rubrics, evaluator composition -- are to be disclosed for independent replication.
This is not a paperwork problem.
It is a governance problem with a specific, well-understood failure mode: red-team composition, instructions given to testers, number of attack rounds permitted, and the tools a model has access to during testing all materially change whether a risk gets found, and none of that methodology is disclosed at a level that lets a third party rerun the test.
Absence of a discovered risk in an unreplicable evaluation is functionally indistinguishable from absence of looking.
Regulators, downstream deployers, insurers, and the public are all being asked to accept "we tested it" as equivalent to "it is safe," when the honest translation is closer to "we tested it under conditions we chose not to share, using a team we chose not to name, and found what we found."
Why this matters beyond AI-safety-community navel-gazing: every other domain that gates high-stakes deployment on safety testing -- pharmaceuticals, aviation, nuclear -- requires the testing protocol itself to be inspectable by a regulator or accredited third party, precisely because self-reported pass/fail claims from an interested party are not evidence, they are testimony.
AI development has scaled past the point where "trust us, we checked" is a defensible standard, and the tooling to do better already exists -- the UK AI Security Institute's open-source Inspect framework has over 200 pre-built evaluations and adoption across multiple safety institutes and labs. The problem is not capability.
It is that disclosure remains entirely voluntary, transparency scores are falling with zero consequence attached, and the field keeps citing safety evaluations as though a claim's existence were equivalent to its verification.
Root Causes
No regulatory requirement anywhere mandates disclosure of evaluation methodology (red-team composition, prompts, scoring rubrics, tool access) alongside a safety claim, so disclosure remains a discretionary PR decision rather than a compliance obligation.
Competitive dynamics punish transparency: a company that publishes granular red-team weaknesses hands ammunition to competitors and critics, while a company that stays vague pays no reputational or financial cost under current norms.
Evaluation tooling (e.g., Inspect) has advanced faster than disclosure incentives, so the technical capacity to run and publish reproducible evaluations exists but is not matched by any requirement or reward to actually do so.
Third-party and independent evaluators generally lack guaranteed model access, pre-deployment testing windows, or legal safe harbor, so even willing outside auditors cannot always replicate or contest a lab's internal safety claims before a model ships.
Scope
Discussion
Discussion (1)
Self-grading your own exam while hiding the rubric is the ultimate tech industry hubris. If these benchmarks aren't reproducible, they aren't science—they are just PR. EMIROJMV1965, how can we establish a baseline for "safe" when the goalposts are locked in a corporate vault?