METR spent 11 weeks embedded inside Anthropic. Model iteration speed is now roughly twice what it was a year ago, and employees self-report a 12x productivity gain. The acceleration stems mainly from AI rather than from stacking more compute—but the public version of the report blacked out most of the key evidence.

On November 5, AI safety research organization METR released an evaluation report on Anthropic following an 11-week investigation conducted inside the company. Its conclusion: Anthropic is using AI to build stronger AI, and the effect is unmistakable—model progress is roughly twice as fast as a year ago, with no sign of slowing. The Verge focused on another angle: the company, even as it publicly calls for slowing AI development, allowed itself to redact a considerable portion of the report before publication.

The report is widely seen as one of the clearest pieces of evidence to date on "AI accelerating its own R&D," moving a question from speculation to measurement: how far are AI systems from improving themselves with little or even no human involvement? The timing is delicate, too—Anthropic is preparing for an IPO while CEO Dario Amodei publicly urges labs to pump the brakes on AI development. METR's investigation was separate from another program that grants safety researchers employee-like access, which was launched only after several hacking incidents this summer.

A Year's Work Accomplishing What Historically Took Two

METR's most notable finding is that benchmark results show Anthropic achieved roughly two years' worth of historical progress in the past year. Researchers stated explicitly that they are confident the acceleration is not merely the result of stacking on more training compute—historically the main driver of AI progress. The report says that as of October, most of the acceleration came from AI, and that contribution is still growing over time. Supporting material comes from informal Slack polls and employee interviews: employees using the company's strongest internal models self-reported productivity about 12 times higher than without AI assistance, whereas a few months earlier, with Mythos 5, that figure was only about half.

For now, most of the acceleration still comes from AI assisting human researchers rather than AI working independently. But METR warns there are signs that fully autonomous contributions will soon account for a larger share. Researchers also found no clear signal of diminishing returns—nor any visible bottleneck that could prevent further acceleration. In other words, the report offers no reassuring ceiling.

Three Unreleased Models and an Old 2022 Model

Image source: The Verge

Anthropic's researchers used models stronger than anything available externally, and METR attributed a considerable portion of the acceleration and efficiency gains to unreleased internal models. Over the past six months, the company completed training of three unreleased models, codenamed Model 2, Model 3, and Model 4, with training periods in July, August–September, and September–October, respectively. METR also identified an unreleased model dating back to 2022, codenamed Unreleased 2022. Researchers noted specifically that these are codenames shared with them, and that the company's internal names differ.

Anthropic gave METR access to confidential research, model performance data, and employee interviews, and METR acknowledged that the information provided exceeded the project's original scope. But the openness was not complete: researchers did not get unrestricted access, nor the employee-like permissions the company had promised other safety researchers. This distinction is key to the controversy that followed.

A 100-Page Report, Most Evidence Blacked Out

Under the agreement between the two parties, Anthropic had the right to redact the final report—and it used that right extensively. METR said on social media that the full evaluation submitted to the company ran about 100 pages, and that in the public version the company removed most of the evidence it considered most informative. The report also notes a relatively narrow information pipeline and that a few data requests were denied. METR repeatedly emphasized the consequences of the redactions: the removed evidence could often be interpreted in multiple ways, making it difficult for the public to assess the conclusions and uncertainties on their own, leaving them to rely more on METR's judgment—and the stakes here are not light.

Anthropic's stated reason was commercial sensitivity, since public details could help competitors. METR said most of the removed information was actually fairly high-level, and that it was not in a position to judge whether the commercial justification held up. Co-founder Sam McCandlish acknowledged in a long post that a lot was removed and that this makes it harder for outsiders to scrutinize METR's work, but the company determined that explaining clearly what drives acceleration would involve trade secrets and amount to handing others a roadmap to go faster—which would instead worsen the very risks METR worries about. He called it a trade-off, said the company judged it had made the right call, and noted it plans to release its own report on AI progress speed and new safety measures within weeks.

China Perspective: The Time Gap Between the Safety Narrative and the IPO Window

The backlash in safety research circles centers on a contradiction: the company that most often invokes safety, transparency, and responsible development was found by its own evaluation to be pushing technology toward a threshold many consider dangerous—while failing to meet its own transparency standards. For the industry, the truly important signal is not any one company's PR line, but that "AI improving AI" now has a measurable productivity number for the first time. Doubling iteration speed and self-reported efficiency in the teens mean the time unit of frontier model competition is shrinking.

Chinese vendors face different constraints: compute supply, engineering deployment, and compliance of generated content. But if the R&D process itself begins to compound, the path of catching up by throwing more people and compute at the problem will fail faster, and the gaps in third-party evaluation and verification mechanisms will become more glaring. China's current labeling rules for AI-generated synthetic content govern how content is labeled—not information like how fast a lab's internal R&D is running. The gap exposed by METR's report is exactly here: when progress data exists only inside the company, and the company can decide how much to disclose, who verifies the curve it publishes itself?

Today's Quote

"For the first time, acceleration has a measurable number—and for the first time, it also has a measurable silence: most of the most informative evidence in a 100-page report was removed, leaving the public to rely on the evaluator's judgment."

Follow [Degao Xing Zhixinglang] for three hardcore tech analyses a day, to understand the industry and national fortunes behind the technology.