Claude's evolution speed has doubled, with 80% of the increase coming from AI

Editor | Zenan

AI helps accelerate the development of the next generation of AI, and specific numbers have finally emerged.

On November 5, the third-party evaluation agency METR released a survey report on Anthropic, indicating that the benchmark capability progress made by the company in the past year is roughly equivalent to the progress made in the previous two years.

From August 12 to October 29, this agency spent about 11 weeks reviewing Anthropic's internal data on model performance, training computing power, research and development efficiency, and conducting interviews with researchers.

Their analysis utilized the Anthropic version of the Comprehensive Capability Index (AECI). The data shows that since October 2025, the progress rate of large models on this index has been approximately 1.9 times that of the previous period, with a 95% confidence interval of 1.5 to 2.8 times.

Since October 7, 2025, this score jumped from about 15 points per year to about 29 points per year, covering roughly two years' worth of progress in just one year, roughly equivalent to Claude 2.1 to Sonnet 4.5. Anthropic's own risk report had previously assumed that Mythos Preview was a one-time leap, after which it would return to the original trend; however, new models have emerged, and the new slope continues.

Looking at the entire industry, the acceleration of cutting-edge models is only 1.4 times.

More critically, METR is quite certain that this acceleration does not stem from the growth rate of AI training computing power. The main reason for the advancement of AI capabilities comes from AI's own assistance.

In interviews, a recurring phenomenon is that researchers can finally try more ideas.

In the past, an experiment might take one or two days to implement, with time and computational costs being insurmountable barriers, causing many ideas to be shelved before they could be validated. Now, researchers can hand multiple ideas to an Agent to see which ones are worth pursuing further.

One researcher described that their work is now mainly limited by ideas and computing power, with code no longer being a bottleneck. Others feel that after using Claude, it seems like they have a postdoc and graduate students to help push projects forward. Experiments and implementations have become cheaper, making researchers more willing to try, and many small experiments that were previously considered not worth the effort now have the chance to become new research directions.

METR's report also reveals some internal progress that was previously unseen by the outside world. Their analysis included three models from Anthropic that have not yet been publicly disclosed, referred to as Model 2, Model 3, and Model 4. They completed training between July and October of this year.

The strongest of these is Model 4 (some speculate it is Claude 6), which has already scored over 90% on many common benchmarks. Anthropic employees made subjective estimates internally in October, voting that it has improved their work efficiency by about 12 times on average.

These models have not been released and may not necessarily become public products, but their existence serves as a reminder that merely looking at release events and public rankings may not provide a complete picture of what is happening internally at major AI companies. From about 1 time at the beginning of 2025, to about 2.5 times for Opus 4.5, about 5 times for Mythos 5, and jumping to 12 times for Model 4, the exponential fit suggests an annual increase of 3.9 times.

METR estimates that as of October this year, about 80% of the current acceleration in AI research is estimated to come from AI itself. The remaining 20% comes from non-AI factors, the details of which Anthropic does not allow to be disclosed.

So, can AI independently develop its next generation?

Currently, there is still a gap before AI can autonomously develop the next generation of models through complete recursive self-improvement (RSI). Anthropic researchers themselves believe that in the most critical part of model training, AI has not yet made highly significant and fully autonomous contributions, and the main benefits still come from AI assisting human development.

However, Anthropic researchers have already described a significant amount of AI's autonomous contributions as "signs of life," and there has been quantitative analysis on the proportion of tasks completed by AI versus humans in various stages, but the data was deleted by Anthropic.

As long as stronger models can help researchers develop the next generation faster, the feedback process may continue to enhance. How far the path of RSI can go depends on the aspects that have not yet been automated, as well as whether resources like computing power can keep up.

In terms of predictions for the near future, METR does not foresee any significant bottlenecks that would disrupt rapid progress in the next 12 months, but they also admit that as information increases, they have changed their judgments multiple times, and their current confidence is limited.

Additionally, although Anthropic allowed the third-party agency METR to conduct an internal investigation, the current 100-page report has undergone significant redaction by them. The most compelling evidence has mostly been removed by Anthropic itself.

A supplementary review released on the same day also presented the differences in risk statements between the two parties: Anthropic had described a certain leading indicator as having a progress rate of "less than twice," while METR calculated it to be about 1.97 times based on its internal data, believing that a more accurate statement should be "about twice." The twofold acceleration involves Anthropic's early warning threshold.

METR believes that the possibility of having crossed the relevant threshold by July 15 cannot be ruled out, but most relevant researchers still tend to believe that it had not yet crossed at that time. Both parties also agree that the direct catastrophic risk of the relevant models at that time was low.

As the speed of model involvement in research and development increases, whether people can timely recognize changes and whether companies can fully disclose uncertainties are becoming questions that must be answered before the arrival of the next generation of AI.

Reference content:

https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/#executive-summary

https://x.com/METR_Evals/status/2118412183860351084

© THE END