Anthropic联合创始人杰克·克拉克将自我改进AI的预测推向2027年
一个新的基准测试称TasteVal测量AI系统自行设计实验的精度和解释结果。 调查结果显示，改进曲线异常陡峭。
根据Anthropic联合创始人杰克·克拉克的说法，自2025年12月以来，这种能力大约每三个月翻一番。 研究实验室P-Zero Research称其为"研究品味"，意为能够选择正确的实验并从中得出正确的结论。 测试的最佳模型以成本的一小部分超过了由24名人类专家组成的比较小组。
克拉克共同创办了Anthropic，并撰写了进口AI通讯。 基于数字，他现在相信递归自我改进(RSI)，即AI系统开发自己的继任者，早在2027年就可以实现。 他之前曾预计到2028年。
基准测试并不能证明AI系统已经在自我改进，因为它只捕捉到狭义的科研工作。 但是，它确实显示了辩论的方向，从原始的模型性能，走向一个问题，即独立的研究工作语言模型可以实际承担多少。
味道仅测量画面的一部分
P-Zero Research将研究品味定义为三种能力：确定哪些研究问题值得追求，设计实验来回答，以及正确解释结果。 尝华只试后二。 选择最有希望的研究方向往往被认为是科学直觉的核心，但在TasteVal中，已经为模型做出了决定。
八项任务中的每一项都从AI模型开发中明确了一个固定的研究目标，并附有可衡量的分数。 任务包括培训数据选择、预训和微调语言模式、人类偏好培训以及防御对抗性提示。
该模型决定下一步运行的实验，并利用结果指导其下一步。 为了不让编码技巧在断断续续的结果上，一个固定编码agent在单一的Nvidia H100 GPU上运行所有实验。 每次运行都经过40GPU小时的计算或120小时的壁时后结束。
人类基线来自最近曾在OpenAI,Google DeepMind,Nvidia,Microsoft研究等地工作的24位专家。 他们执行同样的任务，每人至少有两人。 每个任务中最好的人类尝试是基准测试。
通过计算效率测量研究品味
为了让研究品味量化，P-Zero Research将其转化为计算效率。 推理是，在AI研究中，实验计算很少，因此选择正确的实验并正确读取其结果意味着运行较少，在同一预算上的进展更多。
实验室借鉴其"AI未来模型"，该模型预测，一旦编码自动化，实验计算就会成为最大的瓶颈。 因此，尝试Val衡量一个模型需要多少计算来匹配专家的分数。 一个用半个计算得到的模型拥有这个定义的两倍的研究品味。 P-Zero Research称这个比率为计算乘数。
顶级型号Anthropic的Opus 5.5击出2.3的计算乘数，达到专家级分数，计算分数不足一半。 统计范围为1.15至4.37。 在低端，模型几乎不会领先。 单次运行成本约为专家努力成本的三分之一。
自2025年12月以来，在几代模范中，计算乘数大约每三个月翻一番。 此处的不确定性也很大，报告的翻倍时间从1.7个月到5.0个月不等。
在往年，模型研究的口味增长得慢得多。 曲线只在2025年转向2026年左右陡增。 这是否是一个持久的趋势，不能从一个单基准测试中确定，作者自己也承认他们的结果可能会过分夸大"研究品味是如何迅速改善的". 任务可以快速和廉价地检查，这是AI实验室发现最容易优化的任务类型。 比较组也没有包括任何顶级研究人员。
克拉克把时间推向前进
克拉克长期以来一直称创造力是AI系统如何快速改善自身的关键因素。 今年早些时候，他写道，模型并没有在任何开创性的规模上表现出这种能力。 他现在指出在数学突破和基准测试如TasteVal等方面的进展，作为这个变化的迹象。
在5月的一篇长篇散文中，克拉克提供了更谨慎的数字。 他把一个独立训练更强大的继任者的系统的几率在2028年末达到约60%，到2027年达到30%。 这个预测来自于一个共同创立公司的人，他的模型超过是基准测试，新增对预测有一层偏差。
批评者正是以这种能力划线的
味道很重要，因为它衡量怀疑论者所标注的阻碍AI重大突破的确切能力。 AI研究员赫比·布拉德利(Herbie Bradley)推后克拉克的"梅"(May)散文，认为虽然模型可以令人信服地接管初级研究人员的工作，但研究品味，创造力等要求更高的技能以及连贯的长期研究规划却无处可见。
在规划和评估实验时，TasteVal反对这一评估，尽管称没有选择长期研究议程。
在富丹大学一个团队领导的AI agents的AI开发研究中，实际差距有多大，显示仍然上升。 研究者分析了自己模型的发展，发现AI agents贡献了高达55%的方法建议，但人类仍然做出了超过85%的最后决定。 作者指出，AI如何在结果出现之前产生不同的研究方向并评估其价值，目前还不清楚，这正是TasteVal取消模型盘的决定。
辩论正在转变
基准测试在一场辩论中到达 AI大实验室踢出自己。 6月，Anthropic写道，递归自我改进还没有实现，但"可能比大多数机构准备的要快",AnthropicCEO达里奥·阿莫代现在呼吁在AI开发中实现有控制的减速，而OpenAI则承认完全自主的RSI还没有实现，称只有在人的监督得到保障的情况下才应该追求。
品尝改变什么是进步。 长期以来，重点是模型如何很好地解决诸如数学问题或编码挑战等定义明确的任务，但现在的问题是，它们能否做出推动研究前进的决定。
如果P-Zero研究在其他测试中测得的趋势持续下去，其影响就会同时减少。 AI的开发会更快，更便宜，但与此同时，越来越多的研究份额会更难被人类验证，因为模型会规划和评价自己的实验。
AI News Without the Hype – 由人类治愈
订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。
现在订阅
读取全照。
订阅无韵报道。
- 全面查阅THE DECODER的每篇文章
- 没有广告
- 加入评论和社区讨论
- 每周通过邮件转载AI新闻
- 6x/年："AI雷达"——对AI最重要的话题进行深度潜水。
- 每日AI新闻，总是最新的
- 我们完整的十年档案
- 由一个在AI工作10年以上的团队负责
订阅到 The Decoder
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Anthropic co-founder Jack Clark moves his prediction for self-improving AI forward to 2027
A new benchmark called TasteVal measures how well AI systems design experiments on their own and interpret the results. The findings show an unusually steep improvement curve.
That capability has roughly doubled every three months since December 2025, according to Anthropic co-founder Jack Clark. The research lab P-Zero Research calls it "research taste," meaning the ability to pick the right experiments and draw the right conclusions from them. The best model tested outperformed a comparison group of 24 human experts at a fraction of the cost.
Clark co-founded Anthropic and writes the Import AI newsletter. Based on the numbers, he now believes recursive self-improvement (RSI), where AI systems develop their own successors, could become possible as early as 2027. He had previously expected 2028.
The benchmark doesn't prove that AI systems are already improving themselves, since it only captures a narrow slice of scientific work. But it does show where the debate is heading, away from raw model performance and toward the question of how much independent research work language models can actually take on.
TasteVal only measures part of the picture
P-Zero Research defines research taste as three abilities: identifying which research questions are worth pursuing, designing experiments to answer them, and correctly interpreting the results. TasteVal only tests the last two. Choosing the most promising research direction is often considered the core of scientific intuition, but in TasteVal that decision is already made for the model.
Each of the eight tasks specifies a fixed research goal from AI model development, along with a measurable score. The tasks cover areas like training data selection, pretraining and fine-tuning language models, training on human preferences, and defending against adversarial prompts.
The model decides which experiment to run next and uses the results to guide its following step. To keep coding skill from skewing the outcome, a fixed coding agent runs all experiments on a single Nvidia H100 GPU. Each run ends after 40 GPU-hours of compute or 120 hours of wall-clock time.
The human baseline came from 24 experts who had recently worked at places like OpenAI, Google DeepMind, Nvidia, and Microsoft Research. They tackled the same tasks, with at least two people on each one. The best human attempt per task served as the benchmark.
Research taste measured through compute efficiency
To make research taste quantifiable, P-Zero Research translates it into compute efficiency. The reasoning is that experiment compute is scarce in AI research, so picking the right experiments and reading their results correctly means fewer runs and more progress on the same budget.
The lab draws on its "AI Futures Model," which predicts that experiment compute becomes the biggest bottleneck once coding is automated. So TasteVal measures how much compute a model needs to match the experts' scores. A model that gets there with half the compute has twice the research taste by this definition. P-Zero Research calls this ratio the compute multiplier.
The top model, Anthropic's Opus 5.5, hit a compute multiplier of 2.3, reaching expert-level scores with less than half the compute. The statistical range runs from 1.15 to 4.37, though. At the low end, the model would barely lead. A single run cost roughly one-thirtieth of what an expert attempt cost on average.
Across several model generations, the compute multiplier has roughly doubled every three months since December 2025. The uncertainty is large here too, with the reported doubling time ranging from 1.7 to 5.0 months.
In prior years, model research taste grew much more slowly. The curve only steepened around the turn of 2025 into 2026. Whether that's a lasting trend can't be determined from a single benchmark, and the authors themselves admit their results may overstate "how quickly research taste is improving." The tasks can be checked quickly and cheaply, which is the kind of task AI labs find easiest to optimize for. The comparison group also didn't include any top-tier researchers.
Clark pulls his timeline forward
Clark has long called creativity the key factor in how fast AI systems can improve themselves. Earlier this year, he wrote that models hadn't shown this ability at any groundbreaking scale. He now points to progress in mathematical breakthroughs and benchmarks like TasteVal as signs that this is changing.
In a long essay in May, Clark had offered more cautious numbers. He put the odds of a system that independently trains a stronger successor at roughly 60 percent by the end of 2028 and 30 percent by 2027. That forecast comes from someone who co-founded the company whose model tops the benchmark, which adds a layer of bias to the prediction.
Critics had drawn the line at exactly this capability
TasteVal matters because it measures the exact capability that skeptics had flagged as a barrier to a major AI breakthrough. AI researcher Herbie Bradley pushed back on Clark's May essay, arguing that while models could plausibly take over the work of junior researchers, more demanding skills like research taste, creativity, and coherent long-term research planning were nowhere in sight.
When it comes to planning and evaluating experiments, TasteVal pushes against that assessment, though it says nothing about choosing a long-term research agenda.
How wide the practical gap remains shows up in a study on AI development using AI agents led by a team at Fudan University. The researchers analyzed the development of their own model and found that AI agents contributed up to 55 percent of methodology proposals, but humans still made over 85 percent of the final decisions. The authors noted it remains unclear how AI can generate diverse research directions and assess their value before results are in, and that's exactly the kind of decision TasteVal takes off the models' plate.
The debate is shifting
The benchmark arrives in the middle of a debate the major AI labs kicked off themselves. In June, Anthropic wrote that recursive self-improvement hasn't been achieved yet but "could come sooner than most institutions are prepared for" Anthropic CEO Dario Amodei now calls for a controlled slowdown in AI development, while OpenAI acknowledges that fully autonomous RSI isn't happening yet and says it should only be pursued if human oversight is guaranteed.
TasteVal shifts what counts as progress. For a long time, the focus was on how well models solve well-defined tasks like math problems or coding challenges, but now the question is increasingly whether they can make the decisions that push research forward.
If the trend P-Zero Research measured holds up in other tests, the implications cut both ways. AI development would get faster and cheaper, but at the same time a growing share of the research would become harder for humans to verify, because models would be planning and evaluating their own experiments.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe now
Read on for the full picture.
Subscribe for hype-free coverage.
- Full access to every article on THE DECODER
- No ads
- Join the comments and community discussions
- A weekly AI news recap via mail
- 6x/year: "AI Radar" — deep dives on the AI topics that matter most
- Daily AI news, always up to date
- Our full ten-year archive
- Covered by a team with 10+ years in AI
Subscribe to The Decoder
Auto-fetched and translated by AI Atlas. All rights belong to the original author.