虽然很多注意力都集中在周一经过修订的S-1A中披露的更新的财务Anthropic上(收入飞涨，但费用也随之增加),AI实验室在精细的印刷品中又投下了一个小炸弹壳。
备案似乎警告说，该公司正在建立一个能够递归自我改进的模式。
RSI是AI发展得如此精通训练自身，从而催生AI能力剧增的点。 对AI末日者来说，RSI代表着没有回归的灾难，在灾难中，AI自己积聚了足够强大的东西，以至于它滑落到人类控制之外，可能在实现它的目标时对它所造成的损害漠不关心。
对AI的爱好者来说，RSI或某些版本的AI可靠地利用自己来改进，代表了技术的希望，它成为了无法想象的生产力、科学发现和更多工具。
Anthropic似乎认为RSI既是承诺也是危险。 它在S-1中称在研发中使用AI，但它告诫说："利用AI帮助建立更有能力的AI也会带来相当大的安全风险，特别是如果它能大大加快进度。 这是我们负责任的扩大政策所涵盖的领域之一，该政策提出了我们在接近所谓的`倒退自我改进'时监测和减轻风险的计划。 ` ` '
当RSI一词在今年早些时候流行起来时，许多人认为它是一个响亮的词，或者说，充其量是一个过于难以实现、定义或证明的目标。
但是在星期一，Anthropic在它的"风险因素"一节中加入了新的声明，说1)它已经使用AI来开发新的模型，它在能力上的飞跃在当时并没有得到承认，2)它可能很快会满足它自己的内部对RSI的定义(粗体强调我们的，下面是：
此外，我们越来越多地使用人工智能系统来加快我们自己的研发工作，可能会使我们的模式能够比我们制定、验证和执行相应的安全、配合和安全措施的能力更快地取得进展。 因此，我们过去没有确定，今后可能也没有确定，我们的发展速度将显著加快，直到发展发生之后。
例如，我们的内部估计和我们对未来模型的期望，而不是我们目前的模型，表明我们可能即将达到我们负责任的扩大政策的定义，即“由大赦国际实际加速的进展”。 我们没有将目前的任何模型指定为超过这一阈值，我们继续评估基本数据。
然而，Anthropic在备案中也表示其型号还没有超过这个RSI阈值，并且它有望在可预见的将来将人类留在研发过程中。 “如果技术上可行，我们预计无法安全地使用Claude，在没有有意义的人类监督或理解的情况下，完全自主地设计和建造世代相传的前沿模型。 因此，我们正在考虑更新我们的负责任的扩大政策，使人类控制Anthropic的前沿模式发展。”
怀疑论者可能指出，S-1既具有披露功能，也具有市场营销功能，而声称与一个模型关系密切，如此强大，可以自行建立，可能属于后一类。
然而，毫无疑问，Claude的能力，特别是编码、数学和网络安全的能力，今年已经呈跃进和极限。 如果Anthropic认为RSI的可能性是近在咫尺的，也许我们应该相信它并相应采取行动——也许治理——.
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
While much attention has focused on the updated financials Anthropic revealed in its revised S-1A on Monday, (revenue is soaring, but so are expenses), the AI lab dropped another little bombshell in the fine print.
The filing seems to warn that the company is on the cusp of creating a model capable of recursive self-improvement (RSI).
RSI is the point at which AI grows so good at training itself that it spawns a dramatic acceleration of AI capabilities. To the AI doomers, RSI represents the cataclysm from which there is no return, in which AI builds itself into something powerful enough that it slips beyond human control and may be indifferent to the damage it causes while achieving its goals.
To AI enthusiasts, RSI, or some version of AI reliably using itself to improve, represents the technology’s promise, where it becomes a tool of unimaginable productivity, scientific discovery, and more.
Anthropic seems to believe that RSI is both a promise and a peril. It says in the S-1 that it uses AI in its R&D, but it cautions: “Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress. This is one of the areas covered by our Responsible Scaling Policy, which sets out our plans to monitor and mitigate the risks as we approach what has been termed ‘recursive self-improvement.’”
When the term RSI became popular earlier this year, many believed it was a buzzword or, at best, a goal too hard to achieve, define, or prove.
But on Monday, Anthropic added new statements to its Risk Factors section, saying 1) it has already used AI to develop new models whose leaps in capability it didn’t recognize at the time, and 2) it may soon meet its own internal definition of RSI (bold emphasis ours, below):
In addition, our increasing use of AI systems to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures. As a result, we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.
For example, our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data.
However, Anthropic also said in the filing that none of its models have crossed this RSI threshold yet and that it is promising to keep humans in the R&D process for the foreseeable future. “We do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic.”
Skeptics might point out that an S-1 functions as both disclosure and marketing, and that claims of being close to a model so powerful that it can build itself could fall into the latter category.
Nevertheless, there’s no question that Claude’s capabilities, especially in terms of coding, math, and cybersecurity, have grow in leaps and bounds this year. If Anthropic thinks the possibility of RSI is close, maybe we should believe it and act — perhaps govern — accordingly.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.