CodeScene 发布了一项案例研究：编码智能体在三周内重构了一个包含 30 万行 C 代码的代码库，token 成本约为 4000 美元。这项工作在 726 个文件中产生了 2903 次提交，修改了 252055 行代码，并将代码库的 Code Health 评分从 5.6 提升至 10.0。该代码库来自《街头霸王 III：三度冲击》的一个开源反编译项目。
CodeScene 创始人、《Your Code as a Crime Scene》作者 Adam Tornhill 写道，在参与大型系统开发 30 年后，这是他第一次看到他所谓的大规模超人类 AI 表现。
两项机制支撑了这项工作。第一项是质量信号：CodeHealth MCP Server 为智能体提供了一个可供优化的确定性评分，并用于判断一次转换是否产生了积极效果。第二项是正确性保障：一个回放轨迹测试框架逐帧比较回滚状态哈希，从而在每次更改后检查系统行为。
更具新意的成果，是智能体在工作过程中构建的内容。它们没有应用固定的重构目录，而是逐步积累了一套重构手册，最终形成 22 个方案和 82 条辅助说明。其中包括 Extract Function 和 Guard Clauses 等常见转换，也包括针对该代码库的特定方案。Shared Index Range 用于处理仅起止范围不同的重复循环；Action Parameter 用于处理主要区别在于所调用函数的重复控制结构；Uniform Step Table 则将异构调用转换为表驱动分派。失败的尝试也被记录下来，包括那些使 Code Health 评分下降的转换。
模型选择也很重要。团队最终决定使用 Claude Opus 完成大部分工作，并报告称，在识别和记录逐渐形成的模式方面，搭配 Opus 的 Claude Code 明显优于搭配 Sol 的 Codex。使用较小模型执行任务时，文件的评分经常停滞，看起来像是到达了一个无法进一步突破的局部最优点。
从业者在 LinkedIn 上的反应出现了明显分歧，争议并不在于这件事是否发生，而在于这一结果究竟证明了什么。
Paolo Perrone 说明了为什么应该认真看待这项成果：
我读过的大多数重构成果都建立在测试套件全部通过的基础上，但这只能说明测试没有失败。针对一款格斗游戏进行轨迹回放，设定了高得多的标准。
Omda Response 首席技术官 Mats Iremark 表示，自己也做过类似的事情，并称 CodeScene MCP 与当前智能体的组合几乎就像作弊一样。
质疑者主要关注适用范围。技术负责人兼顾问 Konrad Otrębski 询问这些更改是否已经合并，是通过一次巨大的合并还是多次合并完成，以及这是否只是在开源代码上进行的实验，而不是在真正创造收入的生产代码上进行的工作。NeoSee 首席技术官、参与这项工作的两名工程师之一 Daniel Webb 回应称，这些更改通过 54 个拉取请求合并到了一个分支版本的 main 分支中。
Otrębski 随后的回应提出了更高的标准：
我认为，对 AI 能力真正的检验，是把这种重构服务提供给某个知名开源项目，比如 Grafana。当然，完成的定义是将其合并到 master 分支。
软件架构师兼研究人员 Tracy Bannon 反对的是对成果的描述方式，而不是所采用的方法。她指出，把结果描述为完美是一个相当大胆的说法。
Denis Baltor 对智能体发现的方案本身提出了质疑。他认为，DRY 关注的是知识和意图的重复，而不是完全相同的代码行；如果方案把仅范围不同的循环或仅调用函数不同的控制结构视为重复，就可能混淆了这两者。
Asko Nõmm 提出了方法论问题。他认为，由于 Claude Code 和 Codex 分别针对自家模型进行了调优，因此难以判断结果在多大程度上衡量了模型，又在多大程度上衡量了测试框架，成本也会以同样的方式发生变化。他还指出，架构并未得到衡量，因此代码看起来可以很健康，而根本性问题可能会在日后才暴露出来。
一些尚未解答的问题来自作者自己。Marc Bouvier 询问非功能行为是否有所改善，因为帧率、内存使用量和输入延迟对游戏都很重要；Webb 表示，他们正在引入一位性能专家。有人问测试框架是否捕获了细微的帧时序回归，他回答说，可能没有明确答案，因为该框架作为提交前钩子运行，有些故障在被观察到之前就已经修复。对于差异规模，他也同样直接：如果你不熟悉代码，而且不再逐行审查，那么一次差异可以有多大？他没有给出结论，只提出了问题。
如果这些反应具有代表性，那么分歧并不在数字本身，而在于产生这些数字的条件。回放轨迹测试框架之所以有效，是因为反编译游戏可以提供确定性的逐帧回放。大多数遗留系统不存在与之相当的判定机制，而这也正是重构它们存在风险的原因。Tornhill 写道，自动化测试和等价性检查是绝对必要的保障措施，这恰好把负担落在了不健康代码库往往最缺少的东西上。
团队自己的选择过程也说明了这一点。Webb 表示，他们曾考虑使用一个他参与过的 Gov.UK 海事许可代码库，但它过于健康，不适合后续研究；团队选择这款游戏，部分原因是他们自己也玩这款游戏，因此也是它的用户。
这项研究是下一步工作。此次改进产生了同一系统的两个功能等价版本，一个 Code Health 评分为 5.6，另一个为 10.0。团队将与隆德大学合作开展一项研究，让学生使用前沿模型分别在两个版本中实现功能，并比较成本和质量。
另外两个引人注目的数字属于未来研究，而不是此次案例研究。CodeScene 预计，此次改进可以使 AI 引入的缺陷减少约 70%，并使 token 浪费减少约 45%，但这两个数字都是根据其早期研究推算得出的，并非在此次研究中实际测得。4000 美元的成本和三周的用时才是实际测量结果。
原文链接：https://www.infoq.com/news/2026/09/agentic-refactoring-case-study/
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
CodeScene publish has a case study: code AI agent recreated a code library of 300,000 line C codes within three weeks at a cost of approximately $400,000. The work produced 2903 submissions in 726 documents, modified 252055 line codes and raised Code Health rating from 5.6 to 10.0 in the code library. The code library comes from an open-source inverted translation project, " Street King III: Three degrees of shock ".
The founder of CodeScene, Adam Tornhill, author of Your Code as a Crime Scene, wrote that after 30 years of involvement in large system development, this was the first time he saw what he called a massive superhuman AI.
Two mechanisms underpin this work. The first is a mass signal: CodeHealth MCP Server provides an optimised definitive rating for AI agent and is used to determine whether a conversion has had a positive effect. The second is the security of correctness: a frame-by-frame framework for re-orbit testing compares back-rolling Hashi, thus checking system behaviour after each change.
More innovative results are what AI agent constructs in the course of its work. Instead of applying a fixed reconstructive catalogue, they have built up a reconstruction manual that has resulted in 22 programmes and 82 supporting notes. These include common conversions such as ExtraFunction and Guard Clubs, as well as specific scenarios for this repository. Shared Index Range is used to handle repetition cycles of different ranges only starting and ending; Action Parameter is used to handle duplicate control structures that differ mainly from the function called; Unit Step Table converts isomer calls to table-driven assignments. Attempts to fail were also recorded, including those that reduced the Code Health rating.
Model selection is also important. The team finally decided to use Claude Opus to do most of the work and reported that in terms of identifying and documenting evolving models, Claude Code, which matched Opus, was clearly better than the Codex, which matched Sol. When smaller models are used to carry out mandates, document ratings often stagnate and appear to have reached a local advantage that cannot be broken further.
There was a clear disagreement among practitioners about LinkedIn ' s reaction, and the controversy was not about whether it happened, but what the result proved.
Paolo Perrone explains why this result should be taken seriously:
Most of the heavy components I have read are based on the complete passing of the test package, but this only shows that the test did not fail. A re-orbiting of a combat game sets a much higher standard.
The Chief Technical Officer of Omda Reponse, Mats Iremark, stated that he had done similar things and that CodeScene MCP was almost like cheating in its current combination with AI agent.
The main concern of challengers is the scope of application. Konrad Otrębski, Technical Officer and Adviser, asked whether the changes had been consolidated, whether they had been done through a large merger or multiple consolidations, and whether they had been only experimented with open source codes, rather than with real income-generating production codes. Daniel Webb, Chief Technical Officer of NeoSee, one of the two engineers involved in the work, responded that these changes were consolidated in a branch version of the main branch through 54 pull requests.
Otrębski's subsequent response suggests a higher standard:
I think that the real test of AI's ability is to provide this restructuring to a well-known open source project, such as Grafana. Of course, the definition of completion is to merge it into the master branch.
The software architect and researcher Tracy Bannon objected to the way results are described, not the method used. She noted that it was rather bold to describe the results as perfect.
Denis Baltor questioned the programme itself that AI agent discovered. In his view, DRY was concerned with the repetition of knowledge and intent rather than the exact same code line; if the programme viewed the only cycle of different scopes or only the different control structures of the function as duplication, the two might be confused.
Asko Námm raises methodological questions. In his view, it was difficult to judge the extent to which the results measured the model, the testing framework and the cost would change in the same way as Claude Code and Codex had refined their own models. He also noted that the architecture had not been measured and therefore the code seemed healthy and that fundamental issues could be revealed later.
Some of the outstanding questions came from the authors themselves. Marc Bouvier asked whether there had been any improvement in non-functional behaviour because frame rates, memory usage and input delays were important for the game; Webb indicated that they were introducing a performance specialist. Asked whether the test framework captured a fine frame time-series regression, he replied that there might be no clear answer, as the framework operated as a pre-submission hook and some malfunctions had been repaired prior to being observed. He is equally straightforward in terms of the magnitude of the difference: if you are not familiar with the code and do not review it on a line-by-line basis, what can be the difference at a time? He did not draw conclusions, but merely asked questions.
If these reactions are representative, the difference is not in the numbers themselves, but in the conditions under which they are produced. The back-track test framework is valid because the back-coding game can provide definitive frame-to-frame replay. Most legacy systems do not have comparable determination mechanisms, which is why they are re-constructed. Tornhill writes that automated testing and price-equivalent testing are absolutely necessary safeguards, which precisely places the burden on what is often the most lacking in unhealthy code banks.
This is also illustrated by the team ' s own selection process. Webb stated that they had considered using a Gov.UK maritime licence code repository in which he had participated, but that it was too healthy for follow-up; the team chose the game in part because they themselves played the game and were therefore its users.
This study is a next step job. This improvement resulted in two functional equivalents of the same system, one Code Health rating 5.6 and the other 10.0. The team will work with Lund University on a study to enable students to use the front-line model to function in two versions, and to compare costs and quality.
Two other striking figures are for future research rather than this case study. CodeScene expects that this improvement will reduce the deficiencies introduced by AI by about 70 per cent and token waste by about 45 per cent, but both figures are derived from its early studies and are not actually measured in this study. The cost of $4000 and the time spent for three weeks are the actual measurements.
Original link: https://www.infoq.com/news/2026/09/content-refacturing-case-study/
Auto-fetched and translated by AI Atlas. All rights belong to the original author.