山东大学的 Liu 和 Han 测试了三种代码智能体 harness（Claude Code、OpenCode 和 mini-SWE-agent），发现复杂框架并未改善模型的实际编程能力，主要区别在于消耗了更多 token。所有测试均在离线环境下进行。
研究覆盖了 Qwen3.6、Qwen3.8、Deepseek-V4-Flash、GLM-5.3-Flash 和 HY4-Preview 五个模型。论文标题直接给出结论：harness 买到的主要是 token（What does a harness buy? Mostly tokens）。
性能差异来自配置而非框架
Claude Opus 5 仅在 Claude Code 中作为参考运行。研究者移除 Claude Code 的工具调用功能后，Claude Opus 5 的结果没有变化，说明复杂的工具集成对该模型无实质影响。
OpenCode 的性能差距中，约一半可以追溯到输出限制这一配置参数。研究还在 Terminal-Bench 2.1 上进行了小规模交叉检查，结果显示相同模式。
数据与复现
数据和分析脚本已在 GitHub 上公开，支持对实验结果的验证。研究未涉及 harness 对推理速度或错误恢复能力的影响，仅聚焦于最终解题率与资源消耗的对比。
原文地址：tools.wflynn.cn/aitools/tre…
关注公众号「YourTools」，每天推送 GitHub 趋势与开源工具速报：
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Liu and Han of Shandong University tested three codes AI agent Harness (Claude Code, OpenCode and Mini-SWE-agent) and found that complex frameworks did not improve the actual programming capacity of models, mainly because more token was consumed. All tests were conducted in offline environments.
The study covered five models Qwen3.6, Qwen 3.8, Deepseek-V4-Flash, GLM-5.3-Flash and HY4-Preview. The title of the paper gives a direct conclusion: what does a Harness buy? Mostly tokens.
Performance differences from configuration rather than frame
Claude Opus 5 only runs as a reference in Claude Code. The result of Claude Opus 5 has not changed since the researchers removed Claude Code ' s tool call functionality, suggesting that complex tool integration has no material impact on the model.
About half of the performance gap for OpenCode can be traced back to the configuration parameter of the output limit. The study also conducted small cross-checks on Terminal-Bench 2.1, which showed the same pattern.
Data and recurrence
Data and analysis scripts have been made available on GitHub to support validation of the results of the experiments. The study did not address the impact of harness on the speed of reasoning or the ability to recover wrongly, focusing only on the final resolution rate versus resource consumption.
Original address:tools.wflynn.cn/aitools/tre...
Following the WeChat Official Account "YourTools", GitHub's trend and open source tool is sent daily:
Auto-fetched and translated by AI Atlas. All rights belong to the original author.