新智元报道
这就是RSI!
刚刚,谷歌等开源了一项名为Harness²的重磅框架。
在专业基准测试中,即使是Gemini 3.5 Flash这样的顶尖模型,也能在单步优化后实现性能的狂飙。
模型还是那个模型,但智能体已不再是那个智能体。
在两个层面上,谷歌实现了 RSI:
任务级递归:提出各种修改方案,对每个方案都重新执行任务,保留那些能让任务表现更好的修改方案。
梳理式递归:在完成每项任务后,保存所使用的「梳理方式」以及其中的原因分析,这样在处理下一项任务时就能做出更出色的调整。
除了谷歌云 AI 研究团队成员,来自美国西北大学(Northwestern University)的Ruiyao Xu实习时参与了此项工作。
GitHub 开源地址: https://github.com/google-research/harness2
arXiv 论文链接: https://arxiv.org/abs/2610.14224
项目主页: https://harness2.github.io/
Agent黄金 1 0 年开启,但Agent在真实的业务场景中时,问题接踵而至。
真实世界是开放的。没有标准答案。网页每天在变,API随时更新,用户的指令千奇百怪。
当任务失败时,传统的解法是「微调」。
但这带来了三个致命问题:
但这成本极高:算力贵,数据标注更贵。
而且没有从根本上解决灾难性遗忘。你教会了模型用新API,它可能忘了旧的业务逻辑,「按下葫芦浮起瓢」
更致命的是,很多时候用的是Gemini、Claude等闭源API,你根本摸不到它的权重,怎么微调?
我们需要一种非侵入式的升级方案。
Harness(执行框架)让模型能以智能体的方式行动,Harnessing 就是优化这套系统的过程。
Harness² 让这个优化过程变成递归的:它先针对当前任务完善执行框架,再把可复用的框架和经验更新进「改进框架」,用来指导以后的优化。
Harness,直译为「马具」或「控制带」。
在AI语境中,它指的是包围在语言模型外部,帮助模型与现实世界交互的所有基础设施。
谷歌在Harness²中,将这个概念具象化为八大可编辑组件。
这八个组件,就是智能体的「手、眼、记忆和工具箱」。
1 系统提示词(System Prompt): 定义Agent的角色和基础规则。
2 护栏(Guardrails): 设定安全边界,防止危险操作。
3 记忆(Memory): 存储过去的经验和上下文。
4 工具指导(Tool Guidance): 告诉Agent每个工具的具体用法和避坑指南。
5 子智能体(Sub-agents): 负责把复杂任务拆解,分配给特定的小模型。
6 技能(Skills): 针对特定场景固化的操作步骤。
7 脚本(Scripts): 自动执行的代码段落。
8 插件(Plugins): 连接外部服务的接口。
Harness²框架支持OpenCode与Codex执行器,这意味着它可以直接运行和验证复杂的代码逻辑。
重点来了:Harness²优化的不是模型的参数,而是这八大组件。 它让模型自己去发现工具哪里不好用,提示词哪里写得差,然后自己修改它。
这些组件不仅影响提示词本身,还塑造了进入上下文的内容、工具的使用方式、工作的委派方式,以及工具循环内部发生的一切。改进所能改变的不只是一个提示词。
Harness²最大的创新,在于它的「双层递归优化」架构。
它不仅解决眼前的问题,还在为未来的未知问题做准备。
第一层:任务级优化(Task-Level)——解决眼前的问题
面对一个具体的、从未见过的任务,Agent搞砸了怎么办?
Harness²设计了一套被称为PPC(Propose–Probe–Compose,提议-探测-组合) 的机制。
当遇到一个从未见过的棘手任务时,Harness² 不会闷头乱撞,而是启动三步走策略:
第一步,分析当前执行轨迹,针对暴露出的痛点,提出好几个修改「八大组件」的方案。这一步关键在提案。
第二步是对比不同提案,探测最优提案!框架会让换上不同新装备的智能体去试跑,通过直接对比执行差,像老中医把脉一样,找出哪个方案真正解决了卡点。
最后,一步是合成: 把探测中表现最好的修改方案融合起来,生成当前任务的「终极版战甲」。
一句话总结:它通过比对不同策略的执行结果,自己摸索出正确的通关密码。
任务级递归的两种形式:
(a) 并行递归:同时试探多个候选,各自独立组合出下一代框架。
(b) 顺序递归:每一步都把改进后的框架和积累下来的执行证据带入下一步。
第二层:元级优化(Meta-Level)——积累全局的智慧
如果只停留在任务级,Agent不过是个熟练工。换个新任务,它可能又抓瞎了。
Harness²的元级优化,让Agent进化成了「大师」。
每次任务级优化成功后,Harness²会提取其中的核心逻辑,沉淀经验。比如它发现「在处理表格数据时,先用脚本清洗空值,成功率更高」。
这些经验不是随便写的,而是附带了成功的执行轨迹作为证据。
当Agent面对全新的任务时,元级优化会翻开这本「积累了成功方案与教训的错题本」,用过去的底层逻辑,指导当前任务的Harness编辑。
这就像人类的举一反三。 在任务A学到的工具修改技巧,被提炼成了通用法则,用在了完全不同的任务B上。
光说不练假把式,这套不改权重的「换装大法」到底有多猛?
架构再好,还得看疗效。
Harness²在多项高度专业的基准测试中,交出了令人震撼的成绩单。
谷歌放出了极其硬核的评测数据。
在全部锁定模型权重、完全不依赖真实标签验证的情况下,研究人员拿当前备受瞩目的 GLM-5 和 Gemini 3.5 Flash 进行了测试。
仅仅经过单步(Single-step)的 Harness 优化,两大模型在多个极具挑战的专业基准测试中,成绩直接拉出了一条陡峭的阳线:
在极其考验复杂办公流协作的 WorkBuddy-Bench 上,最高暴涨 +11.5分!
在模拟真实职业任务的 JobBench 上,稳步提升 +4.5分!
在硬核的 LAB 基准测试中,准确率更是飙升了 +11.8个百分点!
更恐怖的是,这只是「单步」优化的威力。
论文指出,随着你给智能体增加递归预算 k(即允许它多进行几轮自我迭代),它的表现还能顺着曲线进一步攀升!
Harness² 的开源,迅速在硅谷AI圈引发了强烈共鸣。
LangChain 的核心成员 Sydney Runkle 第一时间在 X 上转发并深度长评。
她敏锐地指出,谷歌 Harness² 的设计与当前最前沿的 Agent 工程理念——Trace(轨迹证据)与 Eval(验证改进) 达成了惊人的高度一致!
她感叹道:
任务级循环通过分析轨迹(Trace)来提出改进提案;
而元级循环则在不断学习哪些改进模式在验证(Eval)中是真正有效的。
这正是让智能体在生产环境中走向可靠的必经之路!
Harness²等于在LangChain这样的框架之上,提供了一个全自动的、自洽的「Agent医生」。
开发者不再需要一行行去看日志、手动调优提示词。Agent自己看着日志,自己就把提示词和工具修好了。
人类能够统治地球,靠的不是脑容量在几千年里发生突变,而是靠着语言(Prompt)、文字记录(Memory)和工具制造(Tools)的代代相传和迭代。
Harness²,正在赋予Agent同样的能力。
参考资料:
编辑:大卫
不喜欢
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
New Ji-won.
This is RSI!
Just now, Google and others have a heavy-pound framework called Harness2.
In professional benchmarking, even top models such as Gemini 3.5 Flash can achieve performance booms after one-step optimization.
The model is still the model, but AI agent is no longer the AI agent.
On two levels, Google has achieved RSI:
The task level is incremental: various modifications are proposed, each of them is reactivated, and those that would allow for better performance of the mandate are retained.
Combination over time: After each task, save the "combination method" used and the analysis of the reasons for it, so that better adjustments can be made when dealing with the next task.
In addition to members of the Google Cloud AI research team, Ruiyao Xu was involved in an internship at North West University.
GitHub Open Source Address: https://github.com/google-research/harness2
ArXiv paper link: https://arxiv.org/abs/2610.14224
Project homepage: https://harness2.github.io/
Agent gold was opened in 1 0 years, but when Agent was in the real business scene, the problem followed.
The real world is open. There are no standard answers. The web page is changing every day, API is up to date, and the user's instructions are so strange.
When the mission fails, the traditional solution is to fine-tune.
But this raises three fatal questions:
But the cost is very high: it's expensive, it's more expensive.
And there is no fundamental solution to catastrophic oblivion. You taught the model to use a new API, and it probably forgot the old business logic, "Put down and float."
What's more deadly is how to fine-tune it when you can't even touch the weight of Gemini, Claude, and so on.
We need a non-intrusive upgrading programme.
Harnessing is the process of optimizing the system.
Harness2 turns the optimisation process into a regression: It refines the implementation framework for current mandates and updates the reusable framework and experience into the "improved framework" to guide future optimization.
Harnesse, translates it as a "pony kit" or a "control belt".
In the AI context, it means all the infrastructure that surrounds the language model and helps it interact with the real world.
Google translated the concept into eight editable components in Harness 2.
These eight components are AI agent's hand, eye, memory and toolbox.
1 System prompt (System Prompt): Defines the roles and underlying rules of Agent.
2 Guardrails: Set safe borders to prevent dangerous operations.
Memoory: Store past experience and context.
4 Tool Guide (Tool Guidance): Tell Agent about the specific use of each tool and how to avoid pits.
Sub-AI agent (Sub-angents): Responsible for dismantling complex tasks and assigning them to specific small models.
6 Skills (Skills): Operational steps for solidification of specific scenarios.
7 Scripts: Automatically executes a code paragraph.
8 Plugins (Plugins): interface to external services.
The Harness2 framework supports OpenCode and Codex implementers, which means that it can directly run and validate complex code logic.
The point is: Harness 2 does not optimize the parameters of the model, but the eight components. It allows the model to discover itself where the tools are not working and where prompt is not writing and then modify it itself.
These components not only affect prompt itself, but also shape the content of access to context, the way in which tools are used, the way jobs are assigned, and what happens inside the tool cycle. It's not just a prompt that can change.
The biggest innovation of Harness 2 is its "two layers of incremental optimization" architecture.
It not only addresses immediate problems but is also preparing for future unknown problems.
First tier: task-level optimization (Task-Level) - addressing immediate problems
What if Agent screws up in the face of a specific mission that he's never seen before?
Harness2 designed a mechanism known as the PC (Propose-Probe-Compose, Proposal-Detect-Group).
When it comes to a difficult mission that we've never seen before, Harness 2 does not sulk, but starts a three-step strategy:
The first step was to analyse the current implementation trajectory and propose a number of options for modifying the eight components in response to exposure pains. This step is crucial to the proposal.
The second step is to compare the different proposals and to detect the best ones. The framework will allow AI agent, who has been replaced with different new equipment, to run and find out, by directly comparing the poor implementation with the old Chinese doctor, which solution really solved the card points.
Finally, one step is to synthesize: to integrate the best-performing modifications in detection, to produce the current mission's "final armour".
Summarizes: by comparing the results of the implementation of different strategies, it finds itself the correct clearance code.
Two forms of mission-level regression:
(a) Parallel regression: multiple candidates are being tested at the same time and the next generation framework is being independently assembled.
(b) Sequencing: each step brings the improved framework and accumulated evidence of implementation into next step.
Second tier: Meta-Level - building global wisdom
If only in the mission level, Agent is a skilled worker. A new assignment, it's probably blind again.
Harness 2 optimizes the hierarchy, making Agent a master.
After each mission-level optimization success, Harness 2 extracts its core logic and settles its experience. For example, it found that “when processing table data, empty values are cleaned with scripts, success rate is higher”.
These experiences are not simply written, but are supported by evidence of successful implementation trajectory.
When Agent faces a brand-new mission, it turns over this "mistakebook that has accumulated successful programmes and lessons" to guide Harnesse in the current mission using the base logic of the past.
It's like the human race. The tool modification techniques learned in task A have been refined into a general rule and used in completely different task B.
How strong is this "mutual change" without changing weight?
It's a better structure, and it's a cure.
Harness2 handed over a shocking report card in several highly specialized benchmark tests.
Google released extremely hard-core assessment data.
The current high-profile GLM-5 and Gemini 3.5 Flash were tested by researchers with all the weights of the locked models, completely independent of authentic label validation.
After only one step (Single-step) Harness optimization, two LLMs, in several challenging professional benchmarking tests, directly pulled a steep ray of light:
+11.5 points on the WorldBuddy-Bench, which has tested complex workstream collaboration!
On JobBench, simulate the real job, steady up +4.5 points!
In the hard core LAB benchmark, the accuracy rate went up by +11.8 percentage points!
And even worse, it's just the power of one step.
According to the paper, as you increase the budget of AI agent to k (i.e. allow it several more rounds of self-inversion), its performance can rise further along the curve!
The open source of Harness2 quickly resonates in Silicon Valley AI.
Sydney Runkle, a core member of Langchain, forwards it on X and comments in depth.
She was keen to point out that Google Harness2 had achieved an amazing degree of convergence between the design of the Agent Engineering Project, currently at the forefront, and Eval!
She lamented:
The task-level cycle provides proposals for improvements through the analysis of the trajectory (Trace);
The cycle, on the other hand, is truly effective in learning which models of improvement are being tested (Eval).
This is the way to make AI agent reliable in the production environment!
Harness 2 provides a fully automatic and comprehensible "Dr. Agent" based on a framework like LangChain.
The developer will no longer need a line to look at the logs and to make a manual prompt. Agent looked at the log himself and fixed prompt and the tools.
Humans are able to rule the Earth not by mutation of brain capacity over thousands of years, but by intergenerational transmission and iterativeity of languages (Prompt), transcripts (Memory) and tools (Tools).
Harness 2, is giving Agent the same power.
References:
Editor: David
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.