新智元报道
在企业级真实业务与复杂数据分析中，让大模型「一眼看穿数据库并写出完美SQL」，往往只是一个不切实际的幻想。
传统的Text-to-SQL大多将其简化为单轮文本生成：给定问题和静态Schema，让模型一步生成最终SQL。但现实数据库动辄包含上百张表、数千字段与百万级数据，资深分析师绝不会「单轮盲猜」，而是在CLI或IDE 中执行 `SHOW TABLES`、探查表结构（DDL）、跑试探性查询（Probe Query），并根据报错反复修正逻辑后才提交代码。
在Spider2.0、BIRD-Interact和LiveSQLBench等长程交互基准上，传统静态微调模型纷纷暴露致命短板：训练时只学会「直接写终点答案」，部署到交互环境（Harness）后却缺乏探索试错、状态追踪与错误恢复能力，极易陷入死循环或因报错崩溃——即「训练与部署脱节」（Train–Deploy Mismatch）。
针对这一瓶颈，港科大与微软等团队推出了首个原生长程环境交互训练框架HarnessSQL，在密闭沙盒与专用SQL Harness中进行长程轨迹监督微调（SFT）与执行奖励强化学习（RL）。
在Spider 2.0-SQLite 上，HarnessSQL仅凭8B参数即斩获45.2%执行准确率（近基线3倍），大幅超越32B专用模型（OmniSQL 14.8%、Arctic-Text2SQL 16.3%）与多智能体系统（MARS-SQL 12.6%），14B模型更达54.8%！
论文链接：https://arxiv.org/abs/2610.12274
开源代码：https://github.com/YangHaolin0526/HarnessSQL
图1：从单轮Text-to-SQL到具备沙盒与专用Harness的交互式数据库Agent演进。
交互式Text-to-SQL可形式化为由执行环境契约（Execution Harness）调解的部分可观测马尔可夫决策过程（POMDP）。任务实例定义为四元组 x = (q, D, h, v)，分别对应自然语言指令q、底层数据库D、执行Harness h与隐藏真实验证器（ExecutionOracle）v。
在第t ∈ [1, T]轮交互中，智能体根据环境观测o_t∈O，由策略π_θ生成推理与工具调用行动a_t∈A，完整轨迹表述为：
其中y为终止步提交的最终SQL，任务成败由联合奖励函数评估：
其中，
R_result(y, v)∈{0, 1}：在真实数据库D上比对候选产物y与标准答案v的执行结果集外延等价性，打破表层字符匹配局限；
R_protocol(τ, h)∈{0, 1}：校验轨迹是否遵守Harness契约（只读权限、语法合规、无死循环且在T步内显式提交）。
然而，现有后训练框架大多仍停留在传统的静态监督损失：
这种静态训练模式导致了三重核心脱节：
1.抹杀中间探索状态：只将最终SQL作为目标，忽略了中间探测表结构与验证假设的交互过程；
2.面对执行反馈盲目：训练时从未见过运行报错或空结果集，推理时一旦遇到报错极易崩溃；
3.缺乏动作边界约束：模型不理解数据库CLI的权限边界与终止协议，易产生工具格式幻觉。
为打破上述脱节，HarnessSQL构建了三大基石：高保真密闭执行沙盒、极简SQL Harness契约，以及两阶段长程后训练流水线。
1.密闭SQL执行沙盒与蓝图合成
团队基于Spider 2.0-Lite（30个SQLite）与Spider 2.0-DBT（49个DuckDB，零损迁移至SQLite）构建了79个复杂数据库沙盒，并采用SQL-First蓝图合成法：先从活动引擎提取Schema、样本值与外键路径，采样出难度可控的可执行SQL，再反向生成自然语言问题与验证Oracle。
沙盒具备只读绑定、单语句约束与60秒超时保护，并通过语法、非空与语义变异测试（Mutation Tests）保障数据质量。
2. 专为SQL任务定制的极简Harness契约（DSH-SQL）
通用编程Agent的繁杂Bash工具易导致模型注意力涣散。HarnessSQL改造了DeepSeek-Harness（DSH），提炼出四元极简动作契约：
sql_list_tables：枚举数据库内所有数据表及各自的数据行数；
sql_schema：探查指定表的完整DDL定义、字段类型与真实采样数据；
sql_exec：试探性执行只读SQL查询，获取前排真实输出或运行时错误堆栈；
sql_submit：终极提交已验证正确的SQL产物并退出长程会话。
环境中间件同时内置三大护栏：确定性行数截断防止大输出撑爆上下文；循环调用警示器主动纠偏重复无效查询；压力触发上下文压缩确保长程探索保留核心事实。
3. 原生Harness两阶段后训练：SFT筑基+RL登顶
第一阶段：全轨迹长程SFT（冷启动）。利用教师模型在沙盒中自主探索，筛选通过Oracle验证的优质多轮轨迹（平均11.8Ktokens、16轮交互、14次真实执行）进行全序列监督：
关键机制：计算损失时，所有环境观测token（o_t）均被掩码忽略（Masked Out），仅在模型自身生成的推理、工具调用与最终提交上计算梯度，迫使模型学会根据环境反馈推演下一步行动。
第二阶段：执行奖励强化学习（自主进化）。为赋予模型应对未知数据库的试错能力，HarnessSQL采用动态解耦裁剪策略优化（DAPO）在沙盒中持续对齐，计算轨迹组标准化优势：
奖励信号完全依托隐藏Oracle的真实执行对比：结果等价且合规得1分，报错、超时或结果不符均得0分。不设任何人工中间奖励（Reward Shaping），倒逼模型探索出稳健的自纠错策略。
图2：HarnessSQL整体架构流水线与沙盒交互轨迹示例。
团队在三大真实数据库基准上展开评测：域内长程交互基准Spider 2.0-SQLite、含用户模拟器的域外动态交互基准BIRD-Interact Mini，以及跨领域复杂推理基准LiveSQLBench Base-Lite SQLite。
对比基线涵盖通识编程模型（Qwen2.5-Coder-32B）、前沿SQL专用模型（OmniSQL 7B/32B、Arctic-Text2SQL 7B/32B）与多智能体系统（MARS-SQL 3×7B），统一采用执行准确率（Execution Accuracy）评估。
表1：三大基准上的执行准确率对比，粗体为最优，下划线为次优。
实验呈现出三大核心结果：
1. 8B小模型全面反超32B模型：在Spider 2.0-SQLite上，HarnessSQL-8B达到45.2%，较同底座基线（15.5%）提升近3倍，大幅超越OmniSQL-32B（14.8%）、Arctic-Text2SQL-32B（16.3%）和Qwen2.5-Coder-32B（16.3%）。
2. 14B模型逼近55%大关：扩展至Qwen3-14B后，执行准确率进一步攀升至54.8%，验证了原生Harness训练出色的规模扩展性。
3. 强劲的域外泛化能力：在未见过的BIRD-Interact Mini与LiveSQLBench上，HarnessSQL同样遥遥领先。其中在LiveSQLBench上，8B（24.8%）与14B（28.5%）达到所有32B基线（7.0%–8.9%）的3～4倍，证明模型习得了通用探索策略而非机械记忆Schema。
面对交互环境任务，该直接做强化学习（Direct RL）还是先做轨迹微调（SFT）？受控实验给出了清晰对比：
冷启动直接RL提升有限：8B基座直接跑RL仅从15.5%微升至20.0%（+4.5%）。缺乏交互先验时，稀疏二值奖励难以在长轨迹中有效分配信用。
全轨迹SFT奠定交互基础：经验证轨迹SFT后，8B和14B分别跃升至30.4%（+14.9%）与37.8%（+15.6%），快速掌握了工具调用与报错纠偏规范。
SFT+RL实现性能质变：在SFT基础上引入DAPO强化学习，8B再涨14.8%达45.2%，14B再涨17.0%达54.8%（增益达直接RL的3倍以上）。这确立了长程Agent的黄金范式：SFT赋予交互框架与探索常识，RL释放未知环境下的自适应寻优潜能。
围绕长程Agent训练的关键设计选择，消融实验提炼出四项核心发现：
启示一：训练与推理接口的强对齐（Training-Inference Alignment）
若推理采用DSH-SQL交互环境，但训练使用非交互式（Non-harness）SFT数据，Spider2.0准确率仅9.6%，甚至低于未微调基座（15.5%）；而原生HarnessSFT达30.4%。这证明：静态训练会破坏模型与动态环境的协作，训练与部署环境必须严格同构。
非交互式与原生Harness训练和推理在域内及域外基准上的交叉消融对比。
启示二：全序列轨迹组织显著优于单步拆解（Full-Trajectory>Turn-Split）
构建SFT数据时，保留完整轨迹上下文的全序列监督（Full-trajectory）在三大基准上全面优于单步拆解（Turn-split）。长程任务高度依赖前后多轮的因果脉络，拆碎样本会割裂全局决策连贯性。
图3：全轨迹监督（Full-trajectory）显著优于单步拆解监督（Turn-split）。
启示三：超长上下文容量是长程探索的生命线（8K->16K->32K）
真实数据库探查伴随密集Schema与中间结果反馈，轨迹动辄上万tokens。当训练与推理上下文从8K扩展至16K和32K时，Spider2.0准确率从27.4%跃升至45.2%——截断早期历史等同于剥夺模型的复盘与溯源记忆。
图4：上下文窗口长度对执行准确率的影响，32K上下文带来显著提升。
启示四：DAPO算法更加契合稀疏奖励下的长程策略优化
在相同预算与沙盒下对比GRPO（40.7%）、GSPO（43.7%）与DAPO（45.2%），DAPO表现最优。其动态过滤零方差轨迹组、解耦裁剪范围与全批次token归一化机制，有效缓解了稀疏奖励下的策略震荡。
团队也客观指出了当前系统的演进方向：
1.跨数据库方言扩展：从当前的SQLite生态进一步拓展至Snowflake、BigQuery、PostgreSQL等企业级云原生方言，构建更完善的权限隔离与生态适配。
2.多模态与多元分析工具：除命令行查询外，进一步融合可视化图表、文档报告与交互式BI仪表盘等多元分析环境。
大模型正加速从「对话助手」走向「自主执行智能体」。HarnessSQL证明：面对需要工具调用与试错反馈的复杂任务，仅灌输静态「正确答案」远远不够，必须将交互环境（Harness）纳入核心训练闭环，让模型在动态环境中原生习得长程探索与容错能力。
目前，HarnessSQL的代码、沙盒环境与合成轨迹资产已全部在GitHub开源，为Text-to-SQL、数据库智能体与通用Agent研发提供了可直接复用的标杆参考。
参考资料：
https://arxiv.org/abs/2610.12274
编辑：LRST
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
In real business and complex data analysis at the enterprise level, allowing LLM to "see through the database and write perfect SQL" is often an unrealistic illusion.
Traditional Text-to-SQLs have mostly been simplified into single-wheel text generation: problem-setting and static Schema, allowing models to produce final SQLs one step at a time. However, the reality database often contains hundreds of tables, thousands of field and million-class data, and senior analysts will never "one-wheel blind guess" but rather implement `SHOW Tables ' in CLI or IDE, DDL, Probe Query, and re-engineered the logic of submitting the code.
In long-range interactive benchmarks such as Spider 2.0, Bird-Interact and LiveSQLBench, traditional static micromodulation models expose fatal short panels: training is conducted with "direct end-point answers", but deployment to an interactive environment lacks the ability to explore error, state tracking and error recovery, and is vulnerable to a dead-end cycle or to a reported failure — that is, "training-Deploy Mismatch".
In response to this bottleneck, teams such as Hong Kong City and Microsoft launched HarnessSQL, the first original growth environment interactive training framework, which provides long-range track monitoring fine-tuning (SFT) in closed sandboxes and dedicated SQL Harness, and implementation incentives for enhanced learning (RL).
On Spider 2.0-SQL, HarnessSQL achieved 45.2 per cent implementation accuracy (nearly three times the baseline) on 8B only, significantly exceeding 32B dedicated models (OmniSQL 14.8 per cent, Arcic-Text2SQL 16.3 per cent) and more AI agent systems (MARS-SQL 12.6 per cent) and 54.8 per cent for 14B!
Paper link: https://arxiv.org/abs/2610.12274
Open source code: https://github.com/yangHaolin0526/HarnessSQL
Figure 1: An evolution from a single-wheel Text-to-SQL to Agent, an interactive database with sandboxes and Harness.
An interactive Text-to-SQL component of the implementation of the Environmental Compact (Execution Harness) mediation is visible in the Markov decision-making process. The task example is defined as a four-dimensional group x = (q, D, h, v) corresponding to the Natural Language Directiveq, a bottom database D, the execution of Harness h and the hidden Real Validator (Execution Oracle) v.
In the t ∈[1, T] round interaction, AI agent, based on the Environment Watch o t∈O, was generated by the π Q strategy and tool call Action a t∈A, with the full trajectory expressed as:
Of these, the final SQL submitted by y for termination steps is assessed by the joint reward function:
Of which,
R result(y, v)∈0 {0,}: cross-validation of the results of the implementation of candidate y and standard answerv against real database D, breaking the surface character matching limit;
R protocol (and, h)∈, {, {, {, {, {, {, {, {, Z, Z, Z, Z, Z, )。, )。, )。, )。, )。, )。, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, {, )。, {, )。, )。, )。, )。, {, {, {, {.
However, most of the existing post-training frameworks remain in traditional static monitoring losses:
This static training model leads to a triple core disconnect:
1. Erase the intermediate exploration state: the final SQL is only the target, ignoring the interaction between the intermediate detection table structure and the certification hypothesis;
2. In the face of implementation feedback blind: never seen at the time of training a collection of running errors or empty results, and in the case of reasoning, there is a high risk of collapse if errors are reported;
3. Lack of operational boundary constraints: The model does not understand the permission boundary and termination protocols of the CLI database and can create a tool-format illusion.
In order to break this gap, HarnessSQL has built three cornerstones: a sandbox that is closed, a very simple SQL Harness contract, and a two-stage long-distance training line.
1. Closed SQL executes sandbox and blueprint synthesis
The team built 79 complex database sandboxes based on the Spider 2.0-Lite (30 SQLite) and the Spider 2.0-DBT (49 DuckDBs, with zero-loss migration to SQLite) and used the SQL-First blueprint synthesis method: first extracting Schema, sample values and external keys from the active engine, then extracting controllable SQLs, then reverse generating natural language problems and authenticating Oracle.
Sandboxes have read-only bindings, statements bound to 60 seconds timeout, and quality of data is guaranteed through syntax, non-empty and semantic mutations (Mutation Tests).
2. Hardness Compact (DSH-SQL) tailored to SQL tasks
The complex Bash tools of Generic Programming Agent can easily lead to a breakdown of model attention. HarnessSQL adapted DeepSeek-Harness (DSH) to extract a four-dollar compact of extremely simple actions:
sql list tables: all data tables and their respective data rows in the database;
sql schema: look into the full DDL definition of the specified form, field type and real sampling data;
sql exec: Test test to perform a read-only SQL query to get a front row real output or run-time error stack;
sql submit: Finally submit the correct SQL product and exit the long-range session.
The environmental middle is accompanied by three main columns: a determination line cut to prevent large output blasts of context; a circular caller actively corrects invalid queries; and pressure triggers context compression to ensure that the core facts are explored for long distances.
3. Post-stage training of original Harness: SFT Foundation + RL Deng Fuck.
Phase 1: Full-track long-range SFT (cold start). Using teacher models to explore in sandboxes, screening of quality multi-wheel tracks certified by Oracle (average 11.8 Ktokens, 16 rounds of interaction, 14 real-life exercises) for full-series monitoring:
Key mechanisms: When calculating the loss, all environmental observations token (o t) were ignored (Masked Out), only the reasoning generated by the model itself, tool call and the gradients eventually submitted, forcing the model learner to extrapolate next step action based on environmental feedback.
Phase II: implementation of incentives for enhanced learning (self-evolution). To give the model the capability to deal with the error of an unknown database, HarnessSQL uses a dynamic decomposition tailoring strategy (DAPO) to align continuously in the sandbox to calculate the advantage of standardization of track groups:
Incentive signals depend entirely on hiding the true execution comparison of Oracle: the result is equal and the compliance score is one, and the error, the timeout or the discrepancy is zero. There is no incentive for anyone working in the middle, and the reverse model explores a robust self-correction strategy.
Figure 2: Example of how HarnessSQL's overall structure flows between water lines and sandboxes.
The team evaluates the three real database benchmarks: the domain long-range interactive benchmark Spider 2.0-SQLite, the extraterritorial dynamic interactive benchmark BIRD-Interact Mini, with the user simulator, and the cross-cutting complex reasoning benchmark LiveSQLBech Base-Lite SQLite.
The comparative baseline covers the generic programming model (Qwen2.5-Coder-32B), the front-line SQL specialized model (OmniSQL 7B/32B, Arkic-Text2SQL 7B/32B) and the multi AI agent system (MARS-SQL 3 x 7B) and the unified use of Exact Implementation Accuracy assessment.
Table 1: A comparison of the performance accuracy of the three benchmarks with the best in bold and the underlined.
The experiment presented three core results:
1. The 8B small model is a comprehensive counter-32B model: on the Spider 2.0-SQL-8B, HarnessSQL-8B reached 45.2 per cent, nearly three times higher than the base base (15.5 per cent) baseline, significantly exceeding OmniSQL-32B (14.8 per cent), Arcic-Text2SQL-32B (16.3 per cent) and Qwen2.5-Coder-32B (16.3 per cent).
2. The 14B model approaches the 55 per cent level: after its expansion to Qwen 3-14B, the implementation accuracy rate rose further to 54.8 per cent, confirming the scale expansion of the original Harness training.
3. Strong extraterritorial capacity: HarnessSQL is as far ahead as LiveSQLBench in an unknown BIRD-Interact Mini. In LiveSQLBech, 8B (24.8 per cent) and 14B (28.5 per cent) were three to four times as high as all 32B baselines (7.0 per cent-8.9 per cent), proving that the model had a common exploration strategy rather than a mechanical memory, Schema.
In the face of an interactive environmental mission, should we do direct intensive learning (direct RL) or first fine-tuning (SFT)? Controlled experiments offer a clear comparison:
Direct RL upgrades for cold start-up are limited: direct RL runs from base 8B only slightly increased from 15.5% to 20.0% (+4.5%). In the absence of an interactive a priori, it is difficult to allocate credit effectively in long trajectories.
The full track SFT lays the basis for interaction: after the empirical track SFT, 8B and 14B jumped to 30.4 per cent (+14.9 per cent) and 37.8 per cent (+15.6 per cent) respectively, and quickly mastered tool call and errors and errors.
SFT+RL achieves performance change: the introduction of DAPO intensive learning on the SFT basis increased by 14.8 per cent to 45.2 per cent and by 17 per cent to 54.8 per cent (more than three times the direct RL). This establishes the golden paradigm of Agent long-range: SFT gives interactive frameworks and exploration common sense, and RL releases the potential for self-adaptation in unknown environments.
Four core findings were extracted from the digestion experiment around key design options for long-range Agent training:
Revelation I: Strong alignment between training and reasoning interface
If the reasoning is based on the DSH-SQL interactive environment, but the training uses non-interactive (Non-harness) SFT data, the Spider 2.0 accuracy rate is only 9.6 per cent, or even lower than the non-miscal base (15.5 per cent); the original HarnessSFT is 30.4 per cent. This proves that static training undermines collaboration between models and dynamic environments, and that training and deployment environments have to be closely constructed.
Non-interactive contrasts with cross-fertilization of original Harness training and reasoning within the domain and extraterritorial benchmarks.
Revelation II: Full-series trajectories significantly superior to single-step dismantling (Full-Trackory>Turn-Split)
When building SFT data, maintain full sequence monitoring (Full-trajory) of the complete track context, which is better than single-step dismantling (Turn-split) on the three main benchmarks. Long-range missions are highly dependent on the causal veins of multiple rounds, and the break-up of sample breaks global decision-making coherence.
Figure 3: Full-track monitoring is significantly better than single-step dismantling monitoring (Turn-split).
Revelation III: Excess context capacity is the lifeline of long-range exploration (8K->16K->32K)
Real database searches are accompanied by intense feedback on Schema and intermediate results, with a trail of 10,000 tokens. When context expanded from 8K to 16K and 32K, the Spider 2.0 accuracy rate jumped from 27.4% to 45.2% - Interrupting the early history is tantamount to depriving the model of its retrodiscussion and tracement.
Figure 4: Impact of the length of context window on the accuracy rate of implementation, 32 Kcontext resulted in a significant increase.
Revelation four: DAPO algorithms are more compatible with long-range strategy optimization under rare incentives
In comparison with sandboxes in the same budget, GRPO (40.7 per cent), GPS (43.7 per cent) and DAPO (45.2 per cent), DAPO performed best. The dynamic filtering of the zero differential trajectory group, the decoupling of the cut-off range and the harmonization of the whole batch of tokens have effectively mitigated the tactical shock of the thin incentive.
The team also objectively pointed to the direction of the current system:
1. Cross-database dialect extension: from the current SQLite ecology, further expansion to enterprise-level cloudland dialects such as Snowflake, BigQuery, PostgreSQL, building a better permission isolation and ecological adaptation.
Multimodular and Plural Analysis Tool: Further integration of multiple analytical environments such as visual charts, document reports and interactive BI dashboards, in addition to command line queries.
LLM is accelerating from "dialogue assistant" to "self-executing AI agent". HarnessSQL proves that, in the face of complex tasks requiring tool call and error feedback, it is far from sufficient to inculcate static “right answers” and that interactive environments (Harness) need to be integrated into core training closed loops to allow models to develop long-range detection and tolerance capabilities in dynamic environments.
At present, HarnessSQL code, sandbox environment and synthetic trajectory assets are all at GitHub open source, providing direct reuseable poles for Text-to-SQL, database AI agent and General Agent development.
References:
https://arxiv.org/abs/2610.12274
Edit: LRST
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.