💡 一句话总结：LitTraceQA 是个较真的评测——不只看答案，还要求你说出答案出自哪篇论文的哪一页、哪个表格、哪条公式，三样分开打分。TRACE 系统在这条赛道拿下 0.7606，却在复盘里承认：论文找对率高达 97%，证据定位只有 68%。差距不在模型聪明与否，而在系统接口一次次「偷换身份」——这就是作者命名的 grounding contract gap（接地契约鸿沟）。
导语：语义正确 ≠ 评分可见
先讲一个所有做 RAG 的人都遇到过、但很少有人系统讨论的现象。
你问一个科研问答系统：「方法 A、B、C、D 在指标 X 上的对比数据是什么？」系统其实检索到了全部四篇论文，最后也给出了正确数字。但在中间某一步，它把四篇论文的检索结果混进了一个全局排序——A 论文的相关段落排第 3，D 论文的相关段落被挤到第 35。融合排序看第 35 名：弱证据，丢弃。于是 D 的数据没答上。
信息明明在语料里，系统也明明「看过」它——但它在评分器可见的形式上消失了。这就是这篇论文的核心概念：grounding contract gap——语料中可得的信息，与系统以正确身份输出到下游（模型、评分器、用户）的信息之间的落差。
这篇文章是 GroundLM 2026（EMNLP 2026 workshop）LitTraceQA 赛道（2.7 万篇论文候选池）的系统报告，作者 Sachin Gupta 和 Divya Godara 是两位独立研究者——上一期我们刚写过他们的姊妹篇 EVICALC（arXiv 2609.39013），那篇的教训是「证据没进来，算术白算」；这一篇更进一步：证据进来了，但身份不对，照样白干。
想自己动手试？
- 📄 论文：arXiv 2609.38861
- 💻 代码：github.com/sachinkg12/…
这篇论文讲的是什么任务？
LitTraceQA 的输出不是一段回答，而是一条完整轨迹：
研究问题 → 规范论文 ID 集合 → 这些论文内的类型化证据定位 → 契约一致的类型化答案
证据定位还分类型：文本片段、表格、图、公式/算法、引用上下文，精确到页码或对象 ID。三段轨迹分别计分，证据必须闭合于你选中的论文集合。测试集 71 题（50 道选择题 + 21 道表格题），候选池 27,487 篇论文。
这个设计的杀伤力在于：它惩罚一切「语义上的差不多」。你引到相邻页？错。你把「Eq. (6)」规范化错成另一个公式？错。表格里数字抄对了、但行键用了自己的转述而不是评分器的规范键？还是错。两条轨迹合起来的效果是——模型再聪明，也救不回上游接口丢掉的身份信息。
作者的第一版管线就集齐了三种病：全局排序压扁目标组（上文那个 D 论文的惨案）、只输出答案路径引用过的证据（答案没引用的支持材料全丢）、把每篇论文当一行表格行（把「包含对比的论文」当成了「被对比的方法」）。注意：这三种没有一个是模型能力问题，全是表示失败。
它的思路是什么？
TRACE 的架构可以概括成四件事，每件都在对一种「身份丢失」说不。
第一，目标分组检索：别在排序前抹掉身份。 一个零温度规划器先读题，抽出 venue/年份约束、期望论文数、命名锚点，并给每个提及的名称标注语义角色——是答案目标、证据锚点还是约束条件（比如「只在用了数据集 D 的方法里找」，D 就不该被当成答案返回）。检索走五条路（名称/别名/标题解析、BM25、引用上下文、角色感知信号、密集向量），每条候选都带着完整身份证：哪条路来的、属于哪个目标组、组内第几名、什么角色。组内局部排名是整个设计的命门——全局第 35 名在「第四个方法」的组里可能是第 1 名，平坦融合把这个信息抹掉后，任何加权都救不回来。
第二，独立证据定位：别让答案决定证据。 证据检索与答案生成分开跑：Gemini 2.5 Flash 在有界 PDF 文本上提议类型化定位符（页码 + 规范化的对象 ID，比如「Eq. (6)」「Reference [24]」，同时拒绝正文里随便一个数字冒充编号），再按结构有效性、置信度、BM25 相关性排序去重。为什么不用「答案引用了什么就输出什么」？因为那看起来精度很高，实际上把答案没引用的有效支持全砍了——典型的假象精度。
第三，观察单位优先的表格：论文是容器，不是行。 生成表格前先问一个问题：一行代表什么实体？从 schema 的行键列判断——键是 Paper Title，这是「论文轴」；键是 Method/Benchmark/Author，这是「实体轴」，此时论文只是承载对比数据的容器。作者举的例子很典型：问四个具名方法的对比，行实体是那四个方法，而不是「恰好包含这张对比表的论文」。提取按源进行、组装按确定性规则进行（键规范化→合并→类型检查→去重），视觉提取的结果要重复一致才被接受。
第四，fail-closed 执行：别让事故悄悄变成提交。 这段是工程党的狂喜：推理前严格预检（71 个唯一查询 ID、27,487 个池 ID、索引字节数、官方文件哈希）；只有论文-证据闭合检查和官方验证器都通过，才原子写入结果。为什么这么偏执？因为作者真的踩过坑：空索引、常量标签回退、全空行，这些严重事故产出的都是「schema 合法」的输出——不跑契约检查，你根本分不清「模型答错了」和「管道断了」。
效果到底怎么样？
官方终测（71 题）：综合 0.7606。拆开看很有意思：
「找到论文」和「给出答案」都接近满分，「证据在哪一行哪一格」却只有三四成——这个断层本身就是论文的论点：拿到源 ≠ 拿到身份。相邻页在语义上没问题，在评分器眼里就是错的定位元组；描述性行键下的正确数字，在评分器眼里就是错的单元格。
开发集上还有个诚实的发现：坐标感知的视觉填充把表格行 F1 从 0.291 提到 0.411，但证据 F1 反而从 0.460 掉到 0.397——收益是有代价的，这种诊断结果拿去当「官方提升」宣传就是造假，作者把它标得清清楚楚。
能拿来做什么？
- 多目标检索系统：检索候选必须携带「哪个目标组 + 组内排名」的完整身份再融合；回答「对比 A/B/C/D」类问题时，先按目标分组配额，再谈全局排序。
- 带引用的问答：证据定位与答案生成分开跑；审引用时要分清「引用文本转写对不对」和「引用的段落选没选对」两件事。
- 表格抽取：动手前先确定行实体；论文/文档是源容器，不一定是观察单位；视觉提取加「重复一致」门控能显著降噪。
- 管道可观测性：数量检查、内容哈希、闭合检查、语义回退遥测——这套 fail-closed 清单可以直接抄进任何「检索 + 托管模型 + 结构化输出」的系统。
泼冷水时间
- 样本很小：表格诊断只有 11 条开发集记录，说明不了官方测试集的提升；71 题的测试集本身也限定了结论强度。
- 不完全可复现：公开配置独立跑只能完整重现 37/71 条记录（差异主要在表格构建），托管 API 的漂移是硬伤；最终 0.7606 的 JSONL 是作为「审计过的 artifact」单独上传的。
- checkpoint 选择用了官方聚合分数：虽然每条修复都只来自源输入（作者特意区分了这两件事），但「看官方分数选版本 + 源证明的增量修复」这套流程，换一个更严格的评测方可能不允许。
- 纯工程系统报告，数字依赖具体任务与评分器；「契约」那套方法论可迁移，分数不可迁移。
理性看待
把这篇和 EVICALC 连起来读会非常有意思——同一对作者，两道题，一个教训的两个面：上一道题里证据从没进来，系统对着盆栽花解病毒题；这一道题里证据进来了但身份不对，页码差一页、行键换了说法，分数照样归零。合起来的结论朴素得近乎无情：检索-定位-生成这条管道上，模型能力早已不是瓶颈，接口处的表示失败才是。更多模型调用不能可靠地修复身份丢失——作者实测过；显式地把身份信息（目标组、组内排名、规范键、观察单位）做进中间表示，才可以。
评测也在教做人：LitTraceQA 这种「分项计分 + 精确身份」的设计，本质是把「接地」从一句口号变成了可以逐项扣分的契约。当 AI 系统开始被要求「说清楚证据从哪来」时，决定分数的可能不再是模型有多聪明，而是你的管道有没有把每一层身份完整地传递到最后一米。
（数据与结论均来自论文原文，解读部分为作者观点，欢迎交流。）
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
One sentence: LitTraceQA is a more serious assessment -- not just an answer, but an answer from which page, which table, which formula, which formula, which one. The TRACE system took 0.7606 on this track, but admitted in the disc that the result was 97 per cent for the paper and 68 per cent for the evidence. The difference is not whether the model is smart or not, but the system interface is "removed" over and over again -- this is what the author named as the grounding contract Gap.
Introduction: semantic correctness.
Let's start with a phenomenon that everyone who does RAG has experienced, but that is rarely systematically discussed.
You ask a scientific question-and-answer system: "What's the comparison data on indicator X? The system actually retrieved all four papers and finally gave the correct number. But at one step in the middle, it mixed the results of the retrieval of four papers into a global ranking - the relevant paragraphs of the A paper were numbered 3, the relevant paragraphs of the D paper were squeezed to 35. Combining 35th place: weak evidence, discard. So D data doesn't answer.
The information is in the medium and the system is "seen" it -- it disappears in the form that the scorer can see. This is the core concept of the paper: the gap between the information available in the text and the information that the system exports to downstream (models, scorers, users) in the right capacity.
The article is a systematic report of the GrandLM 2026 (EmNLP 2026 workshop) LitTraceQA track (27,000 papers) by Sachin Gupta and Divya Godara, two independent researchers -- - We just wrote their sister EVIC (arXiv 2609.39013), the lesson of which was "The evidence is not in, it's not true"; it goes even further: the evidence is in, but the identity is wrong and it's dry.
You want to try it yourself?
- Paper: arXiv 2609.38861
- Code: github.com/sachinkg12/...
What is this paper about?
The output of LitTraceQA is not an answer, but a complete trajectory:
Research questions, normative papers, ID collections, typology of evidence within these papers, typology of contractually consistent answers.
Evidence location is also of type: text clips, tables, diagrams, formulae/calculations, citing context, accurate to page number or object ID. The three tracks are scored separately, and the evidence must be closed to the collection of papers that you select. Test Series 71 (50 options + 21 tables), candidate pool 27,487 papers.
The lethality of this design is that it punishes everything "similarly." You draw to the adjacent page? Wrong. You've mistook "Eq.(6)" for another formula? But the line key uses its own transposition instead of the scorer's key? Or wrong. The combined effect of the two tracks is that the model is smart enough to save identity information that is lost in the upstream interface.
The author's first version of the pipe brings together three types of diseases: global sorting of the crushing target group (the tragedy of the D paper above), output of only the evidence cited in the answer path (the lack of supporting material cited in the answer), and use of each paper as a table line (the "comparable paper" as the "comparable method"). Note: None of these three are problems of modelling capacity, all a failure.
What's the idea?
The structure of TRACE can be summed up in four things, each saying no to an identity loss.
First, target group search: do not erase identity before sorting. A zero-temperature planner reads first, extracts venge/year bounds, number of expected papers, naming anchors, and indicates the semantic role of each name mentioned - – The answer target, the evidence anchor or the conditionality (e.g. "Only in a D-data set" approach, D should not be taken back as the answer). Five routes were retrieved (name/alias/title resolution, BM25, reference to context, role perception signal, intensive vector) and each candidate carried a complete identity card: which route came from, which target group, which number of players in the group, and which role. The local ranking in the group is the entire design -- the 35th is probably the 1st in the "fourth method" group, and the flat integration erases this information and no weight can be saved.
Second, independent evidence positioning: do not let the answer determine evidence. Evidence retrieval runs separately from the creation of answers: Gemini 2.5 Flash proposes a typed locator (page number + standardized object ID, such as "Eq. (6)" "Reference [24]", while rejecting any number imposting in the body), and reordering it by structural validity, confidence, BM25 relevance. Why don't you just say, "What's the answer?" Because it looks very precise, it actually cuts out the effective support that was not quoted -- the typical pseudo-precision.
Third, a table of priority for the observation unit: the paper is a container, not a row. Before generating the form, ask a question: What does a line represent? Judging from the linebar of schema - the key is PaperSmart Tile, which is the "axis of thesis" and the key is Method/Benchmark/Author, which is the "substantive axis", at which time the paper is only a container carrying comparative data. The example given by the author is typical: asking for a comparison of the four famous methods, which are the four methods, rather than the "paper that happens to contain this comparison table". The extraction is done by source and assembly is performed by the certainty rule (the key-standardized combination type check removes the weight), and the visual extraction results are accepted only after repetition.
Fourth, fail-closed implementation: don't let the accident turn into a submission. This is the ecstasy of the Engineering Party: strict pre-censorship prior to reasoning (71 unique inquiries, 27,487 pool IDs, index Z113Z numbers, official documents Hashi); only thesis - the evidence closed examination and the official certifier are passed, the atom is written down. Why are you so paranoid? Because the author has actually stepped on the pit: empty index, constant label rollback, and all-out lines, all of these serious accidents are produced by the output of "schema legal" — no contract check, you can't tell the difference between "mode wrong" and "pipe broken."
How did it work?
Official Finals (71th Question): Consolidated 0.7666. It's interesting to open up:
"Find the paper" and "Give the answer" are close to full score, and "The evidence is in what line" is only 30-40 percent. The fault itself is the theory of the paper: get the source, get the identity. The adjacent page is semantically sound, and is the wrong position set in the eyes of the scorer; the correct number under the description line key is the wrong cell in the eyes of the scorer.
There is also an honest discovery on the development set: the coordinated visual filling of the table line F1 from 0.291 refers to 0.411, but the evidence F1 falls from 0.460 to 0.397 — there is a price to pay for this diagnosis, which is faked by "official promotion" propaganda, and clearly marked by the author.
What can I do with it?
- Multi-target search system: the search candidate must carry the full identity of which target group + the ranking in the group; in response to the question of comparison of A/B/C/D, group the quota by target and then set the overall order.
- Questions and answers with quotations: the evidence location runs separately from the answer generation; the references are examined in a way that distinguishes between the two things: "The quoted text is transliterated" and "The quoted paragraph is not correctly selected".
- Table extraction: Identification of line entities before action is taken; paper/document is a source container, not necessarily an observation unit; visual extraction plus " repeat consistent " door control can cause significant noise loss.
- Observability of pipelines: quantitative, content-Hashi, closed, semantic rollback telemetry - this list of fail-closed can be copied directly into any system "Research + hosting model + structured output ".
Cold Water Time
- sample is small: only 11 development logs are recorded in the form diagnostics, which do not indicate an increase in the official test set; the test set of questions itself limits the intensity of conclusions.
- Not entirely recapable: Publicly configured stand-alone runs can only be fully duplicated 37/71 records (the difference is mainly in the form construction), hosting the API drift is a hard wound; ultimately, the JSONL is uploaded separately as "audited artifact".
- Checkpoint selects the official aggregate score: While each restoration comes from a source input (the author specifically distinguishes between the two), the process of "seeing the official fraction selected version + the incremental repair of the source certificate" might not allow for a more rigorous rating.
- Pure engineering system reports, numbers rely on specific tasks and scorers; the "contract" methodology is transferable and scores are not.
A rational view.
It'd be very interesting to read this in conjunction with EVICALC -- the same author, two questions, two sides of a lesson: The evidence never came in on the previous question, where the system solved the virus; the evidence came in, but the identity was wrong, and the page number was one page away, the key changed, and the score was zero. The combined conclusion is almost inexhaustible: the search-positioning-generation of this pipeline is no longer a bottleneck in modelling capability, but rather the failure of the interface. More models cannot reliably repair the loss of identity — the author has measured it; identity information (target group, group ranking, norm keys, observation unit) can only be clearly intermediated.
The evaluation is also teaching people: LitTraceQA, the design of "subsequent + precise identification", is essentially a contract that transforms "location" from a slogan to one that can be deducted. When AI began to be asked to "state where the evidence came from", it was possible that the determination of fractions was no longer how smart the model was, but whether your conduit had passed each layer of identity to its last metre.
(The data and conclusions are taken from the original paper, read in part as the author ' s views and welcome the exchange.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.