💡 一句话总结：这是一篇没有新算法、但每个做过企业 RAG 的人都会拍大腿的 MLSys 工业 track 报告：来自花旗、安永和英伟达的三位作者，把「文档怎么变成模型能吃的向量」这个脏活系统化成四相管线，每一步都自带证据。最实用的一个数字：页级路由之后，企业语料里 92% 的页面走零模型成本的原生提取，只有 8.1% 需要请 OCR 模型出山——你还在无脑全量 OCR 的话，这钱省得不冤。
导语：GenAI 时代的「感知层」没有人管
传统机器学习时代，数据准备是「窄」的：任务定死了，schema 定死了，数据清洗对着目标函数做。生成式 AI 把模型解放了——一个基础模型伺候无数下游任务——代价是数据侧的复杂度全数涌到了入口：企业知识长在人用的格式里，PDF、PPT、Excel、扫描件、表单、混合版式，一个文件里同时装着文字、图表、几何布局和结构信息。
这条入口管线坏一点，下游再强的检索器、重排器都救不回来——这是我们在前面几篇（EVICALC、TRACE、FoveDoc）里反复验证过的道理。但这篇论文问的是更前面的问题：这条入口管线本身，该怎么造、怎么测、怎么管？
三位作者（花旗的 Abbas Raza Ali、安永的 Muhammad Ajmal Siddiqui、英伟达的 Moona Zahid）给出了一套四相生产系统，投在 MLSys 2027 工业赛道。论文的自我定位很诚实：不提新算法，贡献是「围绕现有解析器的路由、评分与审计脚手架」。全篇还有一个很可爱的自律——每句话标证据等级：[E] 已确立的研究、[D] 系统自己的实测、[P] 本文提议、[H] 假设。企业系统报告能做到这个程度，本身就是示范。
它的思路是什么？——三原则 + 四相
先记三条设计铁律，后面对号入座：
- 结构先于语义：能用文档自带结构（字体层级、表格线、标题、分页）解决的，不上学习模型——确定性、便宜、可调试；
- 不改所测：评估器只读，且复用生产环节的确切模型与配置——评估测的必须是生产那套系统；
- 预算标注：全量真值不现实，用小型、模态平衡、可追溯采样的样本代替总体置信。
第一相，提取：让 OCR 成为例外。 每一页算一个图像面积比，超过 0.8 才栅格化送去视觉模型，其余全走 PyMuPDF 原生结构提取：字体大小校准标题层级、数字归一化后跨页复现的块当页眉页脚删掉、有线表用线检测、无线表用词对齐聚类（还配一个「这看起来像多栏散文」的守卫防止误伤）。在 Orion 语料的 10,507 页上，只有 8.1% 的页面走了 OCR。顺带一个细节见功力：策展环节判定「扫描页」用的是双信号——文本层少于 40 字符且图像占比高，避免「整页文字配一张大照片」被误判。
第二相，策展与评分：给提取器发考卷的人要靠谱。 两个组件。稀缺优先采样：语料 90% 是纯文本时，随机采样会让扫描页、图表页样本饿死——于是按模态配额做贪心覆盖（最稀缺的模态先挑，挑的时候优先选「还能顺便满足其他配额」的页面），配六表审计工作簿保证每个选择可复现。参考评分器：归一化后算 CER/WER/BLEU/长度比，表格用 TEDS 树编辑距离，最后合成一个分数——关键一笔是乘上覆盖率：一个「跳过所有难文档、只解简单题」的模型刷不出高分。这个反作弊设计值得所有做模型排行榜的人抄走。
第三相，分块：小卡片、大文件夹。 检索的粒度和阅读的粒度不该是同一个：子卡固定 320 token（上限 512）进向量索引，父级段落（≤3500 token）放在 KV 表里；查询时命中卡片、O(1) 查出文件夹喂给模型。四条切割规则里最值得学的是「表格和图是原子的」——绝不被切断；外加 48 token 重叠种子，让跨边界的答案至少完整出现在一张卡上。每个子卡的嵌入文本前还带一个结构面包屑「文档 > 标题 > 章节 > 小节 (p.12)」——这是 contextual retrieval 的确定性版本，零 LLM 成本。
第四相，只读检索评估。 用 Mistral-Medium 从每一页生成「页内可答」的问题（严禁外部知识），用与生产同一个指纹的嵌入模型去检索，命中判据是相似度达标且页距不超过 ±1。评估器只读——它永远不可能污染线上索引。
效果到底怎么样？
- 提取（覆盖率惩罚复合分）：Gemini 3.1 Pro 97.37 分（CER 0.13%）居首；Mistral OCR v3 95.98（CER 1.39%，表格 TEDS 仍在 0.97 以上）；两个紧凑版 Docling 落后 4-7 分，差距主要在表格结构。作为参照，作者早年基于 EDD 的方案在图像表格上只有 85% 词准确率——生产后端的错误率已经低了一到两个数量级，这个领域的进展速度可见一斑。
- 延迟：180 篇文档全量提取耗时 45-57 分钟，但其中约 34 分钟是所有后端共享的预处理固定成本——92% 页面走原生路由后，模型推理已经不是瓶颈，下一个优化对象是预处理。
- 检索：25,050 道合成问题上，hit@1 68.6%、hit@10 92.8%、MRR 0.77，中位排名 1——失误集中在长尾而不是均匀分布。
能拿来做什么？
- 企业 RAG 入口管线的参考架构：四相结构、每相自带证据产物，几乎可以直接照着排开发计划。
- 省 OCR 预算：页级路由 + 双信号扫描页判定，大多数企业语料能省下九成以上的视觉模型调用量。
- 评测排行榜防刷分：复合分乘覆盖率——专治「挑着做」的模型。
- 分块策略：原子表格 + 父子解耦 + 面包屑前缀，是一套零外部依赖（不用 LLM）的确定性方案。
- 评估纪律：只读评估器 + 生产指纹复用，杜绝「评估环境与生产漂移」这类慢性病。
泼冷水时间
- 全部数字是系统自测（论文里标 [D]）：内部语料、无公开代码和数据集，复现只能靠描述；「可审计」是架构属性，不是外部验证。
- 检索评估的问题是 LLM 合成的，生成质量依赖生成器本身；hit 判据里的页距 ±1 也是设计选择。
- 只测到检索层：hit@k 和 MRR 不等于端到端答案质量，生成环节的表现未知。
- 提取 ground truth 的样本规模未披露，稀缺优先采样虽然设计精巧，但「小样本代表总体」的置信度还是取决于标注质量。
- 0.81s 的单查询延迟主要由远程嵌入调用贡献——对延迟敏感的场景，这个数字并不算好看。
理性看待
这篇论文的价值不在任何一个单点技术——路由、采样、分块、评估拆开看都是已知组件的工程化——而在于它把「摄取」从一段没人愿意认领的胶水代码，抬升成了一个有接口、有指标、有审计轨迹的一等公民阶段。三条铁律里，「不改所测」尤其值得所有数据团队自查：你的评估环境和生产环境，用的是同一个 tokenizer、同一个嵌入模型、同一套正则吗？如果不是，那些漂亮的离线指标从一开始就没有在描述你的系统。
把这个系列放在一起看会更有意思：前面几篇论文证明了「入口表示错误，下游神仙难救」，这一篇给出了入口管线的工程答案。对一个要在企业文档上做 RAG 的团队来说，这一篇可能是十五篇里最像「施工图」的一篇——它不酷，但它对齐了踢脚线。
（数据与结论均来自论文原文，解读部分为作者观点，欢迎交流。）
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
One sentence: This is an MLSys Industrial Track report by everyone who does a business RAG without a new algorithm: Three authors from Citi, An Yong and Young Weida, turn the dirty work of "how documents become models of vectors to eat" into four lines, each with evidence. The most practical figure: 92 per cent of the pages in the business language follow the original extraction of zero model costs, only 8.1 per cent need to get the OCR model out of the mountain -- if you're still brainless and full OCR, that's all there is to save.
The "Sension Level" of the Gen AI era is unattended.
In the age of traditional machine learning, data preparation is "simple": the mission is dead, the schema is dead, the data cleansing is done to target functions. Generating AI emancipated the model — a basic model for countless downstream missions — at the cost of the full complexity of the data side flowing to the portal: business knowledge is in human format, PDF, PPT, Excel, scanners, forms, blended layouts, and a document contains both text, charts, geometry and structural information.
This entry line is a bit bad, and the retreaders and retrainers can't be rescued from downstream. That's what we've tested in previous articles. But this paper asks the more advanced question: how does this entry line itself be constructed, measured and managed?
Three authors (Abbas Raza Ali of Citigroup, Muhammad Ajmal Siddiqui of Ernst and Moona Zahid of Inverda) gave a four-phase production system for the MLSys 2027 industrial track. The paper's self-location is honest: without reference to the new algorithm, it contributes to the "routing, scoring and auditing scaffolding around the existing solver". There is also a lovely self-regulation - every sentence points to the level of evidence: [E] established research, [D] system own measurements, [P] proposed in this paper, [H] assumptions. The extent to which enterprise systems can report is a demonstration in itself.
What's the idea? - Three principles + four
Three rules of design first.
- Structure before semantics: those that can be solved by the document's own structure (level of font, table line, title, page break) and not by the learning model - certainty, affordability, debugging;
- Not to change the measure: the evaluator is read-only and reuses the exact model and configuration of the production chain - the assessment must be the production system;
- Budget indications: full true values are unrealistic, with small, modelly balanced, traceable Z111Z to replace overall confidence.
Number one, extraction: make OCR an exception. Each page calculates the size ratio of an image, which exceeds 0.8 to send it to the visual model, while the rest is taken from the PyMuPDF original structure: font size calibration of title levels, block-repeated over page header footers after digitization, cable detection, wireless table word alignment grouping (with a guard "which looks like a multi-column" to prevent errors). On the 10,507 page of Orion language, only 8.1% of the page is gone. By way of illustration, the "scan page" is judged to be a double signal - a text layer less than 40 characters and a high percentage of images - to avoid a miscalculation of "the entire text with a large photograph".
The second is an exhibition and a rating: the person who gives the test to the extractor has to rely on it. Two components. Rare priority sampling: when 90% of the text is pure, random sampling kills scanned pages, chart pages, sample -- Thus, a six-table audit book guarantees that each option will be repeated. Reference scorers: CERs/WER/BLEU/Lymmetric ratio calculated after integration, tables using TEDS tree editing distance, and final synthesis of a fraction - the key one is multiplier coverage: a model that " skips all hard documents and solves only simple questions" does not have high marks. This anti-deception design is worth copying by all those who make models.
The third is a small card, a large folder. Retrieving particles and reading particles should not be the same: subcards fixed 320 token (ceiling 512) vector index, parent paragraphs (≤3500 token) placed in KV tables; Query time-hit cards, O(1) found folders fed to models. The most interesting of the four cutting rules is that "tables and maps are atoms" — never cut; plus 48 token overlapping seeds, so that cross-border answers appear in at least one card. The embedded text of each subcard is preceded by a structured bread crumb "Document > Title > Chapter > Subsection (p.12)" -- this is a definitive version of general retrieval, zero LLM cost.
Fourth, read-only search assessment. Using the Mistral-Medium to generate "respondable" questions from every page (outside knowledge is strictly forbidden) to be retrieved from an embedded model that produces the same fingerprint, with a hit rating of similarity and not exceeding ±1. The evaluator only reads -- it can never contaminate the online index.
How did it work?
- Ripping (acceleration of coverage): Gemini 3.1 Pro 97.37 (CER 0.13%) tops; Mirtral OCR v395.98 (CER 1.39%, table TEDS still above 0.97); two compact versions of Docling lags by 4-7 points, the difference being mainly in the table structure. As a reference, the author's earlier EDD-based scheme had only 85% word accuracy on the image table - – The error rate at the back end of production has been lower by one to two orders of magnitude, as can be seen from the rate of progress in this area.
- Delay: 180 full-volume extraction took 45-57 minutes, of which approximately 34 minutes was the fixed cost of pre-treatment shared at all backends - 92 per cent of the page was not a bottleneck, and the next optimised target was pre-processing.
- Retrieving: 25,050 lines of synthesis, Hit@1 68.6 per cent, Hit@10 92.8 per cent, MRR 0.77, medium ranking 1 — errors focused on long tails rather than even distribution.
What can I do with it?
- Reference structure for the Enterprise RAG entrance line: a four-phase structure, each with its own evidence, can almost directly follow the development plan.
- Budget of OCR: Page + double signal scanning page determines that most business languages can save more than 90% of their visual models.
- Assessment of the cut-off points on the rankings: Composite multiplier coverage — a model that specializes in “choice to do”.
- Part-by-part strategy: Atom Table + Father and Son Decoupling + Bread Crumb Prefix is a zero-external dependency (no LLM) certainty scheme.
- Evaluation of discipline: read-only assessors + production of re-uses of fingerprints, elimination of chronic diseases such as “assessment of the environment and transport of production”.
Cold Water Time
- The full number is the system self-measurement (inverted in the paper) [D]: internal language, no open code and data set, the reproduction can only be described; "auditable" is structural property, not external validation.
- The problem with the search assessment is that LLM is synthesized and produces mass-dependent generators per se; the page spacing in the Hit judgement ±1 is also a design choice.
- Retrieval layer only: Hit@k and MRR are not equal to end-to-end answer quality and the performance of the generation chain is unknown.
- The size of sample for extraction of the ground truth is not disclosed, and while the scarce priority sampling is well designed, the confidence of "small sample represents the whole" depends on the quality of the label.
- 0.81s single query delays are largely the result of remote embedded calls - this number is not very good for delayed sensitive scenes.
A rational view.
The value of this paper is not that of any single-point technology — route, sampling, segmenting, evaluation and decomposition of known components — but rather that it moves “ingestion” from a type of glue code that no one wants to claim, to a first-class civic stage with interfaces, indicators, audit trajectories. In the three iron laws, "no change" is particularly worth checking by all the data teams: do you use the same tokenizer, the same embedded model, the same normal set of rules to evaluate and produce the environment? If not, those pretty offline indicators didn't describe your system from the start.
It would be more interesting to put this series together: the previous papers prove that "the entrance is wrong, the bottom is the fairy" and this article gives the engineering answer to the entrance line. For a team that's going to do RAG on a business file, this is probably the best one in 15 -- it's not cool, but it matches the kick line.
(The data and conclusions are taken from the original paper, read in part as the author ' s views and welcome the exchange.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.