TL;DR：文档 OCR 微调有个隐藏的浪费：SFT 两轮之后模型已经基本能读对页面，剩下的错误就散落在少数位置上，可传统 SFT 还是对全页每个 token 一视同仁地算损失。OPPO 的 SP-DocReader 把训练对准「还没读对的地方」：拿上一轮模型自己生成的阅读和标准答案做最长公共子序列（LCS）对齐，只对错位 token 施加两种损失——相对打分的对比损失 + 直接纠错的保真损失。结果：Qwen3-VL-4B 上三轮自博弈 CER 直降 54%，零样本 DocVQA 反而涨了 3.67；更狠的是数据效率——30k 页标注干翻 SFT 的 120k 页。全程只训 0.5B 的 OCR 小分支，视觉编码器和语言主干纹丝不动。
先看一个训练文档 OCR 模型的日常场景。
你用两万页标注数据 SFT 了两轮，模型在验证集上的字符错误率（CER）从 5.22% 降到 2.40%。看起来不错——但那剩下的 2.40% 里，藏着的是小字号页码读串、形近字认错、公式符号张冠李戴这类「硬骨头」。你想继续压错误率，于是又加数据、加轮次，结果发现：损失的大头依然花在模型早就读对的那些 token 上。
这就是监督微调的均摊悖论：全序列损失不会告诉你哪些位置还不会。
OPPO 研究团队的这篇 SP-DocReader，给出的解法出奇地朴素：把模型自己的错误当成训练信号。
论文地址：arXiv:2610.11148
整体框架：只训 0.5B 的「OCR 外挂」
先说架构。SP-DocReader 延续 DocVLM 的模块化思路：视觉编码器和语言主干（InternVL-3.5-4B 或 Qwen3-VL-4B）完全冻结，只训练一个 OCR 分支——DocFormerV2 式的 OCR 编码器 + 投影层 + 一组可学习查询，把外部 EasyOCR 识别出的文本和坐标框压缩成紧凑表示，注入冻结的 VLM。
可训参数只有 0.5B，对比 OCR-free 全参微调的 3.3B 和完整全参的 3.8B。推理开销约为纯主干方案的 +10%。
这个设计后面会看到回报：域内 CER 略逊于全参微调，但零样本全面占优，训练成本只有全参的 1/5。
核心方法：给模型配一个「错题本」
整个训练分两段。第一段是常规的 SFT-2（两轮监督微调）。第二段开始自博弈（self-play），每轮三步：
第一步，让「上一次的自己」读一遍。 上一轮的模型固定为对手（opponent），对每页生成一份完整阅读 ŷ——里面混着它独有的错误：把 "store" 读成 "stroe"、把 "kilograms" 读成 "kilogramhs"。
第二步，LCS 对齐，圈出错误。 对标准答案 y 和生成阅读 ŷ 在模型 token 序列上做最长公共子序列对齐——两边读对的部分自然对上，剩下的未匹配位置就是「错题」。整个过程看下面这张框架图：
图里那个例子很直观：真值是 "A store ordered 8 boxes of apples..."，对手读出 "A stroe ordered 8 boxes... 15 kilograms... saled 40 kilogramhs... are lefted"——蓝色是读对的（排除出直接损失），红色的错处才进入训练。
这里有个容易被忽略但很讲究的细节：掩码只选损失位置，不截断序列。打分时错位 token 的条件前缀保持完整——匹配的 token 依然提供上下文。论文专门强调不把抽出来的残缺片段当新序列打分，避免了「在残缺文本上算概率」的语义错乱。
第三步，两种损失双管齐下。
- RDL（阅读差异损失）：SPIN 式对比目标——在这些错位上，当前模型给标准答案的分数要比对手给自己错误阅读的分数高。这是「相对」信号：参考优势越小，拉扯越用力。
- FFL（聚焦保真损失）：在标准答案的未匹配位置直接加负对数似然。这是「绝对」信号：纠正不依赖「压低错误阅读」，而是直接把对的抬起来。
为什么两个都要？消融给了干净答案：只留 FFL（去掉对比），Vary-600K CER 是 1.88；只留对比（去掉 FFL），1.57；双管齐下 1.11——而且五项指标全部最优。一个管方向，一个管力度，缺谁都使不上全劲。
每轮结束后刷新对手：新对手、新阅读、新掩码。这一步不是仪式——对照实验显示，刷新分支训 2 个局部 epoch 的平均分（90.3）超过固定对手读物的 5 个 epoch（89.8）。道理很简单：旧错题刷完了，得出新错题才有新东西可学。
成绩单：域内 -54%，零样本涨 3.67 分
Qwen3-VL-4B 上的主要结果：
注意后四行全部是零样本——自博弈只在 Vary-600K 的英中双语页面上训，泛化到学术页（DocBank）和扫描件（IIT-CDIP）照样降错误率，连文档问答都跟着涨。InternVL-3.5-4B 上趋势一致（Vary CER 2.12→1.17）。附加的三个基准也全线上扬：OmniDocBench 编辑距离 0.347→0.291，OCRBench v2 +6.2，CC-OCR +5.6。
数据效率可能是对实践者最友好的一张图：30k 页标注 + 自博弈的五任务平均分 82.46，超过 120k 页 SFT 的 80.73。标注预算紧张时，这条曲线的含义不言自明。
分辨率鲁棒性是另一个惊喜：从 48 DPI 到 150 DPI，SP 版全面占优——48 DPI 极限低清下 CER 是 1.74 对 3.41。对那些「PDF 渲染分辨率上不去」的产品场景，这直接决定了能不能省视觉 token。
模块化 0.5B vs 全参 3.8B：域内小负，零样本全胜
架构消融是全文最有意思的对照。Qwen3-VL-4B 上三种玩法：
全参微调域内 CER 最低（0.98），但那更像对训练分布的过拟合——一到零样本就掉队，而且三轮自博弈后它的零样本均分不升反降（75.92→74.98）。模块化方案域内让出 0.13 个 CER 点，换来零样本三个指标全胜和 1/5 的训练成本。
这一幕是不是有点眼熟？域内刷榜的未必是好的部署选择——在垂直能力上加装小模块，反而保住了主干的泛化底座。
错题本里都有什么错？
定性案例最能说明「聚焦残差」到底在纠什么错。看论文给的两页对比：
左边是美国宪法修正案影印页：SFT 把 "Passed by Congress December 18, 1917" 读成 December 16、把 "Repealed... 1933" 读成 1935、把 "requisite" 拼成 "requsilie"、把 "State legislatures" 误成 "legislators"。SP-DocReader 版本全部修正。
右边是代数论文页：SFT 把维度公式 dim S = dim A + l 读成大写 L、把 "see Theorem 3.14" 读成 3.11、把 join-semilattice 拼错。公式符号和定理编号这类「差一个字符就谬以千里」的位置，恰恰是错题本最该覆盖的。
泼冷水时间
外部 OCR 是拐杖。 框架依赖 EasyOCR 提供文本和坐标框，OCR 分支学的是「如何把现成 OCR 结果压好喂给主干」。EasyOCR 读错的，这套框架无能为力；推理还有 +10% 开销。跟 GOT-OCR2.0、DeepSeek-OCR 这类端到端专精系统比，这是完全不同的技术路线，论文也没有同表对比它们的绝对成绩。
只管单页转录。 论文自己的 Limitations 写得清楚：不做多页阅读、不做结构化抽取，固定生成预算对超长页面不友好。
语种与家族覆盖窄。 只测英中双语训练 + 两个 4B backbone（都是 Qwen/InternVL 系），换语种换模型家族是否成立未知。
自博弈依赖 SFT 起点。 如果 SFT 后的错误分布有系统性盲区（比如整行漏读），LCS 对齐对这种「结构性错误」的覆盖是否充分，论文没有讨论。
理性看待
把这篇放进本系列最近几篇的脉络里看，会更有意思：
- pixels-to-structure 那篇说 OCR 专用小模型做结构化抽取不行；本篇说模块化 OCR 分支做纯转录很强——任务边界再次显形，选型前先问自己要什么。
- HANS 答题卡那篇靠垂直数据全参微调拿精度；本篇证明「冻结主干+小分支+巧目标」是另一种更便宜的路。
- 训练方法层面，「SFT 之后的残差，用自博弈来收割」这个范式，本质上是把 SPIN/DPO 那套偏好优化从「整篇好坏」细化到了「token 级错位」——这个粒度迁移值得所有做生成任务微调的人参考。
一句话总结：当你的模型已经 97.6% 正确，别再全页撒网了，把错题圈出来逐个击破。
（本文是「智能文档」系列第 15 篇。相关阅读：小米 OCR-0：7B 视觉编码的知识瓶颈、PolyOCR：多语言 OCR 的专用路线。）
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
TL; DR: The document OCR fine-tune has a hidden waste: after two rounds of SFT, the model is basically able to read the pages, and the remaining errors are scattered in a few positions, but traditional SFT is the same for every token on the entire page. OPPO's SP-DocReader aligns the training to "places not read correctly": aligning the last set of models with their own reading and standard answers to the longest public sub-series (LCS), imposing two types of losses on the wrong token — the relative scoring of the relative loss + the direct correction of the right loss. Results: Qwen3-VL-4B ' s last three rounds of self-playing CERs dropped by 54 per cent and zero sample DocVQA increased by 3.67 per cent; even worse, data efficiency - 30k page marked 120k of SFT. The OCR branch, the visual encoder and the main texture line remain intact.
First look at the day-to-day scene of a training document OCR model.
You used 20,000 pages of SFT data for two rounds, and the char error rate (CER) of the model on the validation set fell from 5.22% to 2.40%. It looks good -- but the remaining 2.40 percent of it is a small-numbered page-numbered reading string, a near-word misreading, a formula symbol with a "hard bone" like that. You wanted to keep pressuring the error rate, and then you added the data, and then you added the wheel, and you found that the big loss was still on those tokens that the model had read right.
This is the balance paradox of the oversight fine-tuning: full-series losses do not tell you where it is not yet.
This SP-DocReader piece of the OPPO research team gives an amazingly simple solution: using the model itself as a training signal.
Thesis address: arXiv: 2610.1148
Overall framework: only 0.5B "OCR hanger"
First, architecture. SP-DocReader continues the modular thinking of DocVLM: the visual encoder and the language backbone (InternVL-3.5-4B or Qwen3-VL-4B) are completely frozen, training only one OCR branch - the DocFormerV2 type OCR encoder + projection layer + a set of studyable queries, compressing text and coordinate frames identified by external EasyOCR into a compact expression, injecting frozen VLM.
The training parameters are only 0.5B, which compares OCR-free with a full fine-tuned 3.3B and a complete full-blown 3.8B. The reasoning costs approximately +10 per cent of the pure backbone programme.
The design will be followed by returns: CERs in the field are slightly less than fully fine-tuned, but zero sample is fully advantageous and training costs are only one-fifth inclusive.
Core method: A "fault book" for models
The whole training was in two parts. The first is the regular SFT-2 (two rounds of oversight fine-tuning). The second stage starts with the self-play, three steps per round:
The first step, let's read "the last one himself." The last round of models was fixed as rivals, generating a complete reading of each page -- a unique error in it: reading "store" as "stroe" and "kilograms" as "kilogramhs".
Step two, LCS alignment, circle out wrong. Y for standard answers and for generation reading ŷ for maximum common sequence alignment on the token sequence -- both readings are naturally paired, and the remaining unmatched position is "wrong." The whole process looks at the following frame:
The example in the chart is intuitive: "A stone ordered 8 boxes... 15 rivals... - Blue is read right (excludes direct loss) and red wrongs get into training.
There's an easy-to-neglect, but delicate detail here: the mask only selects the location of the loss and does not cut the sequence. The misplaced token prefix remains complete - matching token still provides context. The paper focused specifically on not scoring the fractions drawn as new sequences, avoiding the semantic confusion of "probability on the missing text".
The third step is a combination of two losses.
- RDL (losses of reading differences): SPIN-style comparison target - In these misalignments, the current model gives a higher score to standard answers than the counterparty misreads itself. This is a "relative" signal: the less the reference, the harder the pull.
- FFL (focusing on the loss of authenticity): A direct negative logarithmic in the unmatched position of the standard answer. This is an "absolutely" signal: the correction does not depend on "understood reading" but lifts the right up.
Why both? The solution is clean: leave only FFL, and Vary-600K CER is 1.88; leave only comparison (leave only FFL), 1.57; and two-pronged 1.11 — and all five indicators are the best. One direction, one strength, none of us can do anything.
At the end of each round, the opponent is refreshed: new opponent, new reading, new mask. This step is not a ritual - based on a comparison of experiments, the average score of 2 local epochs (90.3) is higher than that of 5 epochs (89.8) of the fixed counterparty. The point is simple: when old mistakes are cleared, new ones come to light.
Report card: Within-area - 54%, Z111Z increase 3.67 minutes
Main results on Qwen3-VL-4B:
All four lines are zero sample. The self-playing game is conducted only on the Vary-600K bilingual page, which is extended to the academic page (DocBank) and scanned (IIT-CDIP) to reduce the error rate, and even the document questions and answers are raised. The trend is consistent on InternVL-3.5-4B (Vary CER 2.12→1.17). The three additional benchmarks are also up-line: OmniDocBench edits 0.347 →0.291, OCRBech v2 + 6.2, CC-OCR + 5.6.
Data efficiency is probably the most friendly picture for practitioners: 30k page marked + the average score for the five tasks of the self-playing game is 82.46, exceeding 120k SFT 80.73. The meaning of this curve is self-evident when budget constraints are indicated.
Another surprise is resolution ' s greatness: from 48 DPI to 150 DPI, SP editions are fully successful - 48 DPI limit low-cleaning CERs are 1.74 pair 3.41. The product scenes of the PDF rendering resolution do not go up, which directly determines whether token can save sight.
Modularization 0.5B vs full & 3.8B: In-area small negative, zero sample All in.
The breakdown of structures is the most interesting comparison of the whole text. Qwen3-VL-4B:
CERs are the lowest (0.98) in the total fine-tuning field, but it's more like a match for training distribution — sample is out of line as soon as Z to Z, and its zero Z111Z is not up or down since the three rounds (75.92.74.98). Within the modularized programme area, 0.13 CER points were given in exchange for zero sample indicators and one fifth of training costs.
Does this look familiar? It is not necessarily a good deployment option — adding small modules to vertical capabilities, but instead keeping the backbone panorama.
What's wrong with the wrong book?
Qualitative cases are the best examples of what is wrong with "focus disability". Read the two pages of the paper:
On the left is a photocopy page of the United States Constitution Amendment: SFT reads "Passed by Congress Amendment 18, 1917" to December 16, "Repealed... 1933" to 1935, "requisite" to "requisite" and "state legislatures" to "legislators". SP-DocReader Version All amended.
On the right is the algebra thesis page: SFT reads the dimension formula dim S = dim A + l to uppercase L, reading "see Theorem 3.14" to 3.11, misspelled join-semilate. The position of a formula symbol and theorem number, such as "a character missing a thousand miles", is precisely the wrong question that should be covered.
Cold Water Time
Outside OCR is crutches. The framework relies on the EASYOCR to provide text and coordinates, and the OCR branch to learn "how to feed the results to the backbone." EasyOCR reads wrong, and the frame can't do anything; the reasoning has +10% of the expenses. Compared to end-to-end precision systems such as GOT-OCR 2.0 and DeepSeek-OCR, which are completely different technical routes, the papers do not compare their absolute performance with the table.
Just a single page. The paper's own drafts are clear: not to read many pages, not to do structured extracts, and the fixed generation budget is unfriendly for pages that are too long.
Language and family cover is narrow. Only English-Chinese bilingual training + two 4B backbone (both Qwen/InternVL) is unknown.
Self-engineered game relies on SFT starting point. If there is a systematic blind distribution after SFT (e.g. a full line missing), whether the LCS alignment coverage of this "structural error" is adequate, the paper does not discuss it.
A rational view.
It would be even more interesting to see this in the context of the most recent editions of this series:
- Pixels-to-structure says that OCR's special small model can't be structured to extract; this says that modular OCR branches do a strong pure transcrib -- the mission boundary is again visible, and you ask yourself what you want before choosing.
- HANS Query Card, which uses vertical data to fine-tune fine-tuning; it proves that "Freezing the backbone + small branch + goal" is another cheaper route.
- At the level of training methods, the paradigm of "scrutinization after SFT, harvesting with a game" is essentially a refinement of SPIN/DPO's preferences from "good and bad" to "tokeen's faulty" - a particle migration that is worth reference to all those who produce fine-tuning for the task.
One sentence: When your model is 97.6% correct, stop spreading all over the web and break the wrong loop one by one.
(This is part 15 of the Smart Document series. Read: Mi OCR-0:7B Knowledge bottlenecks in visual coding, PolyOCR: Multilingual OCR.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.