💡 一句话总结：视觉语言模型做 OCR 有个顽固毛病——看到图片里印错的词，会顺手「纠正」成语言上通顺的说法，把「读」干成了「改写」。这篇论文先实锤了一个反直觉现象：学生越好，老师的指导越容易帮倒忙，然后据此给「RL + 蒸馏」联合训练装上三个开关：掌握即关闭、越好越减弱、分歧大就缓一缓——CHAOS-Bench 扰动词召回从 51.47% 提到 59.92%，通用解析还不掉分。
导语：你的 OCR 模型，是个「过度热情的校对」
来做个思想实验：往一页论文的截图里，把「important」改成「imgortant」再喂给 OCR 模型，它输出什么？
很多模型会一本正经地输出「important」——它没在读图，它在校对。错别字、形近字、生僻词，只要语言模型那部分觉得「这不像人话」，就自动替换成通顺版本。对通用文档这叫鲁棒，对 archival（档案级）、合规、文献数字化场景这是灾难：源文件本来就印错了，你替它改对了，错就永远流传下去了。研究者给这个现象起了个名字：语言先验幻觉——读图变成了隐式改写。
那怎么让模型学会「图上写的什么就抄什么，哪怕看着不像话」？Infinity-Parser 团队的 GAD-RL 给了一个很「教练视角」的答案，而且它对问题的诊断比药方本身更有意思。
想自己动手试？
- 📄 论文：arXiv 2609.38282
这篇论文到底想解决什么问题？
先明确优化目标：OCR 转写忠实度——当视觉证据和语言先验打架时，保留图中实际内容；同时普通文本、表格、版面的解析质量不能倒退。前者看 GlitchText 和 CHAOS-Bench（往真实学术版面里注入字符腐蚀，考「扰动词召回」），后者看 OmniDocBench。
训练路线上，纯 GRPO（组相对策略优化，一种按组内相对奖励更新策略的 RL 算法）有个天然短板：强化学习只能奖励「采到的好输出」，那种极少被采样到的忠实转写，得不到直接的正向强化。于是很自然的想法是加在线蒸馏（OPD）：冻结一个老师，在学生自己生成的前缀上逐 token 给监督，把老师分布里的好候选「喂」给学生。
但直接把两者加权一混（GRPO+OPD 固定权重），效果平平——为什么？这就是这篇论文最有价值的发现。
一个反直觉的发现：学生越好，老师越碍事
作者拿一个冻结老师给 GRPO 各阶段的 checkpoint 离线打分（1000 页固定分析集），数着 token 看「老师的话到底是帮忙还是捣乱」，结果发现三层递进的事实：
- 训练越往后，老师信号的信噪比越低：学生扰动词召回从 7.18% 涨到 98.06% 的同时，方向性监督信噪比从 12.57 掉到 0.114。
- 老师越来越爱在学生做对的地方指手画脚：在学生已经正确转写的扰动词位置上，老师的 Top-1 建议与标准答案冲突的比例，从第 100 步的 9.52% 涨到第 300 步的 42.88%。
- 同一个 checkpoint 里也一样：把样本按组均奖励分箱，奖励越高的组，老师的有益信号越稀少，而有害信号并不同步减少。
一句话总结：老师的指导是有「适用期」的。学生还在错的时候，老师句句箴言；学生快会了的时候，老师的「纠正」有一半是在把对的改错。固定权重蒸馏的问题不在于加了蒸馏，而在于它对所有学生状态一视同仁。
那怎么办？答案不是「换个更好的老师」，而是让老师学会看学生脸色。
它的思路是什么？
GAD-RL = GRPO + 状态感知的在线蒸馏，核心是三个层次的调节，外加一个稳定器。
🚪 第一层：掌握即关闭（何时蒸馏）
一个 rollout 组里，只要任何一个输出拿到了接近满分的任务奖励（≥0.95，容忍空格、标题层级这类小格式差异），说明学生在当前输入上已经「会了」——这一组直接关闭蒸馏。都答对了还硬灌老师分布，纯属制造噪声和任务目标冲突。
📉 第二层：越好越弱（多强蒸馏）
二值开关太粗：快学会的组和一塌糊涂的组不该用同样强度。GAD-RL 用组均奖励算一个连续衰减因子——组均奖励越高，蒸馏强度越低，从 1 连续滑到 0。论文算了笔账：组均奖励到 0.9 时，蒸馏强度只剩基础系数的 1.83%。和第一层合起来，就是「全会的组不教，快会的组轻声说，不会的组重点讲」。
🌊 第三层：分歧大就缓（怎么稳）
这里有个容易被忽略的工程坑：前向 KL 蒸馏（让学生的分布去贴老师的分布）在师生分歧大的位置会产生比策略梯度强得多的梯度——尤其当学生对老师推的候选几乎没概率时，一个局部更新可能把策略带崩。GAD-RL 的稳定器叫 SA-FKL：用学生分给老师 Top-1 候选的概率去乘 FKL 梯度——学生越不认可老师的候选，这一步的更新幅度越被压小。只调幅度、不改方向，stop-gradient 保证权重本身不被优化。
顺便说下这个老师怎么来的：和学生同底座，在 20 万条转写样本上微调，输入是「参考转写文本 + 保持内容不变的改写指令」——老师不看图，只对照参考答案给建议，避免老师自己也被语言先验带偏。任务奖励也做了混搭：普通文档按编辑相似度，扰动文档按「编辑相似度 + 扰动词召回」加权。
效果到底怎么样？
双底座验证（Qwen3.5-2B 和 Qwen3-VL-2B），300 步短训，主表很干净：
- CHAOS-Bench 扰动词召回：GRPO 51.47% → GAD-RL 59.92%（+8.45pp）；固定权重 GRPO+OPD 只有 55.49%。Qwen3-VL-2B 上同样 +3.82pp。
- GlitchText：异常识别率 93.39（GRPO 91.30），过度纠正率 3.40（SFT 是 10.71）——「手痒改字」的毛病实实在在被摁住了。
- OmniDocBench v1.6：91.18，比 GRPO 的 90.80 还略高——治忠实度没有以牺牲通用解析为代价，这是这类专项训练最容易翻车的地方。
训练动态图也很有讲头：前期 GRPO 领先，GAD-RL 后程反超；随着训练推进，被门控关闭的组比例一路上升，剩余活跃组的平均衰减因子从 0.12 降到 0.02——整个训练过程就是一部「逐步放手」的教学史。
消融还有两个实用结论：其一，固定权重 OPD 对系数极其敏感，四个测试值里只有 0.005 赢过 GRPO，而 GAD-RL 把系数放大十倍召回只掉 1.10pp——状态感知控制顺带解决了调参玄学；其二，对比两个「笨办法」——按响应低分过滤再蒸馏（+2.26pp）、随训练步数线性衰减（+1.44pp）——都不如按组内状态调节，说明「看状态」比「看进度」更对路。
能拿来做什么？
- 档案/合规场景的 OCR 后训练：只要你的业务里「原文错也得照抄」是硬需求（数字化存档、合同扫描、古籍），这套配方直接相关。
- RL+蒸馏联训的通用诊断：论文那套「量教师信噪比、数学生在正确输出上老师犯错率、按组奖励分箱」的离线分析，可以原样搬去检查你自己的蒸馏管线——如果你的蒸馏系数也是拍脑袋定的，大概率能救回一些分。
- 多任务后训练的调度思路：「掌握即撤掉辅助信号」的组级门控，对任何「主任务 + 辅助损失」的训练都有参考价值。
泼冷水时间
- 只验证了 OCR 忠实度一个场景。作者在结论里也老实承认：状态感知蒸馏控制能否推广到其他任务，未验证。
- 扰动词基准都是合成的。GlitchText 是名篇改错、CHAOS-Bench 是字符腐蚀，「真实世界源文档带错」的分布覆盖有限，落地前建议先用自己的脏数据验一遍。
- 老师依赖参考转写。整个前提是有 20 万条可信转写来喂老师——参考本身错了，老师也会跟着错，「老师比学生可靠」这个假设没有在脏参考下做过压力测试。
- GlitchText 的过度纠正率上，固定权重 OPD（3.14）其实仍低于 GAD-RL（3.40），单看这一项不是全面占优。
- 训练/评测代码承诺录用后放出，眼下还复现不了。
理性看待
这篇论文的聪明之处在于顺序：先花大力气把「固定教师监督会随学生进步而变质」这个现象用 token 级统计钉死，药方（门控、衰减、分歧加权）反而成了顺理成章的推论。它没有发明新算法，而是给「RL 和蒸馏怎么共存」这个越来越常见的问题提供了一个基于学生状态的调度视角——老师的作用不是常数，而是随学生能力动态变化的函数。
对正在做大模型后训练的团队，这大概是比数字本身更值得带走的一句话：辅助监督不是加多少的问题，是什么时候撤的问题。
（数据与结论均来自论文原文，解读部分为作者观点，欢迎交流。）
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
In summary, the visual language model OCR has a stubborn problem -- seeing the wrong words in the picture, it "corrects" into words that are consistent, and "read" becomes "rewriting." The paper starts with a counter-intuitive phenomenon: the better the student, the easier the teacher's guidance, the easier it is to help, and then the three switches for the joint RL + distillation training: mastery, the better, the lower the difference, the lower the difference - CHAOS-Bench calls back 51.47% from 51.92%, and the generic resolution does not fall off.
Introduction: Your OCR model is an overzealous proofreading
To do an intellectual experiment: In a page of the paper, replace "important" with "imgortant" and feed it to the OCR model.
A lot of models are actually going to get out of the "important" -- it's not reading, it's in school. If the part of the language model that feels "it's not human," it is automatically replaced by a smooth version. It's called a roulette for generic files, and it's a disaster for archival, compliance, digitization of literature: The source document was printed wrong, and you changed it right, and the error was passed on forever. The researchers gave a name to the phenomenon: the pre-spectrum of language -- the readings turned into a covert rewrite.
So how do you get models to learn to copy what's written on the map, even if it doesn't sound like it? The GAD-RL of the Infinity-Parser team gave a very "trainer perspective" and it was more interesting to diagnose the problem than the drug itself.
You want to try it yourself?
Thesis: arXiv 2609.38282
What is this paper trying to solve?
First, the objective is clearly to optimize: OCR transposes the level of loyalty - when visual evidence and language are fighting a priori, the actual content of the map is kept; while the quality of the analysis of ordinary texts, tables and pages cannot be reversed. The former looked at GlotchText and CHAOS-Bench (infusion of character corrosion into a true academic page, examination of "disturbation callback"), while the latter looked at OmniDocBench.
On the training course, the pure GRPO (relative strategy optimization, an RL algorithm that updates the strategy by relative reward within the group) has a natural shortboard: intensive learning can only reward "good output" that is rarely sampled for faithful writing without direct positive reinforcement. The natural idea is to add online distillation (OPD): freeze a teacher, pick up token on a prefix created by the students themselves, and "feat" the good candidates in the teacher's distribution.
But weighted them directly (GRPO+OPD fixed weights), and the effect was even — why? This is the most valuable discovery of the paper.
A counterintuitive discovery: the better the student, the more the teacher gets in the way.
The author scored checkpoint offline for all stages of GRPO by a frozen teacher, counting token, to see whether the teacher's words were helping or disrupting, and found three layers of progressive facts:
- The higher the training, the lower the signal noise of the teacher: while the student verb recall increased from 7.18 per cent to 98.06 per cent, the directional monitoring of the noise ratio fell from 12.57 to 0.114.
- Teachers are more and more inclined to point at the right places: in places where students have correctly transcribed disturbing verbs, the percentage of teacher Top-1 suggestions that conflict with standard answers has risen from 9.52 per cent of step 100 to 42.88 per cent of step 300.
- The same is true of the checkpoint: the higher the sample group, the less useful the teacher's signal, the less harmful it is.
The teacher's guidance is "applied". When the student is still wrong, the teacher says the same thing; when the student is about to meet, half of the teacher's "correction" is changing the right. The problem with fixed weight distillation is not that of distillation but that it treats all students equally.
So what? The answer is not to "get a better teacher," but to teach them to look at their students.
What's the idea?
GAD-RL = online distillation of GRPO + status perception, with three levels of regulation at the core plus a stabilizer.
First level: mastery is closed (when distillation)
In a rollout group, if any output receives a job award of close to full points (Opium 0.95, tolerance of small-format differences such as spaces, title levels), it means that the student has "yes" on the current input — this group has shut down the distillation directly. The correct answer is also the distribution of teachers, which creates noise and conflicts of mission objectives.
2nd floor: the better, the weaker.
The second-value switch is too thick: fast learning groups should not use the same intensity as a bad group. GAD-RL uses both group rewards to calculate a continuous decay factor - the higher the group rewards, the lower the distillation strength, the continuous slide from 1 to 0. The distillation strength was only 1.83 per cent of the base factor when the group was rewarded with 0.9. The first level, combined, is "the group of the plenary that doesn't teach, the group that comes to the meeting whispers, the group that doesn't focus."
Third floor: Disagreements slow down.
There's an easy-to-neglected engineering pit here: the forward distillation of KL, which creates a much greater gradient than the strategic gradient, where students are divided. – Especially when students have little chance of running for a teacher, a partial update may bring down a strategy. The GAD-RL stabilizer is called SA-FKL: this step is being reduced by taking the FKL gradient with the probability that the student will give the teacher Top-1 candidate. Only increase, not change direction, stop-gradient guarantees are not optimized per se.
By the way, how did the teacher come from the same floor as the students, fine-tuned on 200,000 samples, entered as "Refer to transliterate text + rewriting instruction to keep content unchanged" — the teacher does not look at the picture, advises against the reference answer, and avoids the teacher himself being pre-specified. Job incentives are also mixed: normal documents are similar by editing, disturbed documents are weighted by " editing similarity + callback ".
How did it work?
Double base validation (Qwen3.5-2B and Qwen3-VL-2B), 300 short steps, clear:
- CHAOS-Bench verb recall: GRPO 51.47% → GAD-RL 59.92% (+8.45pp); fixed weight GRPO+OPD only 55.49%. Qwen3-VL-2B on the same +3.82pp.
93.39 (GRPO 91.30), overcorrection rate 3.40 (SFT is 10.71) - the problem of "hand itching" is really blocked.
- OmniDocBench v. 1.6:91.18, slightly higher than GRPO's 90.80 - integrity is not at the expense of generic resolution, which is the easiest place for such specialized training to turn over.
The training dynamic chart is also well documented: the GRPO lead, the GAD-RL backtracking; as the training moves forward, the percentage of the group that is shut down by door increases, and the average decay factor for the remaining active group drops from 0.12 to 0.02 — the whole training process is a history of “step-off”.
There are also two practical conclusions on digestion: first, the fixed weight OPD is extremely sensitive to coefficients, with only 0.005 of the four test values winning GRPO, while the GAD-RL recalls the coefficient ten times magnified only 1.10 pp - state sensory control handles solve the encyclopedia; second, comparing two "dick" methods - filtering by low response score (+2.26 pp), linear decay with training steps (+1.4 pp) - — — — — 。 。 。 。
What can I do with it?
- Post-OCR training for archives/compliance scenes: this formulation is directly relevant as long as you have to copy the original language in your business (digitized archiving, contract scanning, ancient).
- General diagnosis of RL+ distillation joint training: Thesis's offline analysis of the correct output of the teacher's error rate and reward box for students can be moved to check your own distillation line - if your distillation factor is also determined by a headshot, the probability is to save some points.
- Group control of multiple post-mission training: mastering or removing auxiliary signals is of value for any training on "main mission + assistive loss".
Cold Water Time
- Just verify OCR's loyalty to a scene. In his conclusion, the author admitted honestly that the extension of state sensory distillation control to other tasks had not been verified.
- The verbs are synthetic. GlitchText is an error of name, CHAOS-Bench is a character corrosive, and the distribution of "real world source documents" is limited, and it is recommended to check with its own dirty data before landing.
- Teachers rely on reference writing. The whole premise is that there are 200,000 credible transliterations to feed teachers — the reference itself is wrong, and the teacher is wrong, and the assumption that the teacher is more reliable than the student has not been subjected to stress tests under the dirty reference.
- The fixed weight of OPD (3.14) is still lower than GAD-RL (3.40) in the excess correction rate of GlotchText, which alone is not fully advantageous.
- The training/evaluation code commitment is released and not available at this time.
A rational view.
The smart thing about this paper is the sequence: it starts with a lot of effort to crucify the phenomenon of "fixed teacher supervision changes with students' progress" with token-class statistics, and the prescription (door control, decay, division weights) becomes logical. It does not invent new algorithms, but rather provides an increasingly common question of how RLs and distillation co-exist, which is based on student status -- - The role of the teacher is not a constant but a function of the student ' s evolving abilities.
For a team that is training after LLM, this is probably more worth taking than the numbers themselves: Supplementary monitoring is not much of a question, but when.
(The data and conclusions are taken from the original paper, read in part as the author ' s views and welcome the exchange.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.