多式联运agents通常在每个动作前生成自由形式推理。 对于小型模型来说，有限的模型能力可能导致冗长的推理，为行动生成提供很少有用的指导，同时引起大量的推论成本。 为了应对这一挑战，我们采用了基于选择的结构合理性(SSR)，这个框架将推理重新设定为选择，而不是不限成员名额的一代。 作为预先指定、可重复使用的自然语言候选人，安全部门改革是反复出现的高级推理。 每转弯时，模型会根据当前上下文中的可能性从推理候选人中选择，没有辅助任务头。 使用预先指定的推理痕迹可以进行平行的评分，由教师强制进行预填计算，在考生内部和之间同时使用共享上下文 KV 缓存计算符号的可能性。 我们利用2B和4B模型，对七种多式搜索基准测试进行了安全部门改革评估。 通过多种强化学习目标和监督微调，安保部门改革在不牺牲任务业绩的情况下实现重大增效。 安全部门改革取得平均成功率，与同样规模的领先搜索agents竞争 将每个问题模型推论的延迟率减少90%以上，将每个问题模型推论的延迟率减少28-54%。
- 卡内基梅隆大学
- 伊利诺伊大学
- ** Apple部分完成的工作
- *** Apple期间完成的工作
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their likelihoods given the current context, without an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%.
- † Carnegie Mellon University
- ‡ University of Illinois Urbana-Champaign
- ** Work partially done while at Apple
- *** Work done while at Apple
Auto-fetched and translated by AI Atlas. All rights belong to the original author.