在Amazon SageMaker AI上用知识蒸馏法构建高效的agents

人工智能

在Amazon SageMaker AI上用知识蒸馏法构建高效的agents

你在边疆模型上造了个购物助理，它成功了。 agent搜索目录，检查候选人，检查限制条件，并承诺推荐。 但这种互动实际上需要多少？代理助手不是一个模型调用，而是循环：搜索，读取结果，精细，验证，决定。 一个完成的任务可以是多次呼叫那个前沿模型，你支付每一个，每个shopper。 在生产规模上，这种每个任务成本的化合物侵蚀了项目的ROI，并阻止了项目的航运。

提高代理应用效率的一种方法是使用一个较小的模型，这个模型反应更快，主机成本效益更高。 渔获物是质量的：一个较小的模型通常没有完成任务，这就是为什么较大的模型首先被选中的原因。 知识蒸馏通过将更大的“教师”模式的行为转变为较小的“学生”来弥补这一差距，培训学生复制教师的产出，从而保持了较小模式的速度和成本。

在本文中，您将学习如何使用知识蒸馏，特别是序列级知识蒸馏(SeqKD)，到微调 a shopping agent on Amazon SageMaker AI. 我们介绍如何与边疆教师一起生成高质量的多步轨道，并保留解决任务的轨道，如何为监管的微调编码，如何进行无服务器基础设施的培训，以及如何对学生进行针对被搁置的基准测试的评价。

什么是序列级知识蒸馏(SeqKD)?

最简单的适用蒸馏形式是序列级知识蒸馏(SeqKD). 它使用教师模型生成高质量的响应，然后用监督的微调(SFT)在这些响应上培训学生模型。 SeqKD只需要教师的文字，没有教师日志，也没有强化学习的基础设施，因此即使有封闭的或只使用API的教师也能工作。 与基于登录或政策的其他方法相比，它位于蒸馏光谱中最具成本效益和可移植的端。 师写答曰。 师对之微细。 由于这些答案作为数据集持续存在，你可以重新使用它们来训练或测试不同的学生，而不重新生成数据。

原则上，您希望学生能与老师在可能产出上的充分分布相匹配。 这里的警告是，你不能。 可能序列的空间在天文上很大，SeqKD绕过它，与老师实际生产的样本相近。

对于agent来说，老师所制作的序列并不是一个传言的块，而是一个轨迹，一个有序的推理运行，工具呼叫，以及他们返回的观察，其中每个步骤都以它面前的一切为条件。 对学生进行关于这整个序列的培训，教他们如何推理导致每个行动以及多步骤行动方针本身，而不是一个回答文本的段落。 这使得一个代理任务成为序列级KD最干净的插图之一，因为被复制的序列是行为本身。

当 SeqKD 是合适的选择

SeqKD 符合一组具体情况：

您有投入，但没有高质量的目标产出

这是SeqKD的核心案例，它基本上是数据生成。 您提供输入，老师写目标，这样您就可以在一个任务或域上使用微调，没有现有的高质量标签。 目标可以像单一答案一样简单，也可以像多回合工具使用轨迹一样参与。 这个帖子占据了那段频谱的要求结束，代理轨迹，特别难以来源。 单任务分支进入许多部分正确的运行，告诉从失败者中解开的运行需要一个可靠的分级器。 一个基准测试，有一个程序计分器提供精确的，这样你就可以让老师生成轨迹，并且只保留那些解决任务。

您的老师是一个前沿模型你只能通过API联系

这是边境模型的正常情况。 无法亲自访问重心，这排除了基于登录的蒸馏和基于政策的方法，它们都需要内部的准入。 SeqKD只需要老师的文本响应，所以它是唯一仍然有效的方法，它让你在学生接手后

你想要一个强大，成本效益高的第一步

一些最强小推理模型是和SeqKD(在老师推理痕迹上监督微调)一起在相当大程度上构建的，因此它是设定正确时推理转移的有效起点。 DeepSeek-R1蒸馏模型是一个众所周知的例子：一位大老师的推理被微调在老师生成的推理痕迹上蒸馏成较小的Qwen和Llama学生。

先决条件

要运行这个例子，你需要：

- AWS账户，由SageMaker AI的AWS身份与访问管理(IAM)角色负责，该用户已读写Amazon简易存储服务(Amazon S3)桶，并允许引用您的老师模型。

- 一个边疆教师模型，以生成轨迹(这个穿行方式使用Amazon Bedrock上的一条)。

- 在支持的AWS区访问SageMaker AI模型定制。

- Amazon SageMaker Studio(或配置了AWS CLI的本地IDE)，运行一个Python 3.12或更高内核，拥有SageMaker Python SDK v3和boto3(pin your version). 对笔记本环境至少使用ml.m5.4x大例，因为数据编制和本地评分步骤运行在它的CPU上。

- Python的工作知识，监督微调和工具使用agents。

- 附带的代码寄存器和带有程序计分器的任务(这次走过使用购物奔驰测试集). 代替你的任务和分级员。

解决方案概述

这条路从头到尾一直走着 Amazon Bedrock上的边疆教师GLM-5.3担任购物奔驰agent产生轨迹。 基准测试自己的计分器只保留那些解决任务的计分器，而那些解开的轨迹微调 Qwen3.5-4B学生在SageMaker AI模型定制上的无服务器监督微调。 蒸馏学生随后被评价为被搁置的购物奔驰分裂，并服务于运行工具循环的SageMaker AI推断实时端点后。

准备数据集

测试值得模仿的轨迹是困难的，购物奔驰计分器是将成功者与失败者分开的。 购物奔驰出版750个合成购物问题，产品，商店，和券意向各250个，每个都有隐藏的黄金答案和决定成功的因素。 我们曾将它们分成426个训练问题和324个被搁置的评价问题(产品103、商店105、凭证116)

对于每一个训练问题，一个边疆教师GLM-5.3在Amazon Bedrock上用1.0的推理和温度作为购物奔驰agent. 它搜索大约270万个产品的目录，检查候选人，推荐符合限制条件的产品。 我们每个问题都给老师做两次样本，给850个轨迹，温度1.0是故意的。 抽样而不是贪婪的解码使得两次尝试探索了不同的询问和推理路径，这使得更多的问题至少一个解决的轨迹，以及一个更多样化的集合来学习。 这个拒绝-抽样步骤将一个有能力但又不完善的老师变成了一个干净的训练场景，因为老师只需要经常正确，解决的跑步就值得模仿。

每个破解的轨迹都是完整的一集：shopper请求，老师在每个步骤(搜索，检查，推荐，终止)的推理和工具呼叫，以及观察环境返回。 我们把推理和行动放在一起，“结果”显示，是学生最常走的路。

培训的编码轨迹

序列级的蒸馏是在已经存在的轨迹上监督学习，因此在训练期间没有任何东西称它为工具，应用程序只见过文本。 由于这些轨迹是多步工具使用的对话，我们把每个轨迹编码为聊天转折的信息列表，让学生自己的聊天模板制作出来，因此教练看到的符号与agent推论产生的符号相符。

每个轨迹成为每步一列训练。 用户转身携带系统即时和谈话至今。 助手转过身来提出这个步骤的推理，损失落在助手的转弯上，所以模型学会产生下一个推理与动作，而不是预测环境返回的观测。 轨迹有数步扩展进入数行。 在此解析的轨迹扩展至过滤后的约2600个训练和130个验证行。

单行将到目前为止的对话(用户)与下一步学习(助手)联系起来：

以后的每一个步骤都成为自己的一行，新的观察附在用户内容上，下一个推理和工具调用作为助理目标，一直到最后推荐产品和终止调用。 使用您的模型的精确聊天模板逐行渲染。

按问题，而不是按行分割数据。 老师每个问题被取样两次，每个轨迹扩展分为几个步骤，因此许多行都有一个问题，一行平分会漏出几乎相同的背景，贯穿培训和验证。 持有整个问题(约5%) 保持验证的诚实。

每步编码的成本是重复的上下文，因为每行重复了不断增长的对话，而没有服务器的微调会截断任何过去的数据集 max len，所以一行跨越极限会失去它需要的部分内容。 与其让限制决定什么被削减，不如把排数控制在预算范围内。 最有效的修补方式是压缩工具观察，它支配着符号计数。 对于产品搜索结果，只保留下一个决定取决于的字段(id，价格，评分，以及正在检查的限制)，放下原始的上市文本，并限制每次搜索保留多少结果。 如果轨迹仍然超出预算，则上限为每轨迹的阶梯。 这里的dataset max len是16,384个令牌，这些令牌在观测经过修剪后符合大多数的轨迹。

在JSONL每次拆分在S3之后，将每个拆分作为定制数据集资产与DataSet注册。 从 SageMaker Python SDK v3 创建，它返回 train dataset 和 val dataset 所消耗的训练任务。

微调学生

训练使用SageMaker AI模型定制，无服务器的SFT. 你从模型中选择一个基模型，将任务指向编码数据集，管理服务提供计算，运行任务，并返回定制模型，在Amazon SageMaker中用ML流跟踪测量。 由于数据已经以支持的格式存在，因此没有训练脚本，容器图像，也没有实例可以管理。

该任务将其输出写入一个模型包组，该组将它所制作的每个模型进行版本，因此首先用模型包组(ModelPackageGroup.create)创建该组(如果名称已经存在的话再使用)并将其传递给训练员。 然后配置定制任务。 选择基本模型和培训类型，将其指向列车和验证数据集加模型包组，然后命名一个 MLflow e

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

Artificial Intelligence

Build efficient agents with knowledge distillation on Amazon SageMaker AI

You built a shopping assistant on a frontier model, and it works. A shopper asks for “a black waist bag under twenty dollars from a shop with at least 4.5 stars,” and the agent searches the catalog, inspects candidates, checks the constraints, and commits to a recommendation. But what does that interaction actually cost? An agentic assistant is not one model call, it is a loop: search, read results, refine, verify, decide. A single completed task can be multiple calls to that frontier model, and you pay for every one, on every shopper. At production scale, that per-task cost compounds, erodes the project’s ROI, and stops the project from shipping.

One way to make an agentic application more efficient is to use a smaller model, which is faster to respond and more cost-effective to host. The catch is quality: a smaller model usually falls short on the task, which is why the larger one was chosen in the first place. Knowledge distillation closes that gap by transferring the behavior of the larger “teacher” model into a smaller “student”, training the student to reproduce the teacher’s outputs, so it keeps much of the quality at the smaller model’s speed and cost.

In this post you will learn how to use knowledge distillation, and specifically sequence-level knowledge distillation (SeqKD), to fine-tune a shopping agent on Amazon SageMaker AI. We cover how to generate high-quality multi-step trajectories with a frontier teacher and keep the ones that solve the task, how to encode them for supervised fine-tuning, how to run the training on serverless infrastructure, and how to evaluate the student against a held-out benchmark.

What is sequence-level knowledge distillation (SeqKD)?

The simplest applicable form of distillation is sequence-level knowledge distillation (SeqKD). It uses a teacher model to generate high-quality responses, then trains a student model on those responses with supervised fine-tuning (SFT). SeqKD needs only the teacher’s text, with no access to teacher logits and no reinforcement-learning infrastructure, so it works even with a closed or API-only teacher. Compared to other approaches such as logit-based or on-policy, it sits at the most cost-effective and portable end of the distillation spectrum. The teacher writes the answers, and the student is fine-tuned on them. Because those answers persist as a dataset, you can reuse them to train or test different students without regenerating the data.

In principle, you would want the student to match the teacher’s full distribution over possible outputs. The caveat here is that you can’t. The space of possible sequences is astronomically large, so matching it exactly isn’t feasible. SeqKD sidesteps that by approximating it with samples the teacher actually produces.

For an agent, the sequence the teacher produces is not a block of prose but a trajectory, an ordered run of reasoning, tool calls, and the observations they return, where each step is conditioned on everything before it. Training the student on that whole sequence teaches it the reasoning that leads to each action and the multi-step course of action itself, not a single paragraph of answer text. That makes an agentic task one of the cleanest illustrations of sequence-level KD, because the sequence being copied is the behavior itself.

When SeqKD is the right choice

SeqKD fits a specific set of situations:

You have the inputs but no high-quality target outputs

This is the core case for SeqKD, which is fundamentally data generation. You supply the inputs, and the teacher writes the targets, so you can fine-tune on a task or domain that has no existing high-quality labels. The targets can be as simple as single answers or as involved as multi-turn tool-use trajectories. This post takes the demanding end of that spectrum, agentic trajectories, which are especially hard to source. A single task branches into many partially correct runs, and telling a solved run from a failed one takes a reliable grader. A benchmark with a programmatic scorer supplies exactly that, so you can let the teacher generate trajectories and keep only the ones that solve the task.

Your teacher is a frontier model you can only reach through an API

This is the normal case for frontier models. You can’t access the weights to host it yourself, you can’t read its logits, and you can’t fine-tune it. That rules out logit-based distillation and on-policy methods, which all need that internal access. SeqKD needs only the teacher’s text responses, so it is the one approach that still works, and it frees you from paying frontier prices on every request after the student takes over.

You want a strong, cost-effective first step

Some of the strongest small reasoning models were built in significant part with SeqKD (supervised fine-tuning on teacher reasoning traces), so it is an effective starting point for reasoning transfer when the setup is right. The DeepSeek-R1 distilled models are a well-known example: the reasoning of a large teacher was distilled into smaller Qwen and Llama students by fine-tuning them on the teacher’s generated reasoning traces.

Prerequisites

To run this example, you need:

- An AWS account with an AWS Identity and Access Management (IAM) role for SageMaker AI that has read and write access to an Amazon Simple Storage Service (Amazon S3) bucket and permission to invoke your teacher model.

- A frontier teacher model to generate the trajectories (this walkthrough uses one on Amazon Bedrock).

- Access to SageMaker AI model customization in a supported AWS Region.

- Amazon SageMaker Studio (or a local IDE with the AWS CLI configured), running a Python 3.12 or higher kernel, with the SageMaker Python SDK v3 and boto3 (pin your versions). Use an ml.m5.4xlarge instance at minimum for the notebook environment, since the data preparation and local scoring steps run on its CPU.

- Working knowledge of Python, supervised fine-tuning, and tool-using agents.

- The accompanying code repository and a task with a programmatic scorer (this walkthrough uses the ShoppingBench test set. Substitute your own task and grader).

Solution overview

This section walks the pipeline from end to end. A frontier teacher, GLM-5.3 on Amazon Bedrock, acts as the ShoppingBench agent to generate trajectories. The benchmark’s own scorer keeps only the ones that solve the task, and those solved trajectories fine-tune a Qwen3.5-4B student with serverless supervised fine-tuning on SageMaker AI model customization. The distilled student is then evaluated on a held-out ShoppingBench split and served behind a SageMaker AI Inference real-time endpoint that runs the tool loop.

Preparing the dataset

Sourcing trajectories worth imitating is the hard part, and the ShoppingBench scorer is what separates the successful runs from the failed ones. ShoppingBench publishes 750 synthesized shopping problems, 250 each for the product, shop, and voucher intents, each with a hidden gold answer and the constraints that define success. We split them once into 426 training problems and 324 held-out evaluation problems (product 103, shop 105, voucher 116), as fixed indices so the two sets don’t overlap.

For each training problem, a frontier teacher, GLM-5.3 on Amazon Bedrock with reasoning on and temperature 1.0, acts as the ShoppingBench agent. It searches the catalog of about 2.7 million products, inspects candidates, and recommends the ones that meet the constraints. We sample the teacher twice per problem, which gives 850 trajectories, then let ShoppingBench’s own scorer keep only the ones that solve the task. Temperature 1.0 is deliberate. Sampling rather than greedy decoding makes the two attempts explore different queries and reasoning paths, which gives more problems at least one solved trajectory and a more diverse set to learn from. This rejection-sampling step turns a capable-but-imperfect teacher into a clean training set, since the teacher only has to be right often enough that the solved runs are worth imitating.

Each solved trajectory is a complete episode: the shopper request, the teacher’s reasoning and tool calls at each step (search, inspect, recommend, terminate), and the observation the environment returned. We keep the reasoning alongside the actions, which the Results show is what carries the student most of the way to its teacher.

Encoding trajectories for training

Sequence-level distillation is supervised learning over trajectories that already exist, so nothing calls a tool during training, and the application only ever sees text. Because these trajectories are multi-step tool-use conversations, we encode each as a messages list of chat turns and let the student’s own chat template render it, so the tokens the trainer sees match the tokens the agent produces at inference.

Each trajectory becomes one training row per step. The user turn carries the system prompt and the conversation so far. The assistant turn carries that step’s reasoning and the tool call it leads to, a … block followed by a [{…}] in the harness’s native form. Loss falls on the assistant turn only, so the model learns to produce the next reasoning-and-action, not to predict the observations the environment returned. A trajectory with several steps expands into several rows. Here the solved trajectories expand to about 2,600 training and 130 validation rows after filtering.

A single row pairs the conversation so far (user) with the one next step to learn (assistant):

Each later step becomes its own row, with the new observation appended to the user content and the next reasoning-and-tool-call as the assistant target, through to the final recommend_product and terminate calls. Render every row with your model’s exact chat template.

Split the data by problem, not by row. The teacher is sampled twice per problem, and each trajectory expands into several steps, so many rows share one problem, and a row-level split would leak near-identical context across training and validation. Holding out whole problems (about five percent) keeps the validation set honest.

The cost of per-step encoding is repeated context, because each row restates the growing conversation, and serverless fine-tuning truncates anything past dataset_max_len, so a row that crosses the limit loses part of what it needs. Rather than let the limit decide what gets cut, keep rows within budget. The most effective fix is to compress tool observations, which dominate the token count. For product search results, keep only the fields the next decision depends on (id, price, rating, and the constraints being checked), drop the raw listing text, and cap how many results you retain per search. If trajectories are still over budget, cap the steps per trajectory. Here dataset_max_len is 16,384 tokens, which fits most trajectories after observations are trimmed.

After the JSONL for each split is in S3, register each as a customization dataset asset with DataSet.create from the SageMaker Python SDK v3, which returns the train_dataset and val_dataset the training job consumes.

Fine-tuning the student

Training uses SageMaker AI model customization with serverless SFT. You select a base model from the model hub, point the job at the encoded datasets, and the managed service provisions the compute, runs the job, and returns the customized model, with metrics tracked in Amazon SageMaker with MLflow. Because the data is already in a supported format, there is no training script, container image, or instance to manage.

The job writes its output into a model package group that versions every model it produces, so create the group first with ModelPackageGroup.create (reusing it if the name already exists) and pass it to the trainer. Then configure the customization job. Choose the base model and training type, point it at the train and validation datasets plus the model package group, then name an MLflow e

Auto-fetched and translated by AI Atlas. All rights belong to the original author.