人工智能
AI agents的土拨鼠日：在Amazon Bedrock上使用OpenAI模型的快速缓存模式
每次你离开你的agent时，它必须重读它的整个生活故事，然后进行动作。 如何唤醒记忆
在电影"土拨鼠日"中，比尔·默里的角色在每天的2月2日重现。 他醒来后在收音机，同一家客栈，以及同一个小镇上演奏同一首歌，一次又一次地因为一些电影迷估计是34年。
前一天他没有完成任何任务除非他能从头再来一次 AI agents生活于"土拨鼠日"的版本。 agent每次迈出一步(调用工具，读取结果，决定下一步要做什么)，它必须发送模型到目前为止的全部故事来保持连续性：工具定义，指令，每一次之前的工具呼叫，以及每个结果。 模型只根据您提供的输入来预测下一个符号。 它在请求之间没有内部记忆，所以它必须再读一遍所有的内容。 到了编码会话的20转，模型的重读会变为1–19转，而每一次支付这些符号的都是你。
快速缓存会打破周期。 服务存储您的输入提示( 指示、 工具和内容)，并在下次请求时以输入价格的一小部分重用。
我从头到尾，一路走来，我告诉你如何快速的缓存使成本降低并保持agents的响应性和一致性。
agent发送到它的LLM
在谈论缓存之前，它有助于研究典型的agent第一次引用大语言模型(LLM)包含什么：
- 工具列表 - agent每个函数的定义都可以调用。 运送新版本时会改变
- 指令 - 命令 - 命令开发者提示：规则，人物，政策，实例。 也改变每个发行。
— 历史 — 至今为止发生的一切：任务，模型的工具召唤，以及工具结果(每转一转).
- 新转弯最新工具结果或用户消息(每次都可能不同).
agent寄给模型
车盖下如何快速的缓冲
当一个模型读取一个提示时，它首先做一个名为预填的书写。 在预填过程中，它处理每个输入符，并构建其键和值矢量(又称键值，或KV)，它们构成了模型的工作状态。 这发生在模型写出其第一个符号之前。
快速缓存保存了该状态，包含系统即时，工具，内容，历史等。 当后期请求以相同的输入符(即相同的文本)开始时，即时缓存服务会加载保存状态而不是再次计算。 然后它只处理模型下一个有效载荷中的新内容。 快速缓存可以是隐含的，Amazon Bedrock选择可重复使用的前缀结束的地方，或者明确，您使用 poil cache breakpoint 的代码标记结束点的地方。 你们可以一起使用这两种方法，我们建议这样做。
转1转2
隐含或明确，为什么我们需要两者？
对于代理工作量，隐含的缓存让贝德罗克选择随着对话的增长而缓存什么。 通过明显的缓存，可以标记共享内容，如工具和指令，用于跨会话和旋转帮助器agents时的再利用。 你并不总需要两种形式的缓存
新的会话还可以重复使用共享的起点，而每次对话则建立在自己缓存的历史基础上。 在此帖子中，我们从第一个转弯开始，然后展示如何通过后转弯，新工具，以及跨子剂携带缓存内容。
你好，缓存世界
第一个胜利是缓存提示的静态部分：在请求中保持不变的工具和指令。 这些示例使用您现有的 Amazon Bedrock 客户端 TOOLS 和任务。 以您的真实政策和实例替换缩短的规范字符串。 可重复使用的前缀必须达到1,024个令牌才能合格。
当缓存静态内容时，使用明确模式：选择保存的前缀结束位置。
提示 cache key 告诉要共享缓存的服务。 我们回到如何使用它当我们读取缓存在新的会话。 在我们的一次测试中，使用明显的缓存，大约4000个托肯工具和指令前缀在第一次请求上写了4,120个符，在后来的请求上读了4,120个符。 任务( 或用户请求) 位于缓存边界之后，因此不会触发缓存的写入。
记住，只有当你通过完全相同的文本时，即时缓存才有效。 确保agent的指令和工具列表每次都以同样的方式序列化。 模型背景协议(MCP)服务器和其他形式的工具交付需要索引，以确保它们保持一致。 在缓存指令后，保留任何不同的内容，如日期、 用户名称或请求ID。
随着内容的增长和变化(不同的工具和内容)，使用隐含模式让服务跟随对话。 在此，第一个请求没有明确的标记。 它最新的合格消息是用户的任务，所以自动断点是在该任务之后，并包括工具、指示和任务。 该模式在以下例子中被明确显示为清晰，即使默认是隐含的。 设置商店=假名在您的应用程序中留下对话历史。 它不会禁用缓存。
在下一个回合继续执行同样的任务时，这是有用的。 一个不同的任务与旧任务中的前缀不匹配，隐含的书写并不一定只为工具和指令单独创建条目。 当您需要缩短共享前缀时，请在此使用一个清晰的标记，以便跨任务重复使用。
当agent需要共享起点和增长历史时同时使用。 从下面的代码示例中标记的开发者块开始，并用 implicity cache调用函数 ask. 请求级别模式保持隐含。 区块级标记新增是指令后的一个明确保存点。 该服务也可以通过最新的合格消息保存。 这使得共享的前缀在对话增长时仍然可用。
缓存对话历史和工具结果
该模型按名称要求工具，并提供了它们的论点。 您的申请必须执行。 python 函数并返回其结果。 以下循环使用此帮助来运行每个请求的工具。 它将模型的输出和所有工具结果附加到历史列表中，然后再次发送该列表，直到模型返回最后的答案。 在隐含模式中，Amazon Bedrock可以通过最后一个工具结果重新使用不变的前缀并缓存添加的内容。 保持较早的消息和初始的TOOLs不变。
对于后续，请附加新的用户信息，并再次给帮助者打电话。 保持较早的消息不变，使其缓存端点仍然可用。
滚开的断点：每一转都读到它面前的一切
对于单独的显式只运行，从Hello缓存世界综合实例中重现双消息历史。 这个环绕工具在输入文本块中产生结果，并在每批中标记最终结果。 早期的标记留在历史。 参见当前长对话的查询规则。
GPT-6 Astra为这六轮工作量的最好隐含结果捆绑起来：大约62%的输入成本比无缓冲运行低。 明确的滚动断点在同一模型上节省了约61%. 两人随着对话的增加而重新利用了更多的历史。 这些是工作量的结果。 测量您自己的 agent 的储蓄。
当心：
- 仅以明确模式出现的静态断点工具和指令被缓存，但不断增长的历史在每个回合(0.61)再次处理。 这个限制不适用于综合例子
- 移动单个断点可以防止缓存重复使用，并导致以明文唯一模式重复写入。 保留前一个回合的断点。
– 只有通过变化的任务来隐含写作 – 新的任务可能共享工具和指令，而不是之前保存的终点。 如果不同的任务需要重用这个较短的前缀，在共享指令后添加一个明确的标记。
当工具改变时
有时agent的工具列表会改变，或者会添加新的工具。 保持初始列表的稳定：添加，删除，或重排定义可以使缓存前缀无效。 选择适合工具如何可用的方法。
当已知工具在 agent 流量的不同步骤中被引用时：发送同样排序的 TOOLS 并使用允许的 tools 来限制模型的称谓。 使用允许的 tools,"mode":"auto"让模型决定是否调用允许的工具，而"mode":"required"需要至少一个允许的工具调用。 将此作为工具选择。 限制不会改变缓存目录：
会议期间的新工具。 Amazon Bedrock Runtime和Amazon Bedrock Mantle端点都接受一个输入项目类型，称为附加工具，用于GPT-6 Astra,GPT-6.1 Sol，以及GPT-5.6 Luna. 关于AWS区域提供的模型可用性，参见AWS区域在Amazon Bedrock中支持的模型。 在助手函数中注册新工具 get dead date execution 函数，然后从 agent 循环将其计划附加到已完成的历史中。 保持初始TOOLs不变：
新工具在会中到达
继续工具执行循环与新响应。 循环新增模型输出到历史，运行任何请求的工具，并附加其结果。 在以后的请求中保留附加的 tools项，以便保留新工具。 在隐含模式下，Amazon Bedrock可以通过新的用户消息缓存，包括之前的工具定义。 在明文唯一模式中，保留较早的断点，并在该用户消息的输入文本块中添加一个。 附加工具本身不能有断点。
不同的角色，不同的目录。 当不同的agent角色需要不同的工具时，在每个会话开始时选择一个稳定的工具配置，如支持或计费。 每个配置文件都得到了自己的缓存前缀。 标记共享的指令，因此以后的任务与该配置文件可以重用，然后随着对话的增长使用其中任一模式。
缓存工具配置
副代理和副电话
授权不应该让每个助手重读相同的背景。 共享前缀，然后在末尾添加每个助手的任务。 注意在使用AI agents的缓存时，工具和系统提示会泄露或被利用。 建立负责的人工智能做法，包括迅速实施，作为治理的一部分。
热一次，然后扇出。 先发送一个普通请求，等待在发布帮助者之前完成。 在每个请求中明确标记共享上下文，即使模式是隐含的：
帮助者使用与热身相同的TOOLS，型号，和关键。 以输出盖结束的热身仍然可以充充缓存。 用户检查在帮助者开始前确认缓存活动。
子代理：温暖共享前缀一次，然后扇出
给一方打电话相同的开始。 agent循环通过附加每个resp.output及其匹配的工具结果，包括加密推理来构建历史。 父母完成后，复制该完整清单供摘要或法官使用。 同一帮助者使工具、模型和密钥保持不变：
帮助者可以重用历史的缓存部分，但附加的开发者消息不会自动扩展缓存以包含摘要指令。 如果以后要求再使用，请输入相同的历史和指示
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock
Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again.
In the movie Groundhog Day, Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years.
Nothing he accomplished the day before carries over unless he works it out again from scratch. AI agents live in a version of Groundhog Day. Every time an agent takes a step (calls a tool, reads the result, decides what to do next), it has to send the model its entire story so far to maintain continuity: the tool definitions, the instructions, every previous tool call, and every result. The model only predicts the next token based on the input you provide. It has no internal memory between requests, so it must read all of it again. By turn 20 of a coding session, the model re-reads turns 1–19, and it’s you who pays for every one of those tokens, again and again.
Prompt caching breaks the cycle. The service stores your input prompt (instructions, tools, and content) and reuses it on the next request, at a fraction of the input price.
In this post, I walk you through an agent’s life, from its first turn to when it gets new tools and spawns helpers. Along the way, I show you how prompt caching keeps costs down and keeps agents responsive and consistent.
What an agent sends to its LLM
Before talking about caching, it helps to look at what a typical agent’s first invocation to the large language model (LLM) contains:
- Tool list – The definitions of every function the agent can call. Changes when you ship a new release.
- Instructions – The developer prompt: rules, persona, policies, examples. Also changes per release.
- History – Everything that happened in this session so far: the task, the model’s tool calls, and the tool results (which grows every turn).
- New turn – The latest tool result or user message (which might be different every time).
What an agent sends to the model, turn after turn if you don’t set up caching:
How prompt caching works under the hood
When a model reads a prompt, it first does a write called prefill. During prefill, it processes every input token and builds its key and value vectors (also known as key-value, or KV), which form the model’s working state. This happens before the model writes its first token.
Prompt caching saves that state, which contains the system prompt, tools, content, history, and so on. When a later request starts with the same input tokens (that is, the same text), the prompt caching service loads the saved state instead of computing it again. It then processes only what is new in the next payload to the model. Prompt caching can be implicit, where Amazon Bedrock selects where the reusable prefix ends, or explicit, where your code marks that endpoint using prompt_cache_breakpoint. You can use both approaches together, and we recommend doing so.
Turn 1 compared to turn 2
Implicit or explicit, and why do we need both?
For agentic workloads, implicit caching lets Bedrock choose what to cache as the conversation grows. With explicit caching, you can mark shared content, such as tools and instructions, for reuse across sessions and when spinning up helper agents. You don’t always need both forms of caching, but combining them gives you a reusable starting point while Bedrock manages the growing conversation automatically.
New sessions can also reuse that shared starting point while each conversation builds on its own cached history. In this post, we start with the first turn, then show how to carry cached content through later turns, new tools, and across subagents.
Hello cached world
The first win is to cache the static part of the prompt: the tools and instructions that stay the same across requests. These examples use your existing Amazon Bedrock client, TOOLS, and task. Replace the shortened INSTRUCTIONS string with your real policies and examples. The reusable prefix must reach 1,024 tokens to qualify.
When caching static content, use explicit mode: choose where the saved prefix ends.
The prompt_cache_key tells the service which requests are meant to share a cache. We come back to how to use it when we get to reading the cache in new sessions. In one of our tests using explicit caching, the roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 tokens on later requests. The task (or user request) comes after the cache boundary, so this doesn’t trigger a write to the cache.
Keep in mind that prompt caching works only when you pass exactly the same text. Make sure that the agent’s instructions and tool lists are serialized the same way every time. Model Context Protocol (MCP) servers and other forms of tool delivery need to be indexed to make sure they stay consistent. Keep anything that varies, such as the date, the user’s name, or a request ID, after the cached instructions.
As the content grows and varies (different tools and content), use implicit mode to let the service follow the conversation. Here, the first request has no explicit marker. Its latest eligible message is the user’s task, so the automatic breakpoint comes after that task and covers the tools, instructions, and task together. The mode is shown explicitly in the following example for clarity, even though implicit is the default. Setting store=False leaves conversation history in your application. It doesn’t disable caching.
This is useful when the next turn continues the same task. A different task doesn’t match the prefix through the old task, and the implicit write doesn’t necessarily create a separate entry for only the tools and instructions. Use an explicit marker there when you need that shorter shared prefix to be reusable across tasks.
Use both when the agent needs a shared starting point and a growing history. Start with the marked developer block in the following code example and call the function ask_with_implicit_cache. The request-level mode stays implicit. The block-level marker adds an explicit save point after the instructions. The service can also save through the latest eligible message. This keeps the shared prefix available while the conversation grows.
Caching conversation history and tool results
The model requests tools by name and supplies their arguments. Your application must execute them. execute_function is a helper you build to route those requests to the matching Python functions and return their results. The following loop uses this helper to run each requested tool. It appends the model’s output and all tool results to the history list, then sends that list again until the model returns a final answer. In implicit mode, Amazon Bedrock can reuse the unchanged prefix and cache the added content through the last tool result. Keep earlier messages and the initial TOOLS unchanged.
For follow-ups, append a new user message and call the helper again. Keep earlier messages unchanged so their cached endpoints remain available.
Rolling breakpoints: Each turn reads everything before it
For a separate explicit-only run, recreate the two-message history from the combined Hello cached world example. This loop wraps tool results in input_text blocks and marks the final result in each batch. Earlier markers stay in history. See the current lookup rules for long conversations.
GPT-6 Astra tied for the best implicit result in this six-turn workload: about 62 percent lower input cost than running without caching. Explicit rolling breakpoints saved about 61 percent on the same model. Both reused more history as the conversation grew. These are workload results. Measure your own agent’s savings.
Watch out for:
- Only a static breakpoint in explicit-only mode – The tools and instructions are cached, but the growing history is processed again on every turn (0.61). This limitation doesn’t apply to the combined example, which also saves through eligible conversation endings.
- Moving a single breakpoint can prevent cache reuse and cause repeated writes in explicit-only mode. Keep the previous turn’s breakpoint too.
- Only an implicit write through a changing task – A new task might share the tools and instructions but not the previously saved endpoint. Add an explicit marker after the shared instructions if different tasks need to reuse that shorter prefix.
When tools change
Sometimes an agent’s tool list changes, or new tools get added. Keep the initial list stable: adding, removing, or reordering definitions can invalidate the cached prefix. Choose the approach that fits how the tool becomes available.
When known tools are invoked in different steps of the agent’s flow: Send the same sorted TOOLS and use allowed_tools to limit what the model can call. With allowed_tools, "mode": "auto" lets the model decide whether to call an allowed tool, and "mode": "required" requires at least one allowed tool call. Pass this as tool_choice. The restriction doesn’t change the cached catalog:
New tools during a session. Both Amazon Bedrock Runtime and Amazon Bedrock Mantle endpoints accept an input item type called additional_tools for GPT-6 Astra, GPT-6.1 Sol, and GPT-5.6 Luna. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock. Register the new tool get_due_date in the helper function execute_function, then append its schema to the completed history from the agent loop. Keep the initial TOOLS unchanged:
A new tool arrives mid-session
Continue the tool-execution loop with the new response. The loop adds the model’s output to history, runs any requested tools, and appends their results. Keep the additional_tools item in later requests so the new tool remains available. In implicit mode, Amazon Bedrock can cache through the new user message, including the tool definitions that precede it. In explicit-only mode, retain earlier breakpoints and add one to an input_text block in that user message. The additional_tools item itself can’t hold a breakpoint.
Different roles, different catalogs. When different agent roles need different tools, choose a stable tool profile, such as support or billing, at the start of each session. Each profile gets its own cached prefix. Mark the shared instructions so later tasks with that profile can reuse them, then use either mode as the conversation grows.
Cached tool profiles
Subagents and side calls
Delegating shouldn’t make every helper reread the same background. Share the prefix, then add each helper’s task at the end. Be aware that when you use caching with AI agents, tools and system prompts can be leaked or exploited. Establish responsible AI practices that include your prompt caching implementation as part of that governance.
Warm once, then fan out. Send an ordinary request first and wait for it to finish before launching helpers. Mark the shared context explicitly in every request, even when the mode is implicit:
The helpers use the same TOOLS, model, and KEY as the warm-up. A warm-up that ends at the output cap can still populate the cache. The usage check confirms cache activity before the helpers start.
Subagents: Warm the shared prefix once, then fan out
Give side calls the same beginning. The agent loop builds history by appending every resp.output and its matching tool results, including encrypted reasoning. After the parent finishes, copy that complete list for a summarizer or judge. The same helper keeps the tools, model, and key unchanged:
The helper can reuse the cached portion of history, but the appended developer message doesn’t automatically extend the cache to include the summary instruction. If later requests reuse the same history and instruction, place the instruction
Auto-fetched and translated by AI Atlas. All rights belong to the original author.