AI Agent 乱调工具，靠改 prompt 治不好。实测数据摆在这里：模型单工具调用准确率约 95%，工具数涨到 20 个以上就掉到 65%–78%，而规模化后 60%–75% 的失败根本不是选错工具，是参数填错。这篇按「根因 → Schema 约束 → 分类隔离 → 权限白名单 → 运行时风控 → 高危二次确认」拆成四层工程防线，每层都给可落地的判断依据，也说清每层在什么情况下会失效。
目录
- 工具为什么会乱调
- 症状对照表
- 第一层：Schema 约束
- 第二层：工具分类与暴露收敛
- 第三层：权限白名单
- 第四层：运行时风控
- 第四层的最后一环：高危工具二次确认
1. 工具为什么会乱调（Why Agents Mis-call Tools）
先把责任分清楚：模型不是「偶尔手滑」，它是 结构性地 不可靠。
它更像一个记性极好、语感极强、但从不承认自己会错的外包同事。你让他「把报告表整理一下」，他能在零点几秒内产出一份格式漂亮的执行计划；问题在于，他手上同时握着「读报表」和「清空日志」两把钥匙，而他真心认为后者也算「整理」。
这个比喻不是修辞。Salesforce AI Research 的 ToolScan（工具调用错误诊断基准）把工具调用错误拆成七类，覆盖 10 类环境、30 多个任务、150 条人工标注查询，结论是 连最强的模型也会稳定复现这些错误模式（Kokane 等，ICLR 2025 Workshop，DOI：10.48550/arXiv.2411.13547）：
- Insufficient API Calls（IAC，调用次数不足）：该拆成多次调用的任务，只调了一次，任务没做完。
- Incorrect Argument Value（IAV，参数值错误）：参数值填错，也包括漏掉必填参数。
- Incorrect Argument Name（IAN，参数名错误）：幻觉出不存在的参数名，比如要 device_id 却写 equipment_id。
- Incorrect Argument Type（IAT，参数类型错误）：布尔值写成字符串 "True"，列表写成逗号串。
- Repeated API Calls（RAC，重复调用）：用完全相同的参数反复调用，纯属烧钱。
- Incorrect Function Name（IFN，函数名错误）：调了一个根本没注册的工具，比如系统里只有 cancel_order，它生成 refund_order。
- Invalid Format Error（IFE，格式非法）：输出结构本身就不合法，解析器直接报错。
七类里，IAV、IAN、IAT、IFE 四类都发生在 参数层面。这跟规模化之后的比例对得上：第三方对 BFCL（Berkeley Function Calling Leaderboard，伯克利函数调用排行榜）v3 的 2026 年 5 月快照做过一次汇总，规模化部署下 参数不匹配占工具调用失败的 60%–75%，比选错工具更高发（Presenc AI 汇总；原始榜单见 BFCL）。
同一份汇总里还有两组数字值得记住：
以及链式调用的复合衰减：单次 90% 的成功率，连调 3 次是 \(0.9^3 \approx 73\%\)，5 次是 \(0.9^5 \approx 59\%\)，10 次只剩 \(0.9^{10} \approx 35\%\)。三个数字都能自己按幂次复算，这也是「生产环境偏爱 3–5 步短链」的直接原因。
工具变多为什么会更差？2026 年 6 月的 ToolChoiceConfusion 论文给了个干净对照实验：102 个任务、100 个工具、4 个模型后端、2448 次运行。把 100 个工具全暴露时，任务成功率 0.83、平均每次任务 1.25 次错误调用；只暴露当前步因果必需的那一个工具时，成功率 0.99、错误调用 0.01，token 消耗还降了约 90%（arXiv:2606.06284）。
结论很清楚：模型永远会犯错，工程必须兜底。 这跟会议里那句「90% 的开发者只会写几句描述，靠模型自觉」是同一件事的两种说法——把可靠性寄托在 prompt 上，等于把生产环境的稳定性赌在一次采样上。
2. 症状对照表（Symptom, Cause, Fix）
下面这张表是本文的索引。左边是你线上看到的现象，中间是它真正的成因，右边是对应的防线层。建议先按现象定位，再跳到对应章节。
表里最后一行的「第七层」笔误不必纠正——它想说的是第四层的最后一环，也就是第 7 章。这也顺带说明一件事：防线是可以分的，章节编号是会错的，所以兜底必须写在代码里，不能写在文档里。
3. 第一层：Schema 约束（Schema Constraints）
第一层解决的是「参数根本不该进来」。它的定位是闸机，不是保安：不判断意图，只判断格式。
3.1 工具描述要能回答四个问题
很多团队的工具定义长这样：
{"name": "query_data", "description": "查询数据"}
这句话对模型的信息量约等于零。模型不知道它查什么、什么时候该调、有没有副作用，于是只能按字面猜。可用的描述至少要回答四件事：
- 这个工具能做什么；
- 它 不能 做什么（这条最常被漏掉，也最能减少误选）；
- 什么场景下该调；
- 有没有副作用、是否可撤销。
对应的实例：
{
"name": "query_report",
"description": "按报表 ID 查询已生成的报表内容。仅用于读取，不修改、不删除任何数据，也不触发重新生成。当用户询问报表内容、下载报表时调用。",
"parameters": {
"type": "object",
"properties": {
"report_id": {"type": "string", "description": "报表 ID，格式为 RPT- 加 6 位数字，例如 RPT-000123"},
"max_rows": {"type": "integer", "minimum": 1, "maximum": 5000, "description": "本次返回的最大行数"}
},
"required": ["report_id"],
"additionalProperties": false
}
}
additionalProperties: false 这一行是关键。它把「模型幻觉出一个新字段」从「静默通过」变成「显式失败」。
3.2 Schema 只管概率，代码才管确定性
必须说清楚一个边界：Schema 是给模型看的 说明书，不是执行时的 强制约束。模型完全可以在 JSON 里写出 "max_rows": "十"，而 JSON Schema 校验器只有在服务端真的跑一遍校验时才会拒绝它。
所以参数进入业务函数之前，必须再走一次程序化校验。Python 生态里最省事的是 Pydantic（数据校验库）：
from datetime import date
from pydantic import BaseModel, Field, model_validator
class ReportQueryInput(BaseModel):
"""报表查询参数模型，负责类型、范围与跨字段约束校验。"""
report_id: str = Field(pattern=r"^RPT-\d{6}$") # 报表 ID 强格式约束
start_date: date # 起始日期，自动解析 ISO 格式
end_date: date # 结束日期，自动解析 ISO 格式
@model_validator(mode="after")
def check_range(self):
"""校验日期区间合法性，避免把空结果当成「无数据」上报。"""
if self.start_date > self.end_date: # 起始晚于结束属于业务非法
raise ValueError("start_date 不能晚于 end_date") # 抛出后由上层转成结构化错误
return self # 校验通过则原样返回
3.3 别名归一化：只修可以安全修的那部分
实测中最常见的坑不是类型错，是 同义不同名：工具要 entity，模型给 entities、products、product_names。
这类差异语义等价，可以在校验前做一层映射：
PARAM_ALIASES = { # 仅收录语义完全等价的同义字段
"entities": "entity", # 复数形式归一
"products": "entity", # 业务别名归一
"product_names": "entity", # 更长的业务别名归一
}
def normalize_arguments(raw: dict) -> dict:
"""把同义参数名映射为工具定义的规范名，返回新字典不修改入参。"""
return {PARAM_ALIASES.get(key, key): value for key, value in raw.items()} # 命中别名则替换，否则保留原名
但归一化必须有纪律：只处理含义明确、可以安全转换的字段。如果把所有「看起来像」的字段都自动纠正，真正的意图错误会被悄悄抹平——用户要的是 A 对象，模型填了 B 对象，你把它归一成 A 的字段名，错误就从「参数名不匹配」升级成了「静默处理了错误对象」，后者更难查。
这一层的边界到此为止。它保证「进来的参数是合法的」，但完全不保证「这个工具该不该被调用」。后者交给第二层。
4. 第二层：工具分类与暴露收敛（Tool Isolation）
第一层管「参数对不对」，第二层管「这个工具凭什么出现在候选集里」。
4.1 按副作用分三档
工具不是生而平等的。读一个报表和执行一条 shell 命令，风险差好几个数量级，但在很多实现里它们被塞进同一个 tools 数组平铺下发，模型看到的是一份没有权重的清单。
按副作用分档是成本最低的改进：
这个三档划分和网络安全行业实践一致（参考 FreeBuf 的 Agent 权限模型实践），也与国家标准的口径吻合：全国网络安全标准化技术委员会《网络安全标准实践指南——智能体交互安全要求（征求意见稿 v0.23）》在 8.2 条明确要求「工具提供者应实施最小接口暴露原则，仅向智能体暴露最小必要的功能接口」，并要求「涉及高危系统权限的工具调用时，充分告知用户安全风险，获取用户明示同意」（原文下载）。
4.2 高危工具不进默认候选集
回到会议里那个真实事故：用户说「报告表整理一下」，模型调用了删除工具，清空日志数据。
这不是模型「理解错了」，而是工程「给错了」。当一个会话的候选集里同时有 query_report 和 delete_all_logs，你等于在问模型一道二选一的题，而它答错的概率在 20 个工具规模下是 20%–35%。
正确的做法是 默认只暴露 L0。L1、L2 通过显式授权或独立通道引入，而不是一开始就摆在桌面上。CMTF 的对照实验已经证明这条路有效：100 个工具全暴露是 0.83 成功率、1.25 次错误调用；只暴露当前步必需的工具是 0.99 成功率、0.01 次错误调用（arXiv:2606.06284；同一实验的完整参数见第 1 章，此处为同一组数字的二次引用）。
4.3 大规模工具集用「两段式路由」
工具超过 20 个时，单一模型直接选工具的准确率会掉到 65%–78%（BFCL v3 快照数据）。这时应改成两段式：
用户请求
→ 路由模型（只在 L0 中挑 5–10 个候选，不含高危工具）
→ 执行模型（只在候选集中选具体工具并生成参数）
→ 第一层 Schema 校验
→ 第三层权限校验
路由模型的职责是 收敛候选集，不是执行。这样一个职责拆分，把「100 选 1」降级成「10 选 1」，同时也把高危工具从执行模型的视野里彻底摘掉了。
这套思路和 Skill 路由是同一个问题：候选集越大，命中率越低。在 AI03 那篇《Agent Skill 路由设计》里我们讨论过大规模 Skill 的命中率保障，结论可以平移过来——收敛暴露面，比优化提示词有效得多。
5. 第三层：权限白名单（Permission Whitelist）
第二层解决「候选集里有什么」，第三层解决「这个人能调什么」。
5.1 默认拒绝，白名单优先
权限判断的顺序必须是：先查硬禁名单（Denylist，如生产环境禁用 shell 执行），再查白名单（Allowlist），两者都未命中一律拒绝。不是黑名单没命中就放行——那是默认允许，等于没有防线。
TOOL_PERMISSION_LEVELS = { # 工具名 → 风险等级映射，未登记的视为不可用
"query_report": 0, # L0 只读，直接放行
"write_report": 1, # L1 写入，需授权
"delete_all_logs": 2, # L2 高危，默认不暴露
}
DENIED_TOOLS = {"delete_all_logs"} # 生产环境硬禁名单，优先级最高
def check_tool_permission(tool_name: str) -> dict:
"""按默认拒绝策略判断工具是否可被当前执行器调用。"""
if tool_name in DENIED_TOOLS: # 硬禁名单优先，直接拒绝
return {"allowed": False, "reason": "denied_by_denylist"} # 返回结构化拒绝原因便于审计
if tool_name not in TOOL_PERMISSION_LEVELS: # 未登记工具一律不放行
return {"allowed": False, "reason": "not_in_allowlist"} # 避免模型幻觉工具名直接穿透
level = TOOL_PERMISSION_LEVELS[tool_name] # 取出风险等级
if level == 0: # L0 只读
return {"allowed": True, "mode": "direct"} # 直接执行
if level == 1: # L1 写入
return {"allowed": True, "mode": "authorized"} # 需主体已获授权
return {"allowed": True, "mode": "confirm_required"} # L2 高危，必须二次确认
if __name__ == "__main__": # 直接运行即可验证三条分支
for name in ("query_report", "write_report", "delete_all_logs", "refund_order"): # 四个典型工具名
print(name, "->", check_tool_permission(name)) # 逐条打印判定结果
运行输出：
query_report -> {'allowed': True, 'mode': 'direct'}
write_report -> {'allowed': True, 'mode': 'authorized'}
delete_all_logs -> {'allowed': False, 'reason': 'denied_by_denylist'}
refund_order -> {'allowed': False, 'reason': 'not_in_allowlist'}
注意最后一行：refund_order 从未登记，按「未登记即拒绝」被挡下。这就是默认拒绝与默认允许的分界——如果当时写的是「不在黑名单就放行」，模型幻觉出来的这个工具名会直接穿透到执行层。
5.2 权限要绑到「主体 + 会话」
只在全局配一份工具清单是不够的。同一个系统里，普通用户和运维管理员看到的候选集应该不同，临时会话和长期会话的授权也应该不同。
判据有三条：
- 主体维度：普通用户无 L2 权限，管理员全开——做不到这一点，等于所有用户共享管理员的能力。
- 会话维度：会话开始时授予一次权限，会话内不动态提升，防止中途被诱导提权。
- 凭证维度：每个工具使用独立、短时效的凭证，而不是一个全局万能 token。
这套逻辑不是 AI 时代的新发明。NIST SP 800-53 的 AC-6（Least Privilege，最小权限）早就把「只授予完成任务所必需的权限」写成了控制项；OWASP 在 LLM06:2025「Excessive Agency（过度代理）」里把同一个原则翻译到了大模型场景，核心建议就是限制工具与权限范围（AI RiskAtlas 的工具管控条目 对这两条做了逐条映射）。
值得单独提一句 2026 年 1 月的 AgenTRIM：它做的是运行时逐步最小权限——离线从代码与执行轨迹重建工具接口，运行时按每步动态过滤可用工具并做状态感知校验，在 AgentDojo 基准上显著压低了攻击成功率同时保持任务完成度（arXiv:2601.12449）。这条路线说明「权限」不必是静态配置，可以是随任务状态流动的。
6. 第四层：运行时风控（Runtime Guardrails）
前三层都是 调用前 的静态约束。第四层管的是调用 过程中 的失控。
6.1 前置意图预判：先判断「该不该调」，再判断「调哪个」
最容易被忽略的一类错误是：用户根本没要求执行，模型却执行了。
「取消订单会产生费用吗？」——这是在问规则，不是在要求取消。但模型看到一个 cancel_order 工具，很容易顺手调下去。ToolScan 把这类归入「不应该调用工具时仍然调用」，它不在七类参数错误里，危害却往往更大。
工程上可以用确定性规则做前置拦截：
import re
INTERROGATIVE_PATTERN = re.compile(r"(会不会|是否|能不能|要多少钱|会产生|规则|政策)") # 询问规则的表达模式
def looks_like_inquiry(user_input: str) -> bool:
"""判断输入是否只是在询问规则而非下达执行指令。"""
return bool(INTERROGATIVE_PATTERN.search(user_input)) # 命中询问模式返回 True
命中询问模式时，把 L1、L2 工具从候选集里临时摘掉，只留 L0 检索类工具。这是一条「宁可少做，不可做错」的规则——取证式提问和命令式请求，风险不对称。
6.2 熔断：三道保险丝
只做意图预判不够，模型还会在失败后反复重试。需要三道保险丝：
第一道：重复调用指纹。 对 (工具名, 规范化参数) 做哈希，同一指纹在一次任务内出现超过 2 次即熔断。这一道直接针对 ToolScan 的 RAC 类别。
第二道：最大轮次上限。 给任务设一个硬上限，比如 MAX_ITERATIONS = 2 次纠错重试。超过就停止，返回明确失败原因。理由是收益递减：参数错误重试一次可能修好，权限不足重试一百次也不会成功，只会把延迟和 token 成本一起推高。
第三道：错误分类驱动的差异化处理。 把错误分成四类，分别走不同路径：
这里有个容易做错的细节：错误信息 不要 统一压成「工具调用失败，请重试」。这个字符串对模型来说零信息量，它会原地重试。错误信息越具体，模型自我修复的成功率越高；反之，模糊错误 + 无上限重试 = 日志爆炸 + 成本飙升，正好是会议里提到的那条事故链。
这一层的最后一道闸，是任何静态规则都替代不了的部分：高危操作的二次确认。它单独成章。
7. 第四层的最后一环：高危工具二次确认（Human-in-the-loop）
7.1 为什么必须做
删除、修改、批量操作、数据清空——这些工具一旦误调，损失不可逆。国家标准对此的要求是硬性的：《智能体交互安全要求》8.2 g 明确要求高危系统权限的工具调用必须「充分告知用户安全风险，获取用户明示同意」。
原则很清楚：AI 无权自己执行高危操作。Agent 只负责生成操作预览——做什么、为什么、影响范围多大——由用户确认后才真正执行。
7.2 但二次确认不是万能药（反直觉数据）
这里必须泼一盆冷水，否则第 7 章会变成一篇安全安慰剂。
2026 年多项研究指向同一个结论：人类在审批环节的表现远比我们想象的差。
- 一项约 4 万场次、40.9 万次批准/拒绝决策的公开实验显示，人类审批者整体准确率只有 66.3%，约每 3 个威胁漏掉 1 个，且 32.9% 的场次「净收益为负」——放过了威胁、又误拦了安全命令（explainx.ai 数据整理）。
- 一项 1053 名测试者参与的 Claude Code 现场实验里，面对注入的危险命令，人工拦截率仅 13.6%，而同一场景下后台自动分类器的拦截率是 89%（分析）。
- 更糟的是「Invisible Ink Threats」那类攻击：攻击者把恶意目标藏在合法任务背后，此时 HITL（Human-in-the-loop，人在回路）不但没降低攻击成功率，反而平均升高 7.8%，个别配置下从 21.2% 翻到 42%（arXiv:2608.02018）。
上述前两条均为二手转述，原始实验报告未完整公开，引用时请按「方向性证据」使用，不要当精确基准。但三条方向一致，足以支撑一个判断：确认框的质量，决定了二次确认是防线还是橡皮章。
7.3 有效确认框的四个要素
所以第 7 章的正确写法不是「加个确认弹窗就完事」，而是给出确认框的设计约束：
- 给操作预览，不给抽象提问。 不要问「是否允许执行 delete_all_logs」，要写「将删除 /var/log/app 下 3 个文件、共 1.2 GB，操作不可撤销，最近一次备份在 3 天前」。
- 给影响范围，且可核对。 影响的对象数量、涉及的表/目录、是否会触发下游任务——这些必须来自真实的预检，不是模型口述。
- 默认拒绝，而不是默认同意。 弹窗的安全动线应该让「不同意」成为默认选项，避免用户习惯性回车。
- 合并同类项。 连续弹出 20 个确认框，用户必然无脑点同意；批量操作应聚合成一次确认，并明示总量。
第 4 条尤其重要：审批疲劳是 66.3% 和 13.6% 这两个数字最直接的解释。确认次数越多，单次确认的信息价值越低。
7.4 正确的优先级：最小权限 > 二次确认
最后回到一个工程判断上：二次确认应该是 最后一道 防线，不是第一道。
顺序反了会出问题。如果高危工具本来就在候选集里、权限也没收敛，那你只能靠人工审批去兜；而人工审批的准确率已经被上面三组数据否掉了。反过来，如果第二层就把高危工具移出默认候选集、第三层就把权限绑到主体，那么需要人工确认的场景会被压缩到极少数——这时候确认框稀少、信息完整，人工判断才真正有效。
这也解释了第 5 章为什么要强调默认拒绝：每减少一次不必要的确认，剩下的确认就更有分量。
这套四层防线适合工具数超过 5 个、且存在至少一个不可逆操作的生产系统；如果你的 Agent 只接两三个只读工具，加这么多层反而拖慢响应，先把 Schema 校验做扎实就够了。
本专栏《AI Agent 工具》聚焦 tool calling 的可靠性治理，从工具描述、候选集收敛、权限管控一路讲到运行时风控与人工确认，每篇只拆一个能落地的问题，每周更新一篇。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
AI Agent mistook the tool by changing prompt. The measurements are here: the model sheet tool call has an accuracy rate of about 95 per cent, the number of tools has fallen to more than 20, 65 per cent to 78 per cent, while the failure of 60 per cent to 75 per cent of the scale is not a wrong tool at all, but a wrong parameter. This article is broken down into four layers of the project line, each of which gives a determinable basis of judgement and explains the circumstances under which each layer will fail.
Contents
- Why are the tools so messy?
- Symptoms
- First level: Schema binding
- Second tier: tool classification and exposure
- Third floor: permission white list
- Level four: Runtime wind control.
- Last ring on the fourth level: high-risk tool confirmed twice.
1. Why Tools Mis-call Tools
Let's start with clear lines of responsibility: the model is not "sliding occasionally," it's structurally unreliable.
It is more like an outsourced colleague with an excellent memory and a strong tone, but never acknowledging that he would be wrong. You asked him to "assess the report form" and he was able to produce a good-format implementation plan in a few seconds; the problem was that he was holding both the "read the report" and the "empty log" keys, which he really thought were "packaged".
It's not a metaphor. ToolScan of Salesforce AI Research (tool call bug diagnostic benchmark) breaks tool call errors into seven categories, covering 10 environmental groups, over 30 tasks, 150 manual tagging queries, concluding that even the strongest models will stabilize these error patterns (Kokane et al., ICLR 2025 Workshop, DOI: 10.48550/arXiv.2411.13547):
- Insufficient API Calls (IAC, insufficient number of calls): this is to be split into multiple calls, only once, and the task is not completed.
- Interrection Argument Value (IAV, error in parameter values): Error in parameter values, including missing required parameters.
- Incorporate Argument Name (IAN, parameter name error): hallucinating non-existent parameter names, for example, to device id and write equipment id.
- Incorporation Argument Type (IAT, parameter type error): Boolean values are written as a string "True", a list is written as a comma string.
- Repeated API Calls (RAC, duplicate call): repeated calls with exactly the same parameters, only burning money.
- IncorpFunction Name (IFN, function name error): transfers a tool that is not registered at all, such as cancel order, which generates refund order.
- Invalid Format Error (IFE, illegal format): The structure of the output itself is not legal, and the solver reports errors directly.
In the seven categories, IAV, IAN, IAT, IFE, four are at the parameter level. This compares to the scale: A third party summarised the May 2026 snapshot of the BFCL (Berkeley Faction Calling Leaderboard, Berkeley Function Called List) v3 with a scalable deployment of parameters that did not match 60-75 per cent of the tool call failure and were more hairy than the wrong tool (Precenc AI summary; the original list is available in BFCL).
In the same summary, there are two groups of figures worth remembering:
success rate, 90% one time, 3 times three times (0.9^3\approx 73\\), 5 times (0.9^5\approx 59\\), 10 times only. The three figures can be calculated on their own, and this is the direct reason why the production environment favours the 3-5-step short chain.
Why are the tools worse? The ToolChoiceConfusion paper in June 2026 gave a clean comparison experiment: 102 missions, 100 tools, 4 model backends, 2448 operations. success rate 0.99, misdirected 0.01, token consumption was also reduced by about 90 per cent (arXiv: 2606.06284), when the 100 tools were fully exposed and mission success rate 0.83, with an average of 1.25 calls per mission; when only one tool required for current step factors was exposed.
The conclusion was clear: the model would always be wrong and the project would have to be bottom-up. It's the same thing as the "90% developer" in the conference that says, "The model's self-conscious" -- putting reliability on the prompt -- it's like putting stability in the production environment on a single sample.
Symptom, Cause, Fix
The following table is an index to this paper. On the left is what you see on the line, in the middle is the real cause of it, and on the right is the corresponding line of defense. It is suggested to position itself by phenomenon and then jump to the corresponding section.
There is no need to correct the "seventh floor" errors in the last line of the table: It's about the last ring of the fourth floor, chapter 7. Here's one thing: the lines can be divided, the chapter numbers are wrong, so the bottom must be in the code, not in the document.
3. First tier: Schema Constraints
The first level of solution is that "the parameters should never have come in." Its location is a brake, not a security guard: it does not judge intent, it judge format.
3.1 Tool description to answer four questions
Many team tools are defined as follows:
{name: query data, description: Query data}
The amount of information on the model is about zero. The model does not know what it looks at, when it should be adjusted, or whether it has side effects, so it can only be assumed literally. The description available must answer at least four things:
- What can this tool do?
- It can't do anything.
- What's going on?
- There are no side effects. Can they be revoked.
Corresponding examples:
{
"name": "query_report",
"description": "As a report ID queryes the generated report contents. is only for reading, does not modify, delete or trigger a regeneration. Call when the user asks about the contents of the report, downloads the report. " I don't know.
"parameters": {
"type": "object",
"properties": {
"report id": {"type": "string", "description": "Report ID, format RPT-plus 6 bits, e.g. RPT-000123"},
== sync, corrected by elderman == @elder man
},
"required": ["report_id"],
"additionalProperties": false
}
}
This line is critical. It turned "a new field in a model illusion" from "silent passage" to "visibility failure".
3.2 Schema is only certain if it's probability.
It has to be clear: Schema is a description of the model, not a mandatory constraint at the time of execution. The model is perfectly capable of writing "max rows" in JSON: "Ten", and the JSON Schema checker will reject it only if the service does run it over.
So the parameter must go another programmination check before entering the business function. Python's eco-efficient is Pydantic:
from datetime import date
from pydantic import BaseModel, Field, model_validator
class ReportQueryInput(BaseModel):
""Report query parameter model, responsible for type, range and cross-field binding."
Report id:str = Field
Start date: date#start date, automatically parsing ISO format
End date: date# end date, automatically parsing ISO format
@model_validator(mode="after")
def check_range(self):
""Checks the validity of dates and avoids reporting empty results as "no data."
# Starting after the end of business is illegal
"Start date"
Return self # verified returns as it is
3.3 Harmonization of aliases: only those parts that can be repaired safely
The most common pit in the field is not a type error, but a synonym: the tool is an entity, the model is given to entities, products, product names.
The semantic equivalent of these differences can be mapped before the school:
PARAM ALIASES = { # Only synonyms for full parity field
"entities": "entity,"
# Business aliases
# Longer business aliases
}
def normalize_arguments(raw: dict) -> dict:
""Imaging synonyms as a tool-defined norm, returns the new dictionary without modification."
{PARAM ALIASES.get(key, key): value for key, value in raw.items()} # Replace with an aliases if you do not keep the original name
However, decriminalization must be disciplined: only field, which has a clear meaning and is safe to convert. If all the fields that look like are automatically corrected, the real error of intent will be quietly erased. The user wants an A object, the model fills an B object, and you add it to the A name of field, and the error goes from "parametric mismatch" to "quietly processed the wrong object", which is more difficult to trace.
This layer of boundary ends here. It guarantees that "the parameters for entry are legal", but it simply does not guarantee that "this tool should not be called." The latter is to the second floor.
4. Second tier: Tool Classification and Exposure
The first is the "parameters right" and the second is the "how does this tool get in the pool?"
4.1 Three tranches by side
Tools are not born equal. Reads a report and executes a shell command, with several orders of magnitude at risk, but in many cases they are pushed down into the same array of arrays, and the model sees a list without weight.
Slotting by side is the least costly improvement:
This three-tier classification is consistent with the practice of the cybersecurity industry (cf. the FreeBuf Agent permission model practice) and is in line with national standards: the National Technical Committee for the Standardization of Cybersecurity (CNS) Guide to Practice on Cybersecurity Standards — AI agent Interactive Security Requirements (Approval Draft v. 0.23), in article 8.2, explicitly requires that “tool providers should apply the minimum interface exposure principle and only the minimum required functional interface should be exposed to AI agent”, and requires that “tool call, which involves high-risk systems permission, fully inform users of safety risks and obtain express user consent” (scanning original).
4.2 High-risk tools are not default candidates Set
Back at the meeting, the real accident: the user said, "Check up the report form," the model uses the delete tool to empty the log data.
It's not a "misunderstanding" model, it's a "mistake." When there's both query report and delete all logs in the list of candidates for a session, you're asking the question of double-checking the model, and the probability that it's wrong is 20%-35% in 20 tool sizes.
The right thing to do is to default only L0. L1, L2 was introduced by explicit authorization or independent channel, not on the table at the beginning. The CMTF control experiment has proven to be effective: 100 tools were all exposed to 0.83 success rate, 1.25 calls; only the tools necessary to expose the current step were 0.99 success rate, 0.01 calls (arXiv: 2606.06284; for full parameters of the same experiment see Chapter 1, here are two references to the same set of numbers).
4.3 Large-scale tool assembly "two-part route"
When the tool exceeds 20, the accuracy of the single model 's direct selection tool falls to 65%-78% (BFCL v3 snapshot data). This should be replaced by two paragraphs:
User Request
• Route model (only 5-10 candidates in L0, excluding high-risk tools)
• Implementation model (only candidate for specific tools and generate parameters)
First Level Schema Verify
permission, third floor.
The route model's job is to draw up candidates, not execute them. Such a duty, split, downgrades "100, one" to "10, one" and removes high-risk tools from the perspective of the implementation model.
This idea is the same thing as Skyll's route: the bigger the candidate pool, the lower the hit. We've been talking about a massive Skill hit rate guarantee in the AI03 version of "Agent Skill Route Designs," and the conclusions can be smoothed -- to contain exposure, which is much more effective than optimising prompt.
5. Third floor: permission White List (Permission Whitelist)
The second tier deals with "what's in the pool" and the third with "what can this man do?"
5.1 Default rejection, white list priority
The order in which permission must be determined must be the hard-listed list (Denylist, e.g., a ban on shell execution in the production environment) followed by the Allowlist, both of which were rejected without a hit. It's not a blacklist that's released — it's a default, it's a no-line.
TOOL PERMISSION LEVELS = {Toolname → Risk class map, not registered as not available
"query report": 0, #L0 read-only, just release
"write report": 1, #L1 written with authorization
"delete all logs": 2, #L2 high risk, default not exposed
}
DNIED TOOLS = {"delete all logs"} # Production environment hard-listed, highest priority
def check_tool_permission(tool_name: str) -> dict:
"""The tool will be called by the current implementer according to the default rejection strategy."
If tool name in DENIED TOOLS: # hard-listed priority, directly rejected
♪ Back to structured reasons for rejection ♪
If tool name not in TOOL PERMISSION LEVELS: # Unregistered tools are not released Okay.
♪ To avoid a model of hallucinating toolnames
= TOOL PERMISSION LEVELS[tool name]# Take out the risk level
# L0 read-only
# Direct execution
If level = 1: # L1 written
# The subject is mandated
# L2 high-risk.
If name = = " main ": # Directly run can verify three branches
# Four typical tools First Name
Print decision by article
Run output:
query_report -> {'allowed': True, 'mode': 'direct'}
write_report -> {'allowed': True, 'mode': 'authorized'}
delete_all_logs -> {'allowed': False, 'reason': 'denied_by_denylist'}
refund_order -> {'allowed': False, 'reason': 'not_in_allowlist'}
Note that the last line: refund order has never been registered and is blocked by the "not registered or refused". This is the boundary between default refusal and default permission - If it was written "free without a blacklist," this tool name from a model illusion would go directly through the executive level.
5.2 permission to bind to "Main + Session"
It is not enough to have a global list of tools. In the same system, common users and transport administrators should see different sets of candidates and different authorizations for ad hoc and long-term sessions.
There are three convictions:
- Main dimensions: General users do not have L2 permission, administrators open - failure to do so is equal to the ability of all users to share administrators.
- The dimensions of the session: permission was awarded at the beginning of the session, which did not rise dynamically and prevented the right to be induced in the middle.
- Documentary dimensions: each tool uses an independent, short-lived document rather than a global almighty token.
This logic is not the new invention of the AI era. The NIST SP 800-53 AC-6 (Least Privilege, mini. permission) has long written "only the permission necessary to accomplish the task" as control item; the OWASP translated the same principle into LLM scene at LLM06:225 "Exceptive Agency (Extreme Agent), the core recommendation being to limit the tool to the permission (Ai Risk Atlas tool control entry) to a cross-section of the two articles.
It is worth mentioning separately the January 2026 AgenTRIM: it does a step-by-step minimum of permission-offline to re-engineer tool interfaces from code to execution track, operates with a dynamic filtering tool available for each step and a state verification, and significantly lowers the attack on Z51Z while maintaining mission completion on the AgentDojo benchmark (arXiv: 2601.12449). This route indicates that "permission" does not have to be static configuration, but can flow with the task.
6. Fourth tier: Runtime Wind Control (Runtime Guardrails)
The first three floors are static restraints before calling. The fourth layer is called in the course of the out-of-control.
6.1 Pre-intentional presupposition: first to judge whether or not to change, then to judge which.
One of the most easily ignored types of errors is that users do not ask for implementation at all, but the model is implemented.
"Is there a cost to cancel the order?" -- this is a question of rules, not a request for cancellation. But the model sees a cancel order tool, and it's easy to move on. ToolScan classifies this as "no longer called when tools should not be called", which is not in the seven categories of error, but is often more harmful.
The project could be pre-stopped with certainty rules:
import re
# Ask the rule expression
def looks_like_inquiry(user_input: str) -> bool:
""to judge whether the input is merely asking the rules, not giving an enforcement order."
# Hit query mode returns True
When you hit the query mode, the L1 and L2 tools are temporarily removed from the candidate set, leaving only the L0 search type tool. This is a rule of "better do less than do nothing wrong" — evidentiary questions and command requests with asymmetrical risks.
6.2 Melting: three fuses
It is not enough to prejudge intent, and the model will repeat itself after failure. Three fuses are needed:
First: duplicate call fingerprints. Yes (toolname, normative parameters) Do Hashi, the same fingerprints appear more than two times within a mission. This one directly targets the RAC category of ToolScan.
Second course: maximum number of rounds. Sets a hard ceiling for tasks such as MAX ITERATIONS = 2 times correction of retry. Stop and return to a clear cause of failure. The reason for this is a diminishing yield: the error in the parameter retry may be fixed at a time, and the fact that permission is less than a hundred times successful than retry will only push the delay and the cost of the token up.
Third course: Differential processing driven by erroneous classification. Split errors into four categories, with different paths:
Here's the easy details to do wrong: the wrong messages don't flatten into "tool call failed, please retry." This string is zero info for the model, and it's in place for retry. The more specific the error information, the higher the model's self-rehabilitated success rate; on the other hand, the vague error + the unlimited retry = log explosion + the sharp increase in costs, exactly the chain of accidents mentioned at the meeting.
The last gate in this layer is the one that no static rule can replace: a second confirmation of high-risk operations. It's a separate chapter.
7. Last ring of the fourth layer: second confirmation of high-risk tools (Human-in-the-loop)
7.1 Why do you have to do it?
Delete, modify, batch operations, empty data — these tools are irreversible if they are misdirected. The national standard is rigid in this regard: AI agent Interactive Security Requirements 8.2 g expressly require tool call of the high-risk system permission to "fully inform the user of safety risks and obtain the express consent of the user".
The principle is clear: AI does not have the right to perform high-risk operations on its own. Agent is responsible only for generating operational previews — what to do, why, and the extent of the impact — before the actual execution is confirmed by the user.
7.2 But second confirmation is not a panacea.
There must be a bowl of cold water here, or chapter 7 will become a safe placebo.
Many studies in 2026 point to the same conclusion: Human beings perform far worse at the approval stage than we thought.
- A public experiment of approximately 40,000 sessions, 409,000 approvals/denials of decision-making shows that the overall accuracy rate of human approvals is only 66.3 per cent, with about one out of every three threats, and that 32.9 per cent of the sessions "net gains are negative" - free of threats and false security orders (explainx.ai data-processing).
- In an on-site Claude Code experiment involving 1,053 testers, the rate of artificial interception was only 13.6 per cent in the face of an injection of a hazardous order, compared with 89 per cent (analysing) in the back of the same scene.
- Worse still, attacks such as "Invisible Ink Threats": The attackers hid malicious targets behind legitimate missions, at a time when HITL (Human-in-the-loop, man on the way back) not only did not lower the attack on success rate, but on average rose by 7.8 per cent, from 21.2 per cent to 42 per cent under individual configurations (arXiv:2608.0218).
The first two of the above-mentioned articles are reproduced in a secondhand manner, and the original experimental report is not made public in its entirety. But the three directions are consistent enough to support one judgement: the quality of the confirmation box determines whether the second confirmation is a defence or a rubber stamp.
7.3 Four elements of a valid confirmation box
So the correct wording of Chapter 7 is not "a confirmation window is finished" but rather a design constraint for the confirmation box:
- A preview of the operation, no abstract questions. Do not ask whether "delete all logs are allowed to be executed" and write "three files, 1.2 GB, will be deleted from / var/log/app, the operation is irrevocable, the last backup was 3 days ago".
- Scope of impact, which can be checked. Number of objects affected, tables/catalogues involved, whether or not to trigger downstream tasks - - These have to come from a real pre-test, not a model oral.
- Default refusal, not tacit consent. The security line for the bullet window should make "consent" a default option and avoid the user's habitual return.
- Merge the same items. 20 continuous confirmation boxes, with the consent of the user bound to be brainless; the batch operation should be aggregated into a confirmation and the total amount specified.
Article 4 is particularly important: approval fatigue is the most direct explanations for 66.3 per cent and 13.6 per cent. The greater the number of confirmations, the lower the value of the information identified on a single basis.
7.4 Correct priority: Minimal permission > Second confirmation
And finally back to the project judgment: the second confirmation should be the last line of defense, not the first.
The order is wrong. If the high-risk tool is already in the candidate pool and permission is confiscated, you will have to use manual clearance; the accuracy rate of manual clearance has been eliminated from the three data sets above. Conversely, if the second tier moves high-risk tools out of the default pool, and the third layer binds permission to the main body, the scenes that require human confirmation will be condensed to a very small extent — at which point it will be possible to confirm that there are few frames, complete information, and that manual judgement will be effective.
This explains why Chapter 5 emphasizes tacit rejection: The rest of the confirmation is more significant each time unnecessary recognition is reduced.
This set of four-layer defenses is suitable for more than five tools with at least one irreversible production system; if Agent's only takes two or three read-only tools, and then so many layers slows the response, it's enough to get Schema to confirm it.
This column, " AI Agent Tool ", focuses on the governance of reliability of the tool calling, from tool description, candidate stifling, permission controls, to running time wind control and human confirmation, with only one problem of landing per piece, updated once a week.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.