AI 基本要素
你怎么评价一个AI Agent？任务、轨迹、可靠性和成本
评价AI agent需要的不仅仅是检查其最终答案。 强电压测量结果，检查轨迹，重复试验，测试安全界限，跟踪完成可靠工作的成本。
在 Google 上的首选来源中添加 Unite.AI
评价AI agent意味着测试它是否可靠地完成具有代表性的任务，遵循可接受的路径，尊重其边界，并产生足够的成本价值。 单靠一个最后的答案是不够的，因为agent可以通过不安全的行动，不支持的声称，过度的重复，或者运气来达到一个可信的结果。
Agent评价很困难，因为系统在一段时间内运作。 该模型选择工具，观察不断变化的环境，并可能在每次运行中采取不同的路线。 因此，有效的评估衡量反复试验的结果和轨迹。
Agent Evals 词汇表
Anthropic的解密agent评价指南提供了一套有用的术语：
- 任务：一个具有客观、环境、投入和成功标准的试验案例。
- 审判：agent试图完成一项任务。
- 记录或轨迹：审判期间的信息、决定、工具呼叫、观察和行动顺序。
- 年级：规则，程序，模型，或者对部分性能进行评分的人。
- 评价的利用：环境准备、试验、收集痕迹、引用分级器和报告结果的系统。
保持这些概念的分离可以防止一个常见的错误：将一个成功的演示作为agent会可靠表现的证据。
以清晰的任务规格开始
一个有用的评价任务描述了初始状态，可用的工具，制约因素，预期结果，以及重要的被禁止的行为。 测试应该像真正的工作而不透露一个单一的脚本路径。
对agent的支持来说，“解决罚单”太模糊了。 一项更强有力的任务规定了客户的记录、适用的政策、允许的补救办法、必要的沟通，以及记录制度中必须出现哪些变化。 然后可以客观地进行分级。
Agent评价的四层
这些层次在报告方面应分开。 把它们合并成一个平均值可以隐藏不可接受的行为——例如，任务成功率高，掩盖了经常未经授权的工具尝试。 团队可以先定义硬安全门，然后在经过这些门的运行中比较结果质量和效率。
1. 结果质量
agent是否实现了目标？结果分级者可以检查确切的状态变化，需要的字段，测试结果，事实准确性，引用支持，或者基于字段的质量。
在可能的情况下，使用环境的最终状态，而不是agent自己的索赔。 一个编码的agent没有成功，因为这个bug是固定的称；相关的测试和行为必须确认它。
2. 轨迹质量
agent有没有走声音路径？轨迹评价可以发现不必要的数据访问，被禁止的工具，重复的失败，跳过核查，捏造的观察，或违反批准规定。
并非每个偏离引用路径都是错误。 Agents可以找到评估者没有预料到的有效解决方案。 年级生应强制执行真正的要求，同时允许其他高效办法。
3. 遵守安全和政策
agent是否在它的职权范围内？测试应包括提示词注入，相互矛盾的指示，敏感数据，模糊的用户意图，以及需要拒绝或升级的请求。
安全阀需要现实的后果在沙盒。 如果 agent 已经发送消息，更改记录，或者披露数据，那么符合政策的句子是不够的。
4. 业务效率
成功需要多少时间和金钱？轨迹模型标志，工具调用，时钟空闲，重试，人类干预，以及外部服务成本。 在100个不必要的动作后成功的agent可能不如简单的工作流程有用。
使用多个年级类型
决定分级器检查准确的属性：是否通过测试，文件是否正确更改，数据库字段有需要的值，或者使用禁止的终点。 它们速度快，一致，但只涵盖开发者编码的内容。
基于模型的分级者评价清晰度，完整性，相关性，遵守细化标准等品质。 它们的规模比人类审查要好，但需要自己的验证，因为它们可能带有偏见或不一致。
人类分级对于专家判断，模棱两可的案件，以及验证自动化分级器都很重要。 它们的速度较慢，费用也比较昂贵，因此团队经常将它们用于样本，分歧，以及影响大的领域。
最强的评价套件结合了这些办法。 OpenAI的Evals平台同样将评价视为定义测试，分级和重复实验的结构化过程，而不是非正式的检查。
可靠性要求重复审判
运行每个任务的时间足够多，以观测来自采样，变化的工具和环境状态的变异性。 报告平均业绩和严重失败的频率。 对于由几个依附步骤制成的工作流程，每步精确度可以看起来很强，而端到端的可靠性仍然很弱，因为一次失败就足以使整个任务脱轨。
Agent行为是概率论。 同样的任务可以产生不同的计划，工具调用，以及结果。 多次运行每个重要任务并报告通过率或得分分布，而不仅仅是最佳的例子。
两项可靠性措施回答了不同的问题：
- Pass@k：至少一次K尝试成功的可能性。 在可接受多种尝试时，这是有用的。
- Pass^k：所有 k 尝试都成功的概率。 这反映了每次运行必须奏效时的一致性。
一个系统可以有一个有吸引力的通过@k，同时仍然不适合自动操作，因为单个运行是不可靠的。
构建一个代表任务集
从真正的失败和共同的生产要求开始。 包括直截了当的任务，困难的边缘案例，模棱两可的投入，工具错误，陈旧的数据，缺失的信息，以及对抗性的内容。
按任务类型和风险分列的部分结果。 总体平均数可以掩盖财务行动、长期任务或特定应用方面的严重弱点。 保留一个被搁置的套件，这样反复的即时调试不会把评价套件变成培训套件。
评估环境，而不仅仅是模型
agent的性能取决于工具定义，权限，内存，上下文管理，用户界面，网络行为，以及管弦乐。 一个模型升级可以减少一个错误，同时暴露另一个错误。
每次运行所用的整个堆栈版本。 记录模型，提示，工具，数据快照，环境，预算和分级器版本。 否则，分数变化不能追溯到有意义的事业。
常见的评价错误
数据泄露是另一个风险。 如果基准测试任务，预期答案，或分级器模式出现在模型的训练数据或即时上下文中，则分数可以衡量识别而不是概括。 维护私人搁置任务，在系统变化时刷新案例，并记录每个运行所使用的准确模型，即时，工具版本和环境。
——Demo驱动的评价：在看到agent表现良好后选择任务。
- 最后只回答的评分：忽略不安全或浪费的轨迹。
- 单一运作的结论：将幸运的成功视为稳定的能力。
- 低空测试：将答案或参考轨迹置于agent的背景中。
- 未认证的模型分级员：假定自动法官正确而不检查与专家的协议。
- 静态环境：测试一个接口、账户状态或数据安排。
- 不进行负面测试：在没有检查agent是否拒绝或适当升级的情况下测量任务完成情况。
- 忽视经济学：优化成功，而成本和延迟则无法使用。
从离线Evals到生产监测
离线评价得到控制和可重复。 它们对于在发布前比较提示、模型、工具和政策是十分理想的。 生产监测揭示实际用户行为和环境变化的长尾。
一个成熟的程序将两者联系起来。 事件和低信心生产痕迹成为新的离线任务。 候选人更改必须通过回归套件。 部署后，小组将观察任务的成功率、干预率、政策活动、工具错误、成本、耐久性和用户反馈。
实用 Agent 计分卡
记分卡应区分硬门和优化度量衡。 政策遵守情况、授权和所需证据可以是过关或故障；准确性、延迟性、成本和审阅员的努力可以比较为可接受的运行。 这使得各队无法以微小的效率收益换取平均得分会掩盖的安全回归。
有用的记分卡可包括：
- 任务成功率和质量分数；
- 重复审判的一致性；
- 重大政策违规率；
- 工具选择和论据准确性；
- 工具故障或环境故障后的回收率；
- 人员升级和超速率；
- 中位点和尾端纬度；
- 模式、工具和人力成本；
- 轨迹效率与合理基线相比。
报告每项成功任务的成本，而不仅仅是每次运行的成本。 一次又一次失败的廉价agent比一次成功得力的更昂贵。
AI Agent如何评价？
Agent评价是在现实条件下测试完整的目标定向系统的学科。 它衡量作品是否正确，路径是否可接受，行为是否可重复，经济学是否合理。
开始完成一系列相应的小任务，进行多次试验，分层分级，并完成轨迹。 然后将每一个有意义的生产失败都变成回归测试。 Agent能力迅速变化；持久的评价系统是团队如何区分真正的进展和有说服力的演示。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
AI Fundamentals
How Do You Evaluate an AI Agent? Tasks, Trajectories, Reliability, and Cost
Evaluating an AI agent requires more than checking its final answer. Strong evals measure outcomes, inspect trajectories, repeat trials, test safety boundaries, and track the cost of getting reliable work done.
Add Unite.AI to your preferred sources on Google
Evaluating an AI agent means testing whether it completes representative tasks reliably, follows acceptable paths, respects its boundaries, and delivers enough value for its cost. A final answer alone is not sufficient because an agent can reach a plausible outcome through unsafe actions, unsupported claims, excessive retries, or luck.
Agent evaluation is difficult because the system acts over time. The model chooses tools, observes changing environments, and may take a different route in every run. Effective evals therefore measure outcomes and trajectories across repeated trials.
The Vocabulary of Agent Evals
Anthropic’s guide to demystifying agent evaluations provides a useful set of terms:
- Task: one test case with an objective, environment, inputs, and success criteria.
- Trial: one attempt by the agent to complete a task.
- Transcript or trajectory: the sequence of messages, decisions, tool calls, observations, and actions during the trial.
- Grader: a rule, program, model, or person that scores part of the performance.
- Evaluation harness: the system that prepares environments, runs trials, collects traces, invokes graders, and reports results.
Keeping these concepts separate prevents a common mistake: treating one successful demonstration as evidence that the agent will perform reliably.
Start with a Clear Task Specification
A useful evaluation task describes the initial state, available tools, constraints, expected outcome, and important prohibited behavior. The test should resemble real work without revealing a single scripted path.
For a support agent, “resolve the ticket” is too vague. A stronger task specifies the customer’s record, applicable policy, allowed remedies, required communication, and which changes must appear in the system of record. It can then be graded objectively.
Four Layers of Agent Evaluation
These layers should remain separate in reporting. Combining them into one average can hide unacceptable behavior—for example, a high task-success score that masks frequent unauthorized tool attempts. Teams can define hard safety gates first, then compare outcome quality and efficiency among runs that pass those gates.
1. Outcome Quality
Did the agent accomplish the goal? Outcome graders can check exact state changes, required fields, test results, factual accuracy, citation support, or rubric-based quality.
Where possible, use the environment’s final state rather than the agent’s own claim. A coding agent has not succeeded because it says the bug is fixed; the relevant tests and behavior must confirm it.
2. Trajectory Quality
Did the agent take a sound path? Trajectory evaluation can detect unnecessary data access, prohibited tools, repeated failures, skipped verification, fabricated observations, or approval violations.
Not every deviation from a reference path is an error. Agents can find valid solutions that evaluators did not anticipate. Graders should enforce genuine requirements while allowing alternative efficient approaches.
3. Safety and Policy Compliance
Did the agent stay within its authority? Tests should include prompt injection, conflicting instructions, sensitive data, ambiguous user intent, and requests that require refusal or escalation.
Safety evals need realistic consequences in a sandbox. A policy-compliant sentence is not enough if the agent already sent the message, changed the record, or exposed the data.
4. Operational Efficiency
How much time and money did success require? Track model tokens, tool calls, wall-clock latency, retries, human interventions, and external service cost. An agent that succeeds after 100 unnecessary actions may be less useful than a simpler workflow.
Use Multiple Types of Graders
Deterministic graders inspect exact properties: whether a test passed, a file changed correctly, a database field has the required value, or a prohibited endpoint was called. They are fast and consistent but cover only what developers encode.
Model-based graders evaluate qualities such as clarity, completeness, relevance, and adherence to nuanced criteria. They scale better than human review but need their own validation because they can be biased or inconsistent.
Human graders are important for expert judgment, ambiguous cases, and validating automated graders. They are slower and more expensive, so teams often use them on samples, disagreements, and high-impact domains.
The strongest evaluation suites combine these approaches. OpenAI’s Evals platform similarly treats evaluation as a structured process for defining tests, graders, and repeated experiments rather than informal inspection.
Reliability Requires Repeated Trials
Run each task enough times to observe variability from sampling, changing tools, and environmental state. Report both average performance and the frequency of severe failures. For workflows made of several dependent steps, per-step accuracy can look strong while end-to-end reliability remains weak because a single failure is enough to derail the whole task.
Agent behavior is probabilistic. The same task can produce different plans, tool calls, and results. Run each important task multiple times and report pass rates or score distributions, not just the best example.
Two reliability measures answer different questions:
- Pass@k: the probability that at least one of k attempts succeeds. This is useful when multiple attempts are acceptable.
- Pass^k: the probability that all k attempts succeed. This reflects consistency when every run must work.
A system can have an attractive pass@k while remaining unsuitable for automatic operation because individual runs are unreliable.
Build a Representative Task Set
Start with real failures and common production requests. Include straightforward tasks, difficult edge cases, ambiguous inputs, tool errors, stale data, missing information, and adversarial content.
Segment results by task type and risk. An overall average can hide a severe weakness in financial actions, long-horizon tasks, or a particular application. Preserve a held-out set so repeated prompt tuning does not turn the evaluation suite into a training set.
Evaluate the Environment, Not Only the Model
An agent’s performance depends on tool definitions, permissions, memory, context management, user interface, network behavior, and orchestration. A model upgrade can reduce one error while exposing another.
Version the entire stack used for each run. Record the model, prompts, tools, data snapshot, environment, budgets, and grader versions. Otherwise a score change cannot be traced to a meaningful cause.
Common Evaluation Mistakes
Data leakage is another risk. If benchmark tasks, expected answers, or grader patterns appear in the model’s training data or prompt context, the score may measure recognition instead of generalization. Maintain private holdout tasks, refresh cases as systems change, and record the exact model, prompt, tool versions, and environment used for every run.
- Demo-driven evaluation: choosing tasks after seeing what the agent does well.
- Final-answer-only scoring: ignoring unsafe or wasteful trajectories.
- Single-run conclusions: treating a lucky success as stable capability.
- Leaky tests: placing the answer or reference trajectory in the agent’s context.
- Unvalidated model graders: assuming an automated judge is correct without checking agreement with experts.
- Static environments: testing one interface, account state, or data arrangement.
- No negative tests: measuring task completion without checking whether the agent refuses or escalates appropriately.
- Ignoring economics: optimizing success while cost and latency become unusable.
From Offline Evals to Production Monitoring
Offline evaluations are controlled and repeatable. They are ideal for comparing prompts, models, tools, and policies before release. Production monitoring reveals the long tail of actual user behavior and environment change.
A mature program links the two. Incidents and low-confidence production traces become new offline tasks. Candidate changes must pass the regression suite. After deployment, teams watch task success, intervention rates, policy events, tool errors, cost, latency, and user feedback.
A Practical Agent Scorecard
A scorecard should distinguish hard gates from optimization metrics. Policy compliance, authorization, and required evidence can be pass-or-fail gates; accuracy, latency, cost, and reviewer effort can then be compared among acceptable runs. This prevents teams from trading a small efficiency gain for a safety regression that an averaged score would conceal.
A useful scorecard can include:
- task success rate and quality score;
- consistency across repeated trials;
- critical policy violation rate;
- tool-selection and argument accuracy;
- recovery rate after tool or environment failures;
- human escalation and override rate;
- median and tail latency;
- model, tool, and human cost per successful task;
- trajectory efficiency compared with a reasonable baseline.
Report cost per successful task, not only cost per run. A cheap agent that fails repeatedly can be more expensive than a stronger model that succeeds once.
What to Remember About How Do You Evaluate an AI Agent
Agent evaluation is the discipline of testing a complete goal-directed system under realistic conditions. It measures whether the work is correct, whether the path is acceptable, whether the behavior is repeatable, and whether the economics make sense.
Begin with a small set of consequential tasks, multiple trials, layered graders, and complete trajectories. Then turn every meaningful production failure into a regression test. Agent capability changes quickly; a durable evaluation system is how teams distinguish real progress from a persuasive demo.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.