人工智能
用 Amazon Nova 法案建立意图驱动 UI 测试
为了将意图驱动的UI测试范围扩大到其产品团队，Epsilon，一家数据，技术和营销服务公司以及Publicis Groupe的一部分，转向Amazon Nova Act. 随着亿万消费者简介和数千个品牌关系，Epsilon需要UI测试，可以跟上快速的应用变化。 解决方案不能为每个产品组制造维护费。
在Epsilon内部，Agentic平台团队(50名工程师)建设共享的基础设施，以便其他产品团队能够安全地建设，部署，治理，运行AI agents. 通过这个共享的平台，队伍可以获得基于技能的agent建设者。 他们还获得联邦身份和飞行前授权、示范背景协议(MCP)的整合以及交叉账户的提供。 当一个产品团队想要采用AI工具时，他们获得的能力没有额外的配置，也没有自行重新解析身份，安全，浏览器基础设施，或者模型集成。
管理能力之一是UI测试。 凭借Amazon Nova Act和Amazon Bedrock Agent Core浏览器(一种Amazon Bedrock Agent Core的能力)，埃普西隆的平台团队可以提供弹性，意向驱动的测试，作为平台上的团队的服务。
透过Epsilon的架构提供UI测试作为管理平台服务，显示核心Nova法案和Amazon Bedrock Agent Core浏览器工具整合码
挑战：扩大覆盖面，而不是取代行之有效的东西
Epsilon 现有的UI测试实践没有被打破。 该团队维护了一个基于Playwright的回归套件，该套件在他们的网络应用程序中验证了功能，并且效果良好。 剧作家今天仍是他们的回归支柱。
Epsilon被识别的机会是添加剂而不是纠正的：
- 脚本Playwright测试对确定回归检查有效。 然而，它们与选择者和页面结构紧密结合。 综艺UI的改变即使在基础功能是声音时也能打破它们。
- 多个产品组需要针对自己的应用进行全面测试。 建立浏览器基础设施，管理依赖性，与每个团队的身份系统融合，造成了重复的努力。
-平台团队希望UI测试成为符合他们现有的agent模型的管理能力，而不是每个机上团队必须操作的定制CLI.
目标：在现有的Playwright回归套件之上进行层层意图测试。 作为多租户平台服务交付，使每个Epsilon产品团队能够通过他们用于其他agent能力的相同的API表面消耗。
为什么Epsilon选择了Amazon Nova Act和Amazon Bedrock Agent Core浏览器工具
Epsilon根据两项要求评价了各种选项：自然语言测试作者(以补充基于选择器的Playwright测试)和管理的浏览器基础设施(以适应其多租户平台模型). 用Amazon Nova Act与Amazon Bedrock Agent Core浏览器工具结合，Epsilon可以同时解决两者。
- 自然语言步骤。 测试步骤是由Nova Act执行的纯英语指令。 它们具有适应化妆的UI变化的能力，并且可以被非工程师读取。 类似“确认奖励的平衡是显而易见的”这样的一步，其年龄远优于一个简单的CSS选择器。
- AWS管理，ephemeral浏览器会话。 使用Amazon Bedrock Agent Core浏览器工具，团队会获得孤立的浏览器实例，没有二进制符可以发送，也没有显示服务器可以维护。 Amazon Bedrock Agent Core浏览器工具也为人类观测或接收提供直播视图(可用于CAPTCHAs或多要素认证)，会话记录到Amazon简单存储服务(Amazon S3)和AWS CloudTrail审计，均没有附加配置。
- AWS身份和访问管理(AWS IAM)基于认证。 有了Nova Act，可以通过标准AWS认证链(任务角色)认证。 这符合Epsilon 现有的安全模式，不分配单独的 API 密钥。
- 交叉账户友好。 有了Nova Act和Amazon Bedrock Agent Core浏览器工具，您可以将槽插入您现有的AWS安全托肯服务(AWS STS) AssumeRole 提供。 每个登机团队可以在自己的AWS账户内在一个范围化的角色下运行测试。 这保持了孤立和可审计性。
Epsilon选择了Amazon Bedrock Agent Core浏览器工具，具体因为它消除了管理浏览器二进制，显示服务器和会话生命周期的操作负担。 代理平台小组对拥有这一基础设施没有兴趣。
解决办法：结构和执行
Epsilon的解决方案主要有三个组成部分：一个用于创建工作流程的平台API，一个用于执行它们的背景任务执行者，以及驱动浏览器的Nova Act和Amazon Bedrock Agent Core浏览器工具集成。
平台架构
Epsilon的UI测试能力遵循一条清晰的管道：
- 用户通过平台API创建工作流程，指定目标 url和自然语言步骤列表。
- 在创建时，平台在AWS上登记了Nova Act工作流程定义。
- 执行时，一个背景任务运行器(Celery，用于同步任务处理的Python框架)打开了Amazon Bedrock Agent Core浏览器工具浏览器会话。 它通过Nova Act在Chrome DevTools Protocol(CDP) WebSocket(程序上控制Chromium浏览器的标准接口)上运行每个步骤，包括有条件的逻辑(例如，“如果出现旅行盒，跳过它”).
- 系统每步捕捉一个截图，每步跟踪通过/故障，计算一个总体通过率，并存储一切(结果为JSON，截图为Amazon S3，会话ID).
- 完成钩火从而记录结果回到平台。
图1 显示 Epsilon的UI测试服务的分层架构。 用户请求从Agent Cloud前端通过一个API Gateway进行认证和权限检查。 API层(Django REST)暴露了工作流程，执行，自动化套件和统计的端点。 业务逻辑层包括一名工作流程管理员、执行操作员和成套协调员。 测试任务在Amazon简易排队服务(Amazon SQS)中排队，并由运行在Amazon弹性容器服务(Amazon ECS)上的跑步者服务接收，其中包含Nova Act引擎和一个无头铬浏览器。 结果，截图，轨迹，以及HTML报告被存储在Amazon S3中。
该能力被曝光为标准agent(ui-test-agent)，具有两种技能：
- ui-test-recorder：探索一个场景交互并记录自然语言步骤。
- ii-test-executor：运行录制步骤并返回结构化结果。
机载团队消耗UI测试的方式与消耗任何其他平台agent的方式相同。 没有额外的基础设施需要管理。 入职过程很简单。 平台团队为新团队提供跨账户的IAM角色，注册其目标应用URL，并通过现有的平台API授予Ui-test-agent访问权限。 从此，团队创建工作流程，触发处决，并通过他们已经用于其他平台能力的相同接口获取结果。
核心一体化
Nova Act + Amazon Bedrock Agent Core浏览器工具集成简洁：
这种模式(打开一个有管理的浏览器会话，通过CDP WebSocket连接，用自然语言驱动动作)是基础。 Epsilon用他们的平台的多层管弦乐、结果捕捉和可观察层把它包起来。
示例：自动创建 agent 流
以下是Epsilon平台的有代表性的工作流程，它进行登录，表单输入，有条件处理，滚动，以及校验。 以下步骤突出了新法的不同模式：
步骤4和步骤7展示了有条件的逻辑：“ 如果元素出现，处理它 ” 和 “ 如果目标不可见，滚动直到它出现 ”。 这些模式需要在传统框架内明确划分。 Nova Act通过语言理解来自然地处理它们。
运行时间隔离
Epsilon在一条专用的跑车服务(FastAPI侧车)中运行UI测试，它用Workflow和IAM认证支持将nova-act 3.x标注，加上Amazon Bedrock Agent Core浏览器工具客户端。 这使得依赖性与主后端分离，如果赛跑者残疾，工人将回到执行过程中，在部署期间提供复原力。
最佳做法
Epsilon团队根据其将UI测试建设为平台能力的经验，建议：
1. 将步骤写成意向，而不是力学
基于意向的步骤在UI重新设计后幸存。 产品经理和质量保证(QA)领导谁可能不写代码可以直接审查。
2. 隔离新法运行时间
Pin Nova Act和Amazon Bedrock Agent Core浏览器工具在专用服务或容器中的依赖性。 Epsilon得知Nova Act的签名终点针对我们东一区，用自己的依赖锁文件在单独的服务中运行它可以防止难调试的冲突。
3. 重新使用记录的轨迹来降低成本
Epsilon在草稿状态下使用轨迹重试，保存成功的轨迹，并在活动状态下重放。 重复运行跳过冗余的大型语言模型(LLM)“思考，”既减少了延缓性，也降低了回归式重复处决的成本。
4. 从第一天起多租期的设计
使用跨账户的AssumeRole，并带有IAM政策范围。 每个租户的测试都是在他们自己的职责下进行的。 这提供了孤立、可审计性和故障范围控制，没有每个团队的定制基础设施。
5. 对管理下的可观察性有优势
使用Amazon Bedrock Agent Core浏览器工具，团队获得直播视图，会话记录，以及AWS CloudTrail集成。 Epsilon使用这些内置能力，而不是建立定制的可观察性，减少维护，让合规小组熟悉的审计表面。
6. Amazon Bedrock Agent Core浏览器账户工具查看器行为
在提示Nova Act步骤时，说明Amazon Bedrock Agent Core浏览器工具的浏览器视图端默认值。 引用滚动的步骤(例如“滚动到目标语句文本框可见”)证实Nova Act正确处理页面下部的内容。
结果和影响
Epsilon的UI测试能力在多个团队中运行，多租户架构在生产中得到了证明。
以下是衡量的结果：
- 4-6x更快的测试作者(基于Epsilon的内部基准测试). 一个18步的代表工作流程用5-10分钟的时间来编剧Nova Act，而对应的Playwright剧本(80-150行代码)则需要30-90分钟。 Playwright脚本中的每一个互动步骤都需要检查DOM，选择一个稳定的选择器，并编码明确的等待，断言，以及有条件的案件的分支。 Nova Act用18个自然语言指令和零选择器或DOM检查来取代每步的间接费用，并用简单的英语表示有条件的逻辑(如旅行箱和滚动至可见步骤).
- 轨迹重播100%的通过率(基于Epsilon的内部基准测试). 在工作流程通过初始探索阶段(典型的4–5 即时迭代)后，Epsilon会保存成功的轨迹，并重播其决定性。 节省的轨迹在重复运行中实现了100%的通过率。
- 低屏障测试作者。 步步读作普通英语。 核心测试工程师以外的团队成员可以阅读，审查和提议测试，扩大质量所有权。
- 内设审计线索 AWS Cl 计算机
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Building intent-driven UI testing with Amazon Nova Act
To scale intent-driven UI testing across its product teams, Epsilon, a data, technology, and marketing-services company and part of Publicis Groupe, turned to Amazon Nova Act. With hundreds of millions of consumer profiles and thousands of brand relationships, Epsilon needed UI testing that could keep pace with rapid application changes. The solution could not create maintenance overhead for every product team.
Within Epsilon, the Agentic Platform team (50 engineers) builds shared infrastructure so other product teams can build, deploy, govern, and run AI agents safely. With this shared platform, teams get a skills-based agent builder. They also get federated identity and pre-flight authorization, Model Context Protocol (MCP) integration, and cross-account provisioning. When a product team wants to adopt AI tooling, they get capabilities without additional configuration and without re-solving identity, security, browser infrastructure, or model integration on their own.
One of those managed capabilities is UI testing. With Amazon Nova Act and Amazon Bedrock AgentCore Browser (a capability of Amazon Bedrock AgentCore), Epsilon’s platform team could deliver resilient, intent-driven testing as a service to teams on the platform.
In this post, we walk through Epsilon’s architecture for delivering UI testing as a managed platform service, show the core Nova Act and Amazon Bedrock AgentCore Browser Tool integration code, highlight a concrete 18-step test workflow, and share best practices and measured results.
The challenge: Enhancing coverage, not replacing what works
Epsilon’s existing UI testing practice was not broken. The team maintained a Playwright-based regression suite that validated functionality across their web applications, and it worked well. Playwright remains their regression backbone today.
The opportunity Epsilon identified was additive, not corrective:
- Scripted Playwright tests are effective for deterministic regression checks. However, they are tightly coupled to selectors and page structure. Cosmetic UI changes can break them even when the underlying functionality is sound.
- Multiple product teams needed full testing coverage against their own applications. Standing up browser infrastructure, managing dependencies, and integrating with identity systems per team created duplicative effort.
- The platform team wanted UI testing to be a managed capability that fit their existing agent model, not a custom CLI that every onboarded team had to operate.
The goal: layer intent-based testing on top of the existing Playwright regression suite. Deliver it as a multi-tenant platform service so that each Epsilon product team could consume through the same API surface they used for every other agent capability.
Why Epsilon chose Amazon Nova Act and the Amazon Bedrock AgentCore Browser Tool
Epsilon evaluated options against two requirements: natural-language test authoring (to complement selector-based Playwright tests) and managed browser infrastructure (to fit their multi-tenant platform model). With Amazon Nova Act combined with Amazon Bedrock AgentCore Browser Tool, Epsilon could address both.
- Natural-language steps. Test steps are plain-English instructions executed by Nova Act. They are resilient to cosmetic UI changes and readable by non-engineers. A step like “Confirm the rewards balance is visible” ages far better than a brittle CSS selector.
- AWS managed, ephemeral browser sessions. With Amazon Bedrock AgentCore Browser Tool, teams get isolated browser instances with no binaries to ship and no display server to maintain. Amazon Bedrock AgentCore Browser Tool also offers live view for human observation or takeover (useful for CAPTCHAs or multi-factor authentication), session recording to Amazon Simple Storage Service (Amazon S3), and AWS CloudTrail audit, all without additional configuration.
- AWS Identity and Access Management (AWS IAM)-based authentication. With Nova Act, you can authenticate through the standard AWS credential chain (the task role). This fits Epsilon’s existing security model without distributing separate API keys.
- Cross-account friendly. With Nova Act and Amazon Bedrock AgentCore Browser Tool, you can slot into your existing AWS Security Token Service (AWS STS) AssumeRole provisioning. Each onboarded team can run tests inside their own AWS account under a scoped role. This maintains isolation and auditability.
Epsilon chose Amazon Bedrock AgentCore Browser Tool specifically because it eliminates the operational burden of managing browser binaries, display servers, and session lifecycle. The Agentic Platform team had no interest in owning that infrastructure.
The solution: Architecture and implementation
Epsilon’s solution has three main components: a platform API for creating workflows, a background task runner for executing them, and the Nova Act and Amazon Bedrock AgentCore Browser Tool integration that drives the browser.
Platform architecture
Epsilon’s UI testing capability follows a clear pipeline:
- A user creates a workflow through the platform API, specifying a target_url and a list of natural-language steps.
- On create, the platform registers a Nova Act workflow definition on AWS.
- On execute, a background task runner (Celery, a Python framework for asynchronous job processing) opens an Amazon Bedrock AgentCore Browser Tool browser session. It runs each step through Nova Act over a Chrome DevTools Protocol (CDP) WebSocket (the standard interface for programmatically controlling Chromium browsers), including conditional logic (for example, “if a tour box appears, skip it”).
- The system captures a screenshot per step, tracks pass/fail per step, computes an overall pass rate, and stores everything (results JSON, screenshots in Amazon S3, session ID).
- A completion hook fires so the result is recorded back into the platform.
Figure 1 shows the layered architecture of Epsilon’s UI Testing Service. User requests flow from the Agent Cloud Frontend through an API Gateway with authentication and permission checks. The API layer (Django REST) exposes endpoints for workflows, executions, automation suites, and stats. The business logic layer includes a Workflow Manager, Execution Orchestrator, and Suite Coordinator. Test jobs are queued in Amazon Simple Queue Service (Amazon SQS) and picked up by the Runner Service running on Amazon Elastic Container Service (Amazon ECS), which contains the Nova Act engine and a headless Chromium browser. Results, screenshots, trajectories, and HTML reports are stored in Amazon S3.
The capability is exposed as a standard agent (ui-test-agent) with two skills:
- ui-test-recorder: Explore a scenario interactively and record natural-language steps.
- ui-test-executor: Run recorded steps and return structured results.
Onboarded teams consume UI testing the same way they consume any other platform agent. No additional infrastructure to manage. The onboarding process is straightforward. The platform team provisions a cross-account IAM role for the new team, registers their target application URLs, and grants access to the ui-test-agent through the existing platform API. From that point, teams create workflows, trigger executions, and retrieve results through the same interface they already use for other platform capabilities.
The core integration
The Nova Act + Amazon Bedrock AgentCore Browser Tool integration is concise:
This pattern (open a managed browser session, connect through the CDP WebSocket, and drive actions with natural language) is the foundation. Epsilon wraps it with their platform’s multi-tenant orchestration, result capture, and observability layers.
Example: Automating the ‘Create an agent’ flow
To illustrate intent-based testing, here is a representative workflow from Epsilon’s platform. It exercises login, form entry, conditional handling, scrolling, and verification. The following steps highlight the differentiating patterns of Nova Act:
Steps 4 and 7 demonstrate conditional logic: “if an element appears, handle it” and “if a target isn’t visible, scroll until it is.” These patterns would require explicit branching in a traditional framework. Nova Act handles them naturally through language understanding.
Runtime isolation
Epsilon runs UI tests in a dedicated runner service (a FastAPI sidecar) that pins nova-act 3.x with Workflow and IAM authentication support, plus the Amazon Bedrock AgentCore Browser Tool client. This keeps dependencies separate from the main backend so they don’t collide with the Strands agent stack. The worker falls back to in-process execution if the runner is disabled, providing resilience during deployments.
Best practices
Based on their experience building UI testing as a platform capability, the Epsilon team recommends:
1. Write steps as intent, not mechanics
Intent-based steps survive UI redesigns. Product managers and quality assurance (QA) leads who might not write code can review them directly.
2. Isolate the Nova Act runtime
Pin Nova Act and Amazon Bedrock AgentCore Browser Tool dependencies in a dedicated service or container. Epsilon learned that the signing endpoint for Nova Act targets us-east-1 regardless of the browser session’s region. Running it in a separate service with its own dependency lockfile prevents hard-to-debug conflicts.
3. Reuse recorded trajectories to cut cost
Epsilon uses trajectory retries in draft state, saves the successful trajectory, and replays it in active state. Repeat runs skip redundant large language model (LLM) “thinking,” reducing both latency and cost for regression-style repeated executions.
4. Design for multi-tenancy from day one
Use cross-account AssumeRole with scoped IAM policies. Each tenant’s tests run under their own role in their own account. This provides isolation, auditability, and failure scope containment without custom infrastructure per team.
5. Lean on managed observability
With Amazon Bedrock AgentCore Browser Tool, teams get live view, session recording, and AWS CloudTrail integration. Epsilon uses these built-in capabilities instead of building custom observability, reducing maintenance and giving compliance teams familiar audit surfaces.
6. Account for Amazon Bedrock AgentCore Browser Tool viewport behavior
When prompting Nova Act steps, account for the browser viewport defaults of Amazon Bedrock AgentCore Browser Tool. Steps that reference scrolling (for example, “scroll down until the Goal Statement textbox is visible”) verify that Nova Act handles content in the lower portion of the page correctly.
Results and impact
Epsilon’s UI testing capability is operational across multiple teams, with the multi-tenant architecture proven in production.
Here are the measured outcomes:
- 4-6x faster test authoring (based on Epsilon’s internal benchmarks). A representative 18-step workflow took 5–10 minutes to author with Nova Act, compared to 30–90 minutes for the equivalent Playwright script (80–150 lines of code). Each interactive step in the Playwright script requires inspecting the DOM, choosing a stable selector, and coding explicit waits, assertions, and branching for conditional cases. Nova Act replaces that per-step overhead with 18 natural-language instructions and zero selector or DOM inspection, with conditional logic (such as the tour-box and scroll-until-visible steps) expressed in plain English.
- 100 percent pass rate on trajectory replay (based on Epsilon’s internal benchmarks). After a workflow passes the initial explore phase (typically 4–5 prompt iterations), Epsilon saves the successful trajectory and replays it deterministically. Saved trajectories achieve a 100 percent pass rate across repeated runs.
- Lower barrier to test authoring. Steps read like plain English. Team members beyond core test engineers can read, review, and propose tests, broadening ownership of quality.
- Built-in audit trail. AWS Cl
Auto-fetched and translated by AI Atlas. All rights belong to the original author.