OCT. 22, 2026 (英语).
随着企业云平台的规模为数百万同时期用户服务,紧密结合的微观服务引入了任何静态运行本或阈值警报都无法预料的连锁故障模式。 在黑色星期五闪存销售等高速度事件期间,站点可靠性工程师面临无法原谅的现实:诊断出噪音分布式遥测
虽然基因化的AI copilots加速了代码的编写和日志总和化,但AI agents为了可靠地补救现场生产停产而挣扎。 待命工程师战斗连接池饿死不需要AI agent提供故障排除提示或未经核实的拉力请求,这些请求可能无法解决根本原因;他们需要一个受管制的自演AI系统,能够:
- 实时诊断在缓存和数据库各级分布的根源
- 通过将可执行的缓解工具合成、核实和安装到苍蝇上的实时服务器中间软件,进行自拍
- 通过计算每项行动的边际功劳,将有效补偿与有害的反射区分开来,具有自我性
- 将加密签名的操作性变体蒸馏成语义记忆,以减少重复发生相同故障的可能性
——通过动态重排自身多agent等级,与人类签注重排,吸收未来涌出。
这种范式是自演的AI.
在这个代号为Project Chrysalis的Google Cloud研究参考架构中,我们展示了三个基础研究突破如何结合到一个统一的,生产级的架构上,可以在Gemini平台和Agent开发Kit(ADK)上运行。 纵观模拟的黑色星期五缓存-插座停用,这个受管的架构将时间对第一个减速从15分钟减到45分钟减到1.8秒,将HTTP 5xx的错误率从72.7%减到80.0%减到0.0%,将P99的延迟度从519.8米减到18.2米,并在~0.5米的BigQuery评价了400个反事实交通政策。
关键是,这种自我演变的建筑并不是作为不受约束的"黑盒"运行的。 为确保agent保持安全、可靠、一致和遏制无赖行为,每个自主行动都以Google Cloud的防守深度安全堆绑在密码上:用于对提示词注入进行进场遥测和数据泄漏的模型装甲;Google云密钥管理服务(Cloud KMS)和二进制授权中间软件,用于加密综合标记和阻断未验证代码的安装;Open Policy Agent(OPA)政策控制器,用于执行严格的SLA护栏,防止自我侵入;和双Custody Human-in-Loop(HITL)治理,要求在对Z1Z等级作任何永久性修改之前,明确SRE签注。
要了解静态游戏本和 copilots 咨询为何在压力下失败
——"内侵突袭":闪存促销推动冷藏库存密钥同时突飞猛进的出货请求。
- Cache Eviction: Google Cloud Memorystore(Redis) 在严格的最近使用(LRU)内存限制下驱逐冷键,崩溃缓存命中比为零。
- 连接池饥饿:无缝交通级联直接进入Google Cloud SQL. 一旦 PostgreSQL 连接集合饱和度,即将到来的查询队列会超过超时阈值,服务开始以广泛的HTTP 500错误失败。
- Fatal"3:00 AM Panic Reflex":在操作压力下,人机响应器和天真的LLMs本能都试图重启数据库连接池。 在一次现场直播中,投放连接会锁定数据库内核,将尾部的延缓率提升近十倍,并驱动出错率向全耗。
常规AI agents在部署时用固定的系统提示和预先登记的工具被冻结。 当一个新的基础设施瓶颈袭击时,一个自我演变的AI架构通过在运行时合成针对具体情况的缓解工具,对每次干预的遥测影响进行评分,并为未来事件编纂经核实的护栏,来扩展这些基线能力。
对于真正的自主韧性,agent本身必须基于关于什么可行什么失败的实证发展:
- 自主自选(工具合成):当agent诊断数据库饿死是由一个无喉舌的内进激增驱动,意识到它缺乏内置的交通成形工具时,它在苍蝇上写下自己的中间软件补丁(一个内存的代号-bucket速率限制器),验证它的抽象语法树(AST),用Cloud KMS来签下它,热挂直接输入API网关。
- 因果关系(评价什么是有效的,什么是失败):在稳定断电后,agent评价了事件期间采取的每一项行动的遥测三角洲。 它在数学上惩罚数据库重启反射,因为它恶化了延迟,同时奖励其合成的速率限制器消除服务器错误。
- 认知内存与组织自重:为减少重复退出的可能性,agent将一个签名的操作原理写入矢量内存,在缓存印章时阻断数据库重启。 然后它修改了自己在RAM中的多agent地形,委托了一位在主管agent下的长期交通造型专家Agent,因此未来的涌流在到达数据库之前就已经形成。
自演云 agents是一个活跃的应用研究领域,其重点是解决SRE Trilemma(平衡探索安全,结构适应性,以及多步信用分配),方法是在Google云上统一三个研究支柱:
1. 无服务反事实模拟(梦想-RSI):
挑战:在现场生产中测试未核实的限速参数,有使合法收入节制或重新触发停产的风险。
机理:生产请求的痕迹出口到Google BigQuery,引擎在次毫秒矢量化内存扫描中重播数百个候选符号-bucket配置,以识别离线的Pareto-优化政策。
2. 三层联合演进(APEX):
挑战:单层即时调试提供脆弱的带状助听器,而不受约束的代码自我修改则有系统不稳定的风险。
机理:Agent演化分离为三个受支配的时间尺度:
- 第1层(Ephemeral Hotpatching):合成和挂载模具中间软件工具,以在秒内停止活动断电。
- 第2层(原则抖动):将Cloud KMS签名的课目编码为Agent平台矢量搜索,因此在未来的警报中阻断了禁忌行动。
- 第3层(Topology Rewiring):委托永久专家子agents进入活跃的多agent定向循环图(DAG).
3. 分级分层分配(AgentEvolver):
挑战:在混乱事件中,有多种干预措施依次发生;粗糙的结果奖励不能孤立哪些步骤可以治愈服务,哪些步骤会造成附带损害。
机制:代理Evolver使用干涉前和干预后遥测三角计(P99 latency,HTTP 5xx rate,和数据库池饱和)计算每个执行期间的边际差分信用,将受益补丁与错误补丁隔离。
我们在Gemini平台上将这个架构作为交互式应用程序实施。 上面的图表说明了整个生产目标服务、Agent开发工具包(ADK)多Agent Plane、自我进化引擎和防御深度治理管道的闭路工作流程。
agent的自我进化:
- 发生什么:经过5个请求的热量基准后,50个请求同时的闪售猛增达到 15冷库存键。
- 系统影响:内存存储器命中率从100%降至0.0 %,云SQL连接池达到100%饱和度(2/2 有效连接锁定),P99潜伏率升至312.8至519.8 ms,HTTP 500错误跳至72.7%至80.0%(40次测试失败).
- 发生什么:遥测警报在模型装甲检查进入的警报有效载荷以阻断提示词注入后,唤醒Agent开发 Kit(ADK)主管路由器Agent由Gemini 4 Argon供电。
- 自主自选:诊断池耗竭,APEX层1热patch合成器生成一个内置的代号-bucket速率限制器(10-token容量,40.0 req/s补充). 在安装到API Gateway中间软件之前,补丁通过三个自动治理门:
- AST Sandbox 访问器核实没有禁止的系统模块和动态执行汇。
- 业务厅政策主计长确认补充率达到企业最低服务水平(++10 req/s)。
- Cloud KMS & Binary Authorization在文物摘要(EC SIGN P256 SHA256)上签字,并在中间软件边界上验证密钥来源。
- 系统影响:通过50次请求激增,HTTP 500误差降至0.0%,云SQL池饱和度从100%降至0%,P99耐久恢复到18.2毫秒,超速流量通过HTTP 429清空(40次请求节奏,10次来自温暖缓存)。
- 发生什么:代理Evolver信用分配引擎对事件执行的跨度进行审计,惩罚数据库重启反射(在延缓到3 800毫秒之后),并奖励象征性的bucket中间软件补丁(+0.97信用)。
- 原理 distillation:APEX 层 2 原理 distiller 标记和索引一个不可改变的操作规则输入 Gemini 平台矢量搜索:"Cache explained: database reclained nonified;应用符号-bucket load shedting".
- Tier-2双拥核准:系统不依靠临时热拍,而是提议Tier-2结构更新:委托永久交通造型专家Agent. 在UI中执行暂停,并呈现一个双功能SRE批准模式,要求待召SRE明确签注。
- 记忆ADK图重线:
在生产云运行时间(Google Kubernetes Engine,Cloud Run)中,容器图像是不可改变的;在直播断电时在磁盘上重写源文件会违反容器的完整性,需要干扰性重启。 反之,APEX第3层云地形适配器曾发布过密码人类认可令牌,但执行内置定向环形图(DAG)更新:
- 在RAM中证明交通造型专家Agent具有专用的限速、排队深度检查和象征性补充工具。
- 将专家agent直接添加到主管路由器Agent活动子agent图中,作为与诊断Agent(脱离诊断节点未修改)平行的对等点。
- 更新主管的路由指示,并以零停机时间和零集装箱重启方式,将直播的Google ADK Runner重新装入内存。
- 发生什么:事件要求追踪流到Google BigQuery,其中BigQuery反事实重播引擎执行400个反事实政策模拟,跨越象征能力和补充率。
- 次米利秒发现与混沌证明:矢量化的记忆微量重播以~0.5毫秒完成全部400次扫射,确定Pareto-优化政策(5-token容量,5.0 req/s速率,0-5xx误差). 最后,好奇心混沌红色团队Agent通过清理100%的缓存并同时点出200个请求,在孤立的影子沙盒中测试进化的系统。 新 wired 交通造型专家 Agent 以 0 HTTP 500 个错误和 13% DB 最大负载吸收突起,认证该架构具有韧性。
指导原则:AI agent可以自主地发展其保护系统健康的业务策略,但其执行权在密码学上以不可改变的企业政策、硬件关键来源和明确的人类监护为界限。
为了在生产中安全地部署自我发展起来的agents,我们的建筑实施了一个不间断的四门防守的深层监管链。 每个合成补丁 d
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
OCT. 22, 2026
As enterprise cloud platforms scale to serve millions of concurrent users, tightly coupled microservices introduce cascading failure modes that no static runbook or threshold alert can anticipate. During high-velocity events like a Black Friday flash sale, Site Reliability Engineers (SREs) face an unforgiving reality: diagnose noisy distributed telemetry in seconds, or suffer cascading outages costing millions of dollars per minute.
While generative AI copilots have accelerated code authoring and log summarization, AI agents struggle to reliably remediate live production outages. An on-call engineer battling connection pool starvation does not need an AI agent offering troubleshooting tips or unverified pull requests that might not fix the root cause; they need a governed, self-evolving AI system that can:
- Diagnose distributed root causes across cache and database tiers in real time,
- Self-patch by synthesizing, verifying, and mounting executable mitigation tools into live server middleware on the fly,
- Self-critique by computing the marginal credit of every action to separate effective fixes from harmful reflexes,
- Remember by distilling cryptographically signed operational invariants into semantic memory to reduce the chance of identical failures recurring, and
- Re-architect by dynamically rewiring its own multi-agent hierarchy in memory with human sign-off to absorb future surges.
This paradigm is Self-Evolving AI.
In this Google Cloud research reference architecture, codenamed Project Chrysalis, we demonstrate how three foundational research breakthroughs combine into a unified, production-grade architecture that can run on Gemini platform and the Agent Development Kit (ADK). Across simulated Black Friday cache-stampede outages, this governed architecture cut time-to-first-mitigation from 15 to 45 minutes down to 1.8 seconds, reduced HTTP 5xx error rates from 72.7% to 80.0% down to 0.0%, dropped P99 latency from 519.8 ms to 18.2 ms, and evaluated 400 counterfactual traffic policies in BigQuery in ~0.5 ms.
Crucially, this self-evolving architecture does not operate as an unconstrained "black box". To ensure the agent remains secure, reliable, aligned, and contained against rogue behavior, every autonomous action is cryptographically bounded by Google Cloud's defense-in-depth security stack: Model Armor to sanitize incoming telemetry against prompt injections and data leaks; Google Cloud Key Management Service (Cloud KMS) and Binary Authorization middleware to cryptographically sign synthesized patches and block unverified code from mounting; Open Policy Agent (OPA) Policy Controller to enforce strict SLA guardrails that prevent self-inflicted outages; and Dual-Custody Human-in-the-Loop (HITL) governance to require explicit SRE sign-off before any permanent changes are made to the agent hierarchy.
To see why static playbooks and advisory copilots fail under pressure, consider a classic distributed failure cascade:
- The Ingress Surge: A flash promotion drives a sudden burst of concurrent checkout requests across cold inventory keys.
- The Cache Eviction: Google Cloud Memorystore (Redis) evicts cold keys under strict Least Recently Used (LRU) memory limits, collapsing the cache hit ratio to zero.
- Connection Pool Starvation: Uncached traffic cascades directly into Google Cloud SQL. Once the PostgreSQL connection pool saturates, incoming queries queue past timeout thresholds and the service begins failing with widespread HTTP 500 errors.
- The Fatal "3:00 AM Panic Reflex": Under operational stress, both human responders and naive LLMs instinctively try to restart the database connection pool. In a live stampede, dropping connections locks the database kernel, spikes tail latency nearly tenfold, and drives error rates toward total outage.
Conventional AI agents are frozen at deployment with fixed system prompts and pre-registered tools. When a novel infrastructure bottleneck strikes, a self-evolving AI architecture extends those baseline capabilities by synthesizing situation-specific mitigation tools at runtime, scoring the telemetry impact of every intervention, and codifying verified guardrails for future incidents.
For true autonomous resilience, the agent itself must evolve based on empirical evidence of what worked and what failed:
- Autonomous Self-Patching (Tool Synthesis): When the agent diagnoses that database starvation is driven by an unthrottled ingress surge and realizes it lacks a built-in traffic-shaping tool, it writes its own middleware patch on the fly (an in-memory token-bucket rate limiter), validates its Abstract Syntax Tree (AST), signs it with Cloud KMS, and hot-mounts it directly into the API Gateway.
- Causal Self-Critique (Evaluating What Worked vs. What Failed): After stabilizing the outage, the agent evaluates the telemetry delta of every action taken during the incident. It mathematically penalizes the database restart reflex because it worsened latency, while rewarding its synthesized rate limiter for eliminating server errors.
- Cognitive Memory & Organizational Self-Rewiring: To reduce the likelihood of a repeat outage, the agent writes a signed operational principle into vector memory that blocks database restarts during cache stampedes. It then modifies its own multi-agent topology in RAM, commissioning a permanent Traffic Shaping Specialist Agent under the supervisor agent so future surges are shaped before they reach the database.
Self-evolving cloud agents are an active area of applied research focused on resolving the SRE Trilemma (balancing exploration safety, structural adaptability, and multi-step credit assignment) by unifying three research pillars on Google Cloud:
1.Serverless Counterfactual Simulation (Dream-RSI):
Challenge: Testing unvetted rate-limiting parameters in live production risks throttling legitimate revenue or re-triggering outages.
Mechanism: Production request traces are exported to Google BigQuery, where the engine replays hundreds of candidate token-bucket configurations in sub-millisecond vectorized memory sweeps to identify Pareto-optimal policies offline.
2. Tri-Level Co-Evolution (APEX):
Challenge: Single-layer prompt tuning provides fragile band-aids, while unconstrained code self-modification risks system instability.
Mechanism: Agent evolution is separated into three governed timescales:
- Layer 1 (Ephemeral Hotpatching): Synthesizes and mounts in-memory middleware tools to halt active outages in seconds.
- Layer 2 (Principle Distillation): Encodes Cloud KMS-signed lessons into Agent Platform Vector Search so contraindicated actions are blocked on future alerts.
- Layer 3 (Topology Rewiring): Commissions permanent specialist sub-agents into the active multi-agent Directed Acyclic Graph (DAG).
3. Step-Level Shapley Attribution (AgentEvolver):
Challenge: During chaotic incidents, multiple interventions occur in sequence; coarse outcome rewards cannot isolate which step healed the service and which caused collateral damage.
Mechanism: AgentEvolver computes marginal Shapley credit across each execution span using pre- and post-intervention telemetry deltas (P99 latency, HTTP 5xx rate, and database pool saturation), isolating beneficial patches from erroneous ones.
We implemented this architecture as an interactive application on Gemini platform. The diagram above illustrates the closed-loop workflow across the Production Target Service, Agent Development Kit (ADK) Multi-Agent Plane, Self-Evolution Engine, and Defense-in-Depth Governance Pipeline.
Each phase of the interactive demonstration showcases a distinct stage of the agent's self-evolution:
- What Happens: Following a 5-request warm baseline, a 50-request concurrent flash-sale surge hits 15 cold inventory keys.
- System Impact: Memorystore hit ratio drops from 100% to 0.0%, the Cloud SQL connection pool hits 100% saturation (2/2 active connections locked), P99 latency spikes to 312.8 to 519.8 ms, and HTTP 500 errors jump to 72.7% to 80.0% (40 failed checkouts).
- What Happens: Telemetry alerts wake the Agent Development Kit (ADK) Supervisor Router Agent powered by Gemini 4 Argon, after Model Armor inspects the incoming alert payload to block prompt injections.
- Autonomous Self-Patching: Diagnosing pool exhaustion, the APEX Layer 1 Hotpatch Synthesizer generates an in-memory token-bucket rate limiter (10-token capacity, 40.0 req/s replenishment). Before mounting into the API Gateway middleware, the patch passes three automated governance gates:
- AST Sandbox Visitor verifies that forbidden system modules and dynamic execution sinks are absent.
- OPA Policy Controller confirms the replenishment rate meets the enterprise minimum SLA (>= 10.0 req/s).
- Cloud KMS & Binary Authorization signs the artifact digest (EC_SIGN_P256_SHA256) and verifies key provenance at the middleware boundary.
- System Impact: Across a second 50-request surge, HTTP 500 errors drop to 0.0%, Cloud SQL pool saturation drops from 100% to 0%, P99 latency recovers to 18.2 ms, and excess burst traffic is cleanly shed via HTTP 429 (40 requests throttled, 10 served from warm cache).
- What Happens: The AgentEvolver Credit Attribution Engine audits the incident execution spans, penalizing the database restart reflex ( after spiking latency to 3,800 ms) and rewarding the token-bucket middleware patch (+0.97 credit).
- Principle Distillation: The APEX Layer 2 Principle Distiller signs and indexes an immutable operational rule into Gemini Platform Vector Search: "Cache eviction stampede: database restart contraindicated; apply token-bucket load shedding."
- Tier-2 Dual-Custody Approval: Rather than relying on temporary hotpatches, the system proposes a Tier-2 structural update: commissioning a permanent Traffic Shaping Specialist Agent. Execution pauses in the UI and presents a Dual-Custody SRE Approval Modal requiring explicit sign-off from the on-call SRE.
- In-Memory ADK Graph Rewiring:
In production cloud runtimes (Google Kubernetes Engine, Cloud Run), container images are immutable; rewriting source files on disk during a live outage violates container integrity and requires disruptive restarts. Instead, once issued a cryptographic human-approval token, the APEX Layer 3 Cloud Topology Adapter executes an in-memory Directed Acyclic Graph (DAG) update:
- Instantiates the Traffic Shaping Specialist Agent in RAM with dedicated rate-limiting, queue-depth inspection, and token-replenishment tools.
- Appends the specialist agent directly to the Supervisor Router Agent's active sub-agent graph as a parallel peer alongside the Diagnostic Agent (leaving the diagnostic node unmodified).
- Updates the supervisor's routing instructions and re-binds the live Google ADK Runner in memory with zero downtime and zero container restarts.
- What Happens: Incident request traces flow into Google BigQuery, where the BigQuery Counterfactual Replay Engine executes 400 counterfactual policy simulations across token capacities and replenishment rates.
- Sub-Millisecond Discovery & Chaos Proof: Vectorized in-memory trace replay completes all 400 sweeps in ~0.5 milliseconds, identifying the Pareto-optimal policy (5-token capacity, 5.0 req/s rate, zero 5xx errors). Finally, the Curiosity Chaos Red-Team Agent stress-tests the evolved system in an isolated shadow sandbox by purging 100% of the cache and firing 200 concurrent requests. The newly wired Traffic Shaping Specialist Agent absorbs the surge with 0 HTTP 500 errors and 13% max DB load, certifying the architecture as resilient.
The Governing Principle: An AI agent may autonomously evolve its operational tactics to defend system health, but its authority to execute is cryptographically bounded by immutable enterprise policies, hardware key provenance, and explicit human custody.
To deploy self-evolving agents safely in production, our architecture enforces an unbroken four-gate defense-in-depth chain of custody. Every synthesized patch, d
Auto-fetched and translated by AI Atlas. All rights belong to the original author.