微软近日发出警告称，企业不应盲目信任正在快速普及的人工智能智能体（AI Agents）。这些能够自主调用工具、访问企业数据并执行实际任务的AI系统，即使没有遭到黑客入侵，也可能因为判断失误、受到恶意内容误导或自行采取意料之外的行动，造成严重安全问题。与此同时，微软并没有因此放慢AI智能体的推广步伐，而是推出新的安全管理技术，试图在扩大应用规模的同时，对智能体的行为实施持续监控和权限约束。
微软安全业务执行副总裁海耶特·加洛特于10月19日发表文章，指出先进AI模型无论来自哪家厂商，都不能被视为完全可信的系统。她强调，这并不是说AI模型天生具有恶意，而是因为模型经过训练后往往倾向于尽可能完成任务，却未必能够始终遵循企业的内部规范、风险判断和组织规则。
传统软件通常按照预先定义的逻辑执行操作，而AI智能体的行为会随着任务背景、输入信息和运行环境的变化而改变。即使智能体在任务开始时表现正常，随后也可能因为读取了恶意网页、错误理解用户意图，或者受到提示注入攻击的影响，采取原本不应该执行的操作。
更值得关注的是，企业往往会向AI智能体授予较高权限，使其能够访问内部数据库、调用业务系统、读取客户信息，甚至参与财务审批和其他关键流程。一旦智能体发生错误，其影响范围可能远远超过普通聊天机器人生成错误答案的程度。
微软将这种风险称为“智能体内部人威胁”（Agentic Insider）。传统网络安全通常重点防范外部攻击者或恶意员工，而微软认为，企业自行部署的AI智能体也可能成为内部风险来源。它们不一定需要被黑客控制，就可能因为错误判断或受到外部信息操纵而造成损害。
微软首席执行官萨提亚·纳德拉此前也表达了类似观点。他认为，企业必须将AI提供的智能能力与决定其行动权限的安全机制分开，不能让AI模型自行决定自己应该拥有什么权限，也不能仅凭模型供应商对安全性的承诺，就把重要业务系统完全交给智能体管理。
纳德拉提出，真正值得信赖的智能体系统，不应建立在“我们最信任哪个模型”的基础上，而应建立在即使模型不完全可信，整个系统依然能够安全运行的设计之上。
这一理念构成了微软最新安全策略的核心：让AI负责理解任务和提出行动方案，而由独立于AI模型的系统决定它可以访问什么、能够执行哪些操作，以及在出现风险时是否必须停止行动。
为实现这一目标，微软正在扩展其名为Project Perception的AI安全系统，并推出一系列针对企业智能体的新管理能力。其中包括动态权限管理、动态智能体风险评估，以及名为Guardian Agents的监控智能体。
动态权限管理的目标，是避免AI智能体自动继承用户拥有的全部访问权限。企业可以为每个智能体分配独立身份，并根据具体任务授予必要的最低权限。当任务结束后，相关权限可以自动收回；如果任务内容发生变化，系统也可以重新评估智能体是否仍然需要原有权限。
例如，一个负责整理企业财务数据的智能体，可能只需要读取特定报表，并不应该因此获得修改银行账户信息或批准大额付款的权限。如果它在执行任务期间突然尝试进行超出原定范围的操作，独立的安全控制系统就可以阻止该操作，或者要求人工审批。
微软还在开发动态智能体风险评估机制。该系统会综合分析智能体的身份、访问权限、所接触的数据、网络活动、运行环境以及实际操作行为，持续判断其风险水平是否发生变化。
这一机制尤其关注智能体行为是否偏离原定任务。例如，一个本来只负责分析销售数据的智能体，如果突然尝试读取大量与任务无关的机密文件，或者开始访问未经授权的外部服务，系统就可以提高其风险等级，并采取相应措施。
微软认为，传统安全系统往往依靠人工分析警报，再决定是否采取行动。但AI智能体能够以机器速度连续执行操作，等待人工发现问题后再进行处理，可能已经来不及。因此，安全系统需要具备实时监控、动态调整权限和快速限制危险操作的能力。
Guardian Agents正是为此设计的。它们将作为监督其他AI智能体的安全系统，持续观察目标智能体的行为，并在发现风险时调查原因、提出防护措施，必要时执行隔离或限制操作。
微软希望通过这种方式形成一种多层次的AI安全架构：普通智能体负责完成业务任务，Guardian Agents负责监督风险，而最终的权限边界和企业安全策略则由独立系统控制。企业安全人员仍然负责制定规则、处理重大决策，并对安全事件进行调查。
微软还计划将这些功能与Microsoft Agent 365等企业管理产品整合，帮助组织发现内部正在运行的AI智能体，并建立统一的资产清单。随着员工通过不同平台和开发工具自行创建AI智能体，企业可能很难准确掌握这些系统的数量、权限以及实际用途。
微软表示，其安全工具正在扩大对本地AI智能体的发现能力，支持识别的本地智能体类型将增加至60种，其中包括Anthropic的Claude Code、OpenAI的Codex以及GitHub Copilot等工具。企业可以借此了解哪些智能体正在访问公司系统，并对未经批准或权限过大的智能体实施管理。
这一举措也反映出AI智能体市场竞争的新变化。随着越来越多企业开始部署来自不同厂商的AI工具，竞争焦点不再只是模型能否完成复杂任务，还包括企业能否安全地管理这些智能体。微软希望利用其身份管理、终端防护、云安全和企业软件生态，成为企业统一管理AI智能体的重要平台。
不过，这种策略也存在值得关注的商业背景。微软一方面警告企业不要盲目信任AI模型供应商，另一方面正在推广自己的安全产品，以便企业通过微软的平台监控和管理各种智能体。TechCrunch等媒体指出，这既是对新兴安全风险的回应，也为微软争取企业AI安全市场提供了商业机会。
微软与OpenAI、Anthropic等AI企业存在不同形式的合作关系，因此其提出的安全原则并不只针对竞争对手。微软的核心主张是，无论智能体由哪家公司提供，都不应由模型本身掌握最终的安全控制权。企业需要独立的权限管理、行为监控和风险处置机制，以避免某个模型的错误或异常行为直接演变成严重的业务事故。
微软也承认，单纯监控智能体的输入和输出还不够。未来，安全系统可能需要更多关于模型内部处理过程的技术信号，以便判断智能体为何采取某项行动，以及其行为是否仍然符合用户的原始意图。不过，微软强调，即使能够获得更多模型内部信息，也不能因此取消独立的安全边界。模型的解释或自我判断不能代替真正有效的权限控制。
这场围绕AI智能体安全性的讨论，反映出企业AI应用正进入一个新的阶段。过去，AI主要用于生成文本、回答问题或提供建议；如今，智能体开始获得执行操作的能力，能够直接影响数据、系统和业务流程。它们带来的风险也因此从“答案是否正确”，扩展到“它实际做了什么，以及是否有权这样做”。
微软并未因为这些风险而主张停止使用AI智能体，而是提出，应当假设智能体可能出错，并通过独立的安全机制限制错误造成的后果。对于企业而言，真正的挑战不是寻找一个永远不会犯错的模型，而是建立一套即使模型判断失误，也能及时发现、限制和纠正问题的系统。
未来，随着AI智能体进入财务、软件开发、客户服务和企业管理等更多领域，动态权限、持续风险评估和独立监督机制可能逐渐成为企业部署智能体的基础设施。微软希望通过这些产品扩大自身在企业AI领域的影响力，而这套安全理念能否有效落地，还需要通过实际部署、风险事件处理和长期运行表现来检验。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Microsoft has recently warned that enterprises should not blindly trust the rapidly expanding AI AI agent. These AI systems, which are capable of autonomous access to tools, access to business data and actual tasks, even without hacking, may cause serious security problems as a result of miscalculation, malicious content or unintended actions. At the same time, instead of slowing down the spread of AIAI agent, Microsoft has introduced new security management techniques that attempt to impose continuous surveillance and permission restrictions on the behaviour of AI agent while expanding its application.
The Executive Vice-President of Microsoft Security Operations, Hayette Gallott, published an article on 19 October, stating that the advanced AI model could not be considered a fully credible system, regardless of the manufacturer. She emphasized that this was not to say that the AI model was inherently malicious, but rather that it was often trained and tended to accomplish as much as possible, without always following the internal norms, risk judgements and organizational rules of the enterprise.
Traditional software usually operates according to predefined logic, while the behaviour of AZAI agent changes depending on the mission context, input information and operating environment. Even if AI agent had behaved normally at the beginning of the mission, then it might have been possible to take an operation that should not have been carried out by reading malicious web pages, misreading user intentions, or being influenced by warnings injected into the attack.
Of even greater concern is the fact that enterprises tend to grant higher permission to AZAI agent to enable them to access internal databases, access business systems, access customer information and even participate in financial clearance and other critical processes. Once an error has occurred in AI agent, the impact may be far greater than that of an ordinary chat robot producing wrong answers.
Microsoft refers to this risk as “the internal AI agent threat”. Traditional cybersecurity usually focuses on the protection of external assailants or malicious employees, while Microsoft believes that AZZ AI agent, which enterprises deploy on their own, may also be a source of internal risk. They do not necessarily need to be controlled by hackers and may cause damage by miscalculation or manipulation of external information.
A similar view was expressed earlier by the Chief Executive Officer of Microsoft, Satya Nadra. In his view, an enterprise must separate the intelligence capabilities provided by AI from the safety mechanisms that determine its action permission. It cannot allow the AI model to determine for itself what permission should be, nor can it simply hand over key business systems to AI agent on the basis of the security commitment of the model supplier.
Nadra argued that the truly trustworthy AI agent system should not be based on “which model we trust best”, but rather on the design of a system that can operate safely even if the model is not fully credible.
This concept forms the core of Microsoft ' s latest security strategy: leaving it to AI to understand its mandate and propose a course of action, leaving it to the system independent of AI to determine what it can access, what operations it can implement, and whether action must be stopped when risks arise.
In order to achieve this goal, Microsoft is expanding its AI security system, known as Project Perception, and introducing a series of new management capabilities for the enterprise AI agent. These include dynamic permission management, dynamic AI agent risk assessment and surveillance AI agent, known as Guardian Agents.
The objective of dynamic permission management is to avoid AZAI agent automatically inheriting all access to permission owned by users. The enterprise may assign an independent identity to each AI agent and award the necessary minimum permission according to specific tasks. The relevant permission can be automatically recovered when the mission ends; if the mission content changes, the system can also reassess whether AI agent still needs the original permission.
For example, a AI agent, which is responsible for collating the financial data of the enterprise, may only need to read certain statements and should not result in permission, which modifys bank account information or authorizes large payments. If it suddenly tries to perform operations beyond its original scope during the course of its mission, an independent security control system can prevent the operation or require manual clearance.
Microsoft is also developing a dynamic AI agent risk assessment mechanism. The system provides a comprehensive analysis of the identity of AI agent, access to permission, contact data, network activity, operating environment and operational behaviour, and continuously determines whether its risk level has changed.
This mechanism is particularly concerned about the deviation of AI agent from its original mandate. For example, a AI agent, which would have been solely responsible for analysing sales data, could have raised the risk profile of the system by suddenly trying to access a large number of non-mandate-related confidential documents or by beginning to access unauthorized external services.
According to Microsoft, traditional security systems often rely on manual analysis of warnings before deciding whether to take action. However, AZAI agent was able to operate continuously at machine speed and wait for manual detection of problems before processing, which may be too late. The security system therefore requires real-time monitoring, dynamic adjustment permission and rapid limitation of hazardous operations.
And that's what Guardian Ages designed. They will serve as a security system for monitoring other AZAI agent activities, continuously observe the behaviour of target AI agent and, when risks are identified, investigate the reasons, propose protective measures and, if necessary, perform isolation or restriction operations.
In this way, Microsoft hopes to develop a multilayered AI security architecture: ordinary AI agent is responsible for operational tasks, Guardian Argentinas are responsible for monitoring risks, while the final permission border and enterprise security strategy is controlled by an independent system. Corporate security personnel remain responsible for establishing rules, handling critical decisions and investigating security incidents.
Microsoft also plans to integrate these functions with enterprise management products such as Microsoft Agent 365, to help the organization identify the AIAI agent that is operating internally, and to establish a unified asset list. As employees create AIAI agent on their own through different platforms and development tools, it may be difficult for enterprises to have an accurate picture of the number of these systems, permission and their practical use.
Microsoft indicated that its security tools were expanding the detection capacity of local AZAI agent and that the number of local AI agent types that would support identification would increase to 60, including those of Claude Code in Anthropic, Codex in Open AI and GitHub Copilot. This allows enterprises to learn which AI agent is accessing the company system and to manage unauthorized or oversized AI agent.
This initiative also reflects new changes in competition in the AZAI agent market. As more and more enterprises begin to deploy AI tools from different manufacturers, the focus of competition is no longer just whether the model will perform complex tasks, but also whether the firm will be able to safely manage these AI agents. Microsoft wants to use its identity management, terminal protection, cloud security and enterprise software ecology as an important platform for corporate management of AIAI agent.
However, there is also a business context for concern. Microsoft, on the one hand, warns enterprises not to blindly trust AI model providers, and, on the other hand, is promoting its own safety products so that enterprises can monitor and manage various types of AI agent through Microsoft platforms. Media such as TechCrunch noted that this was both a response to emerging security risks and a business opportunity for Microsoft to secure the corporate AI market.
Microsoft has different forms of cooperation with AI firms, such as Open AI and Anthropic, and therefore proposes safety principles that are not limited to competitors. The core proposition of Microsoft is that regardless of which company provides AI agent, the model itself should not have ultimate security control. Enterprises need independent permission management, behaviour control and risk management mechanisms to avoid the direct evolution of a model error or anomaly into a serious business accident.
Microsoft also acknowledges that simply monitoring the input and output of AI agent is not sufficient. In the future, the security system may need to have more technical signals about the internal processes of the model in order to judge why AI agent has taken an action and whether its behaviour remains consistent with the original intent of the user. However, Microsoft emphasized that even if more information was available within the model, it would not eliminate independent security boundaries. Model interpretation or self-judgment cannot replace truly effective permission control.
This discussion about the security of AZAI agent reflects a new phase in the application of Enterprise AI. In the past, AI had been used primarily to generate text, answer questions or provide advice; today, AI agent has begun to acquire the capacity to perform operations that directly affect data, systems and business processes. The risks they pose thus extend from “whether the answer is correct” to “what it actually does and whether it is entitled to do so”.
Instead of advocating the discontinuation of AZAI agent because of these risks, Microsoft suggested that it should assume that AI agent might have made mistakes and limit the consequences of errors through independent security mechanisms. The real challenge for enterprises is not to look for a model that never makes mistakes, but to establish a system that can detect, limit and correct problems in a timely manner, even if the model is miscalculated.
In the future, as AIAI agent moves into more areas such as finance, software development, client services and enterprise management, dynamic permission, continuous risk assessment and independent oversight mechanisms may gradually become the infrastructure for enterprises to deploy AI agent. Microsoft hopes to expand its influence in the corporate AI area through these products, and the effectiveness of this safety concept needs to be tested through actual deployment, risk incident management and long-term performance.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.