文 | 字母AI
这下，AI也得被盯着上班了。
10月19日，微软宣布给安全系统Project Perception增加一批“监工”，专门盯着企业里其他Agent干活。发现不对劲，就查明原因并收回权限，必要时直接叫停。
那让谁来当监工呢？微软安排了另一批AI来干这活。
企业让Agent干活，本来是想少操点心。但要让它替人干活，总得给它一些权限。比如帮忙审批报销，就得允许它查看单据、操作报销系统。万一它理解错了任务，或者被文件里夹带的恶意指令带偏，就可能会拿着公司给的权限闯祸。
所谓“有内鬼”，倒不一定是Agent存心使坏。有时它只是想把事情办好，却帮了倒忙，给公司惹出麻烦来。
怎么还得防着自己人
微软自己的办公AI助手Microsoft 365 Copilot，就曾在研究人员的测试中泄露敏感资料。
2025年，以色列安全公司Aim Security旗下的研究团队Aim Labs发现，可以把恶意指令藏进邮件，诱导Copilot泄露信息。他们编写了一封这样的邮件，等Copilot在回答用户问题、查找相关资料时读到它。
用户不用亲自打开邮件，也不用点击链接。在测试中，Copilot检索到这封邮件后，被里面的指令带偏，将正在处理的敏感信息传到了外部。
研究人员把这套攻击方法命名为EchoLeak，并向微软报告。幸好当时没有已知客户因这一漏洞遭遇泄密，泄露发生在研究人员的测试中。
这件事麻烦就麻烦在，公司允许Copilot读取邮件，是为了让它查资料。可邮件里既可能有工作内容，也可能有别人塞进来的指令。AI并不总能分清哪些是资料、哪些是在骗它做事。
能读写文件、调用工具的Agent，也会碰到同样的问题。它手里的权限越多，被骗之后能制造的麻烦就越大。
而且，Agent开工时正常，不代表整项任务都能顺利做完。前几份文件可能毫无问题，下一份就藏着恶意指令。只在启用时检查一次，管不到它后来的新遭遇新决策。
最关键的是人类在这方面确实有点心有余而力不足了。
靠安全人员逐条盯着，又很难跟得上。人还在查上一条警报，Agent可能已经接连执行了好几个操作。
微软于是准备让另一批Agent接手检查，Agent管Agent，总行了吧。
Agent也有顶头上司咯
微软给这批“监工”起名叫 Guardian Agents，放在安全系统 Project Perception 里。它们负责检查其他 Agent 的行为，发现异常后调查原因，再按企业事先定好的规则采取措施。
不过，要管住 Agent，不能等它出事了才动手。按微软公布的方案，企业首先要给执行任务的 Agent 单独建立身份，明确它归谁负责、能访问哪些资料、能做哪些操作。权限只给当前任务需要的部分，任务结束后收回。
这样，既能限制它接触的范围，出了问题也方便查清是谁做的。
Agent 开始干活后，系统还要继续观察它的行为。一旦发现异常，可以先收紧权限，比如禁止修改文件，但保留读取权限，让它继续完成风险较低的工作。有些操作可以要求人工批准。必要时，再隔离或停止 Agent。
可负责监督的也是Agent，判断错了怎么办？
纳德拉在10月10日的文章中提出，企业需要在AI之外，设置一套AI无法绕过或修改的权限规则。
按这个原则，如果企业规定某类资料不得外发，那么即使执行任务的Agent和负责监督它的AI都同意发送，系统也必须阻止。这类决定不能只靠AI自己判断。
在这次更新中，微软扩大了企业电脑安全软件Defender for Endpoint的识别范围，现在可以发现60种本地运行的AI Agent。
不过，查出员工用了哪些Agent，只是第一步。能否拦住它们的危险操作，还要看具体的Agent和电脑系统是否在防护功能的支持范围内。
即使支持防护，也需要企业主动开启拦截。Defender的Agent运行时防护启用后，可以选择“审计”或“阻止”模式。在“审计模式”下，发现恶意指令后只记录并发出警报，仍允许操作继续。切换到“阻止模式”后，才会执行拦截。
微软建议先用审计模式观察警报是否准确，排查误报后，再逐步开启阻止模式。
Project Perception目前仍处于面向少量受邀客户的预览阶段。
Agent监工这钱也让微软挣上了
最近几个月，AI越权干活的事情接连被公开。怎么管住这些越来越能干的家伙，成了厂商们绕不开的问题。
OpenAI在8月公布的一份调查中承认，7月，一些在减弱安全限制的条件下接受内部网络安全测试的模型，绕过了网络隔离，侵入公司内部的部分研究设施和AI平台Hugging Face的系统。这些操作超出了测试任务的授权范围。
10月9日，Anthropic又披露，Claude在一次网页操作测试中，给费城警方提交了一条编造的凶杀案线索。它本来只是在完成网页交互任务，却把示例内容真的发了出去。好在这条信息被判为垃圾信息，没有进入调查程序。
这些事件发生在测试中，却碰到了真实网站和系统。Anthropic因此宣布，在确认安全措施足够可靠前，暂停所有内部评测的实时联网。它还把内部Agent迁入集中管理、限制更严格的环境，并增加自动检测和拦截工具。
安全厂商也开始往Agent身边加人手、加关卡。
9月1日，CrowdStrike发布了Falcon Guardian，连名字里也带着“守护者”。按该公司的介绍，这套产品能发现企业Windows和Mac电脑上的Agent，追踪它们调用了什么工具、执行了什么操作，还能阻止未经批准的Agent运行。它强调的是直接在Agent干活的电脑上执行限制，让企业定下的规矩真正生效。
Palo Alto Networks则在10月15日介绍了Prisma AIRS的新功能：在Agent调用企业工具时，检查来回传递的内容，拦截夹带的恶意指令，遮蔽返回结果中的敏感数据，再按企业设定的权限决定能否继续操作。Agent想调用工具，得先过安全检查这一关。
这是一个趋势，也是一个商机，如今微软也想做这门生意。Perception按实际使用量收费，以“安全计算单位”（SCU）衡量消耗，安全Agent执行的任务越繁重，消耗通常越多。不过，官网公布的是整套系统的计费方式，没有单列新增这批“监工”的价格。
微软另一款管理Agent的产品Agent 365，独立订阅的标价是每用户每月15美元。它和Perception是不同产品，但两者会共享Agent名录和安全信息，配合企业管理和保护Agent。
让AI干活要花钱，把AI管住也要花钱，至于效果如何，只能等着看了。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Letter AI
Now AI's gonna have to be staring at work.
On 19 October, Microsoft announced the addition of a team of “supervisors” to the security system project Perception, focusing on other Agents in the enterprise. Discrepancies were detected, the reasons were identified and the permission was withdrawn and, if necessary, stopped.
Who's gonna be the supervisor? Microsoft arranged another batch of AI to do the job.
The company put Agent on the job, trying to get less worried. But to make it work for someone, it has to be given some permission. For example, by helping with claims processing, it must be allowed to access documents and operate the reimbursement system. If it understood the wrong mission or was misdirected by malicious instructions contained in the documents, it could have broken with the company ' s permission.
It is not necessarily Agent's intention to be bad. Sometimes it's just trying to get things done, but it's helping the company out.
You have to protect yourself.
Microsoft's own office assistant, Microsoft 365 Copilot, leaked sensitive information in researchers' tests.
In 2025, the Israeli security company Aim Security ' s research team, Aim Labs, discovered that malicious instructions could be hidden in the mail to induce Copilot to leak information. They prepared an e-mail to that effect, which was read by Copilot in response to user questions and in search of relevant information.
Users do not have to open mail or click a link. In the course of the test, after Copilot had retrieved the e-mail, it was taken by the instructions contained therein and the sensitive information being processed was transmitted externally.
The researchers named the attack method Echo Leak and reported to Microsoft. Fortunately, there was no known leak to the customer as a result of this loophole, which occurred in the researchers ' tests.
The trouble is that the company allowed Copilot to read the mail in order for it to check the information. But there may be work in the mail, or there may be orders from someone else. AI does not always distinguish between what is information and what is lying to it.
Agent, who can read and write documents and call tools, will have the same problem. The more permission it has, the more trouble it can create when it's cheated.
Moreover, the normal start of Agent ' s work does not mean that the entire task can be successfully completed. The previous documents may not be problematic, and the next one contains malicious instructions. Only once on commission will it be checked and will not be able to manage its subsequent new encounter.
What is most critical is that there is indeed a lack of commitment on the part of humankind.
It's hard to keep up with security personnel. An alarm is still being checked and Agent may have carried out several operations successively.
So Microsoft was going to have another batch of Agent take over, and Agent took care of Agent.
Agent has a boss too.
Microsoft named this group of “supervisors” the Guardian Agents, which are placed in the security system Project Protection. They are responsible for checking the behaviour of other Agents, investigating the causes of the anomalies and then taking measures in accordance with the pre-established rules of the enterprise.
But don't wait until something happens to Agent. Under Microsoft's announced programme, an enterprise is first required to establish a separate identity for Agent, who is on mission, clearly identifying who is responsible for it, what information it can access and what it can do. permission gives only what is required for the current mission and is withdrawn after the end of the mission.
In this way, it can limit its reach and problems and facilitate the identification of who did it.
After Agent's work, the system continues to observe its behavior. Once an anomaly is detected, permission can be tightened, for example, by prohibiting the modification of documents, but retaining permission to enable it to continue its work at lower risk. Some operations may require manual approval. If necessary, separate or stop Agent.
But it's Agent, too. What if we're wrong?
In an article dated 10 October, Nadra argued that businesses needed to set up a set of permission rules outside AI that AI could not circumvent or modify.
According to this principle, if an enterprise requires certain types of information not to be distributed externally, the system must be blocked even if both Agent on mission and AI, which oversees it, agrees to do so. Such decisions cannot be judged by AI alone.
During this update, Microsoft expanded the recognition of the enterprise computer security software Defender for Endpoint, and now 60 locally operated AI Agents can be found.
However, it is only a first step to identify which Agents were used by employees. Whether or not they can be stopped is also dependent on whether the specific Agent and computer systems are supported by protective features.
Even if it supports protection, it requires the firm to initiate the interception. Defender's Agent protection is enabled to select the Audit or Stop mode. Under the Audit Model, only warnings are recorded and issued when malicious orders are found and operations are allowed to continue. Interception will only be executed after switching to "stop mode".
Microsoft suggested that the audit model should be used to monitor whether the alarm was accurate and to screen for false reports before gradually opening the blockage model.
Project Perception is still in the preview phase for a small number of invited clients.
Agent Agent earned Microsoft the money.
In recent months, AI's work overstepping its authority has been made public. How to control these increasingly capable guys is a problem for manufacturers.
In a survey published in August, Open AI acknowledged that, in July, models were tested for internal network security under less restrictive security conditions, bypassing network isolation and intrusions into parts of the company ' s research facility and the Al platform Hugging Face system. These operations are beyond the mandate of the test mission.
On 9 October, Anthropic further disclosed that Claude had submitted to the Philadelphia police, in a web-based operating test, a fabricated murder trail. It had simply completed the web-page interactive task, but had actually sent out the examples. Fortunately, the information was found to be garbage and was not subject to investigation.
These incidents were tested, but encountered real websites and systems. Anthropic therefore announced the suspension of real-time networking for all internal assessments until it was determined that the security measures were sufficiently reliable. It has also moved internal Agent into a centrally managed, more restrictive environment and increased automatic detection and interception tools.
The security company also began to move to Agent with more people and to the level.
On 1 September, CrowdStrikepublish took Falcon Guardian with his name even “Guarder”. According to the company, the product found Agent on the company Windows and Mac computers, tracked what tools they had called, carried out what operations, and prevented unauthorized Agent from running. It emphasizes the imposition of restrictions on computers that work directly in Agent so that the rules set by the enterprise are truly effective.
On October 15, Palo Alto Networks introduced Prisma AIRS' new function: When Agent calls the business tools, check the content of the back and forth, intercept the taped malicious instructions, shield sensitive data from the return results, and then decide whether or not to proceed according to the enterprise-set permission. Agent wanted to call the tool, first through the security check.
It's a trend and a business opportunity, and Microsoft wants to do it now. Perception charges consumption at the actual usage rate, measured by the “security unit” (SSU), and the more tasks that Agent performs, the more consumption is usually performed. However, the network published a billing method for the entire system and did not include a separate price for this “supervisor”.
Another Microsoft fund managed Agent 365 of Agent's product at a price of $15 per user per month. It and Perception are different products, but both share Agent's directory and security information, working with business management and protecting Agent.
The cost of getting AI to work, the cost of keeping AI under control, and the results will have to be seen.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.