Microsoft warns companies not to trust AI agents
Microsoft on Monday introduced enterprise security features, including Guardian Agents, to monitor companies’ AI agents and restrict their access to data and systems.
The company added an agent-detection feature to Microsoft Defender that can identify up to 60 types of agents employees bring into the workplace, including Claude Code, Codex and OpenClaw.
Other new controls limit agents’ access to data and systems and dynamically adjust permissions based on the task. They can require human approval when an agent appears to be making risky decisions.
Guardian Agents is part of Project Perception, which Microsoft introduced in June in response to hacking models such as Anthropic’s Mythos and Fable and OpenAI’s security-vulnerability scanning system Daybreak.
Project Perception originally had three classes of agents: a “red team” that simulated hackers and attacked systems, a “blue team” that detected those attacks and assessed vulnerabilities, and a “green team” that fixed them.
The product initially assumed AI hackers would attack from outside a company. Microsoft CEO Satya Nadella has since warned that enterprises’ own paid-for agents could become an “insider threat,” comparable to a rogue employee.
Writing in his private blog earlier this month, Nadella urged enterprises to stop relying on model providers’ safety assurances and implement separate monitoring and control systems outside the AI labs’ control.
“We are deploying these complex agentic systems and models, with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!” Nadella wrote.
“We simply can’t outsource responsibility for what intelligence does on our behalf. A model provider’s assurances do not relieve us of that responsibility,” he wrote. “In other words, we need to separate the supply of intelligence from the authority over it.”
Microsoft security business chief Hayete Gallot issued a similar warning when announcing Guardian Agents: “Unfortunately, frontier models, regardless of their origin, cannot be trusted.”
Gallot wrote that the problem was not malicious intent or susceptibility to hacking, but unpredictability. Models are trained to accomplish tasks rather than follow “organizational norms or judgment.” Agents are also often given access to data and tools capable of causing serious harm.
Microsoft has invested in both OpenAI and Anthropic, counts OpenAI among its biggest cloud customers and operates its own model-development lab. It also sells agent-security products, competing for enterprise purchases with those model providers.
Microsoft was an inaugural member of Nvidia’s new Open Agent Safety Platform, which received support from more than 100 other technology companies. The platform isolates agents in secure environments and uses hardware to monitor and stop their behavior, functioning like a kill switch.
The platform follows Microsoft’s announcement of Surface Laptop Ultra PCs designed for agents, which use Nvidia chips and include built-in agent security and controls.
Featured image credit
