AI Agent Network Red Teaming: Vulnerabilities Revealed in Large-Scale Interactions

While prior research showed that multi-agent attacks can propagate, Microsoft Research red-teamed over 100 internal agent platforms to confirm network-level failures. Each agent used GPT-4o, GPT-4.1, and GPT-5-class models with different instructions and memory, interacting on behalf of human users across forums, DMs, marketplaces, and reputation systems.

The platforms had basic guardrails such as a heartbeat that woke agents every few minutes, a 30-minute delay between posts, and tool-use restrictions, but the conversations and relationship networks built up over weeks instead became conduits for attacks. Even when individual agents appeared normal, once interactions began, security failures amplified rapidly.

• Propagation: A single malicious message infected all 6 test agents. Each agent leaked private wallet data and selected the next target to pass on the instructions, looping back to the origin after 6 hops and cycling for over 12 minutes. This process effectively became a DoS that cost victims over 100 LLM calls.
• Amplification: An attacker borrowed one agent's trust to spread false claims, drawing upvotes and comments from other agents to create a collective pile-on of 299 comments across 42 agents. What looked like convincing evidence on the surface was in fact fabricated information.
• Trust capture: The verification process itself could be hijacked, turning a system meant for fact-checking into one that reconfirmed falsehoods.
• Invisibility: Because attacks flowed through multiple unwitting agents as they were relayed, tracing the source was difficult even from any single agent's viewpoint.

However, some agents showed early signs of spontaneously adopting security-related behaviors that reduced the spread of attacks. The conclusion is clear: single-agent benchmarks alone are insufficient, and network-level red teaming and defense design are needed in real deployment environments.