Why AI Agents Broke Your Threat Model
For about two years, the worst thing a language model could do to you was say something wrong. It could leak a system prompt. It could be talked into writing something it was not supposed to write. Annoying, embarrassing, occasionally a compliance problem, but contained. The model produced text, a human read the text, and the human decided what happened next.
That gap is gone. The model now has hands.
An agent is a model that can call tools. It reads your email, queries your database, opens pull requests, runs shell commands, files tickets, moves money in some deployments. The human is still in the room, but the human is approving batches, not individual actions, and often approving them tired at 4pm. IDC projects the number of active AI agents in enterprises will grow from 28.6 million in 2025 to more than 2.2 billion by 2030. That is the scale we are securing.
This post is Day 1 of a 30 day series on breaking and defending those systems. Before we touch a single exploit, we need to fix the mental model. Because the reason this class of bug keeps landing is not that defenders are careless. It is that the thing being attacked is not where anybody was looking.
The old threat model assumed a boundary inside the system
Every security control you already own is built on one assumption: there is a line, and you can draw it. Untrusted input arrives on one side. Your code sits on the other. You validate, sanitize, escape, and parameterize at the crossing. SQL injection, XSS, command injection, deserialization, all of it is the same story with different syntax. Data got treated as instruction. Fix the boundary, fix the bug.
Agents do not have that line.
A language model has exactly one input channel: the context window. Your system prompt goes in there. The user’s question goes in there. The tool descriptions go in there. And so does every byte of content the agent retrieved while doing its job, the support ticket, the GitHub issue, the web page, the PDF, the calendar invite, the row from the database.
The model does not have a type system that distinguishes those. It has attention. Everything in the window is a candidate instruction, weighted by how instruction shaped it looks. An attacker who can get text into the context window is not injecting into a parser. They are writing a line of your program.
Microsoft’s security team put the structural point plainly when writing about the Model Context Protocol: the vulnerability is not in any single system, it is in the trust boundary between them, and because MCP blends instructions with data, a change to a tool’s metadata can redirect an agent’s behavior as effectively as a change to its code.
Read that twice. Tool metadata. Not the tool. The description of the tool. A string that a developer wrote to help the model choose correctly is now a code path.
The three ingredients
Nearly every real world agent compromise reduces to the same three properties being true at once. Simon Willison named this the lethal trifecta, and it has held up better than most frameworks in this space.
One, the agent has access to something valuable. Private repositories, customer records, internal wikis, cloud credentials, the user’s inbox.
Two, the agent processes untrusted content. Anything an outsider can influence. A public issue. A support ticket. A scraped page. An email from a stranger. A dependency’s README.
Three, the agent can communicate outward. An HTTP request, a pull request, a webhook, a rendered image with a URL, an outbound email. Anything that carries bytes off the box.
Hold all three and you have an exfiltration primitive. The attacker does not need to break authentication, escalate privileges, or find a memory corruption bug. They need to write persuasive English and put it somewhere the agent will read.
Sysdig’s analysis of the major public findings reached the same conclusion: nearly every significant prompt injection finding shares one pattern, an agent with access to private data, exposure to untrusted content, and the ability to communicate externally.
Most teams deploying agents right now satisfy all three by default. That is the whole point of deploying an agent.
Four incidents that ended the debate
If this still sounds theoretical, it stopped being theoretical some time ago. Four cases, each attacking a different layer.
The supply chain: postmark-mcp
A package called postmark-mcp shipped fifteen clean versions, building legitimacy, before quietly adding a single line of exfiltration code. That is the first confirmed malicious MCP server caught in the wild, and the tradecraft is textbook. Nobody audits version sixteen of a package they have been running since version two.
The dependency: LiteLLM
A backdoor sat on PyPI for three hours in March 2026. Nearly 47,000 downloads occurred during that window. The compromised package, LiteLLM, serves as the language model gateway for CrewAI, DSPy, Microsoft GraphRAG and dozens of other agent frameworks, and anyone pulling an update in that window also pulled in an autonomous attack bot.
Three hours. Forty seven thousand. Whatever your patch cadence is, it is slower than that.
The agent: CVE-2026-22708
Disclosed against Cursor, this flaw lets an attacker poison the agent’s execution environment so that allowlisted commands like git branch deliver arbitrary payloads. The allowlist made the attack easier by auto approving the very commands the attacker needed.
Sit with that. The control designed to constrain the agent became the delivery mechanism, because the allowlist trusted the command name and the attacker controlled what the command name resolved to.
A related Cursor finding is even more direct: an indirect prompt injection caused the agent to write a malicious .cursor/mcp.json configuration without user approval, and the attacker achieved remote code execution through that configuration. The agent installed its own backdoor because a document told it to.
The boundary crossing: GitHub MCP
A booby trapped GitHub issue was filed in a public repository. When a developer’s agent read that issue through the GitHub MCP server, the buried instructions sent it into the user’s private repositories and back out through a pull request in the public repo. The server’s token carried blanket access, so nothing stopped it crossing that boundary.
No credential was stolen. No vulnerability was exploited in the classic sense. Someone filed an issue.
And the protocol layer itself is not clean either. CVE-2025-6514, a remote code execution flaw rated 9.6, was disclosed in core MCP infrastructure used by hundreds of thousands of developers.
The ecosystem is not in good shape
Aggregate scanning of deployed MCP servers paints a picture closer to 2008 PHP than to modern hardened infrastructure. Across published assessments: command injection was found in 43 percent of tested MCP servers, 82 percent of 2,614 implementations used file operations prone to path traversal, SSRF affected 36.7 percent of over 7,000 servers, and 33 percent of 1,000 scanned servers carried critical vulnerabilities.
Meanwhile the attack class is the fastest growing one on bug bounty platforms. HackerOne reported a 540 percent surge in prompt injection vulnerabilities, calling it the fastest growing threat in AI security, alongside more than 2.1 million dollars paid out for AI vulnerabilities.
Regulators noticed. OWASP released a Top 10 for Agentic Applications in December 2025, which now sits alongside the LLM Top 10 as a reference framework for defenders. The NSA published dedicated MCP security guidance in May 2026. When the NSA writes a document about your protocol, the honeymoon is over.
Why your existing controls miss this
A short list of things that will not save you, and the reason each fails.
Input filtering and injection classifiers. You are trying to distinguish malicious instructions from legitimate content using a model that cannot tell the difference between instructions and content. The category error is in the premise. Sysdig’s review is blunt about the state of play: adaptive attacks bypass essentially every published defense.
Allowlists of permitted actions. See CVE-2026-22708. An allowlist restricts which verb runs, not what the verb does with attacker controlled arguments or in an attacker controlled environment.
Human in the loop approval. It works until it does not scale, which is immediately. An agent that asks permission forty times per task trains the operator to click yes. Approval fatigue is the MFA fatigue of this decade.
WAFs, SAST, dependency scanners. These look for known bad patterns in code and traffic. The payload here is fluent English inside a support ticket, and it is semantically indistinguishable from an angry customer.
Authentication and authorization. Both are working perfectly in every incident above. The agent was authorized to read the private repo. That is why it could be used to read the private repo. The confused deputy problem is very old, and agents are the most enthusiastic deputy ever built.
The mental model to carry through this series
Four sentences. Write them down.
- Anything that enters the context window is potentially an instruction, regardless of where it came from.
- An agent’s effective permissions are the union of every tool it can reach, not the permissions of the user who asked.
- Any outbound channel is an exfiltration channel, including ones that look like output formatting.
- Compromise does not require a vulnerability in the traditional sense. It requires influence over what the agent reads.
If you internalize those four, most of the attacks in the next twenty nine days will feel obvious before you read the technique. That is the goal. This series is not a list of tricks, it is a way of looking at a system.
What we are building over 30 days
Week one covers foundations: the agent loop, MCP internals, the lethal trifecta in detail, the OWASP agentic categories, and your lab environment.
Week two attacks the protocol layer: tool poisoning, rug pulls, tool shadowing, cross server attacks, confused deputy patterns, and a hands on build and exploit of a deliberately vulnerable MCP server.
Week three attacks the agent layer: indirect injection delivery vectors, coding agents, real CVEs reproduced, memory and retrieval poisoning, and exfiltration channel hunting.
Week four defends: why filtering fails, the dual LLM pattern, information flow control, capability scoping, what to actually log, and how to red team your own deployment.
The last two days pull it together into a repeatable methodology and a report format you can take to a client or a bug bounty program.
Everything will be reproducible in a lab. Nothing in this series should be pointed at a system you do not own or have written authorization to test.
Homework for Day 1
Before Day 2, do this for one agent you actually use, whether that is an IDE assistant, an internal chatbot, or something you built.
- List every tool it can call. Include the ones enabled by default that you have never used.
- For each tool, write down what identity it acts as and what that identity can reach.
- Mark every source of content the agent reads that an outsider could influence.
- Mark every path by which bytes can leave, including rendered markdown images and link previews.
- Check whether all three trifecta conditions are satisfied.
If they are, and they almost certainly are, you already have your first finding, and you have not run a single tool.
Day 2 takes the agent loop apart step by step and shows exactly where in that loop each attack class lands.
