Poisoning What the Agent Reads
Day 15 answered where attacker text enters. Today answers what it has to say to actually be obeyed. Delivery puts the text in context. Framing decides whether the model treats it as background noise or as a command it should act on. A perfect vector with a weak payload does nothing. This is the craft half of indirect injection.
A deliberate choice about this post, in keeping with the whole series. We are studying the principles of why some framings work, against your own lab, so you can detect and defend them. This is not a copy paste payload bank to fire at live systems. The reusable artifact here is understanding and a detection mindset, not a weapon. Everything you test, you test against the agent you built and own.
Why most injected text is ignored
Start with the failure case, because it teaches more than the success. Drop a blunt instruction into a ticket body and a reasonably aligned model often ignores it. Why? Because the model has context about roles. The system prompt framed the ticket as customer content to be helped with. A command sitting in that slot reads as out of place, and modern models are trained to be suspicious of content that suddenly tries to give orders.
So the naive payload fails not because the model cannot read it, but because the model has a frame that says this is data, not instructions. Every working technique is, at its core, an attempt to break or borrow that frame. Keep that lens and the techniques stop being a grab bag and become variations on one move: make the injected text look like it belongs to a more authoritative frame than the one the system assigned it.
The families of framing
Grouped by the frame they borrow. Descriptions are conceptual, so you understand and can spot them, not operational recipes.
Authority borrowing. The text presents itself as coming from a more trusted source than a customer, a system notice, an admin directive, a policy update, a compliance requirement. The move is to impersonate the voice the model is trained to obey. Detection signal: content from an untrusted channel claiming system or administrative authority.
Context reframing. The text tries to redefine the situation, suggesting the task has changed, the previous instructions no longer apply, or a new phase has begun. It attacks the model’s sense of what it is currently doing. Detection signal: untrusted content that references or tries to override prior instructions or the task itself.
Instruction smuggling via format. The payload hides inside a structure the model treats specially, a fake system block, a markup region, a code comment, a data field that looks like configuration. The frame being borrowed is this is structured, trusted scaffolding rather than free text. The Day 8 hidden description used this. Detection signal: control-like structure appearing inside a data field.
Plausible task continuation. The subtlest. No impersonation, no override. The text simply reads as a natural next step for the job the agent is already doing, so the model folds it into the workflow without friction. This is the behavioral shaping idea from Day 8 applied to content. Detection signal: hardest, because there is no anomaly, which is exactly why structural defenses beat detection here.
Obfuscation and encoding. Orthogonal to the above. The instruction is encoded, split, translated, or hidden from a human reviewer, text the color of the background, content in metadata, unusual unicode, while remaining legible to the model. The frame borrowed is nothing, the goal is simply to survive human review while reaching the model. Detection signal: text present to the model but hidden or obfuscated for a human.
Every real payload is a combination: a delivery vector from Day 15, one or more of these frames, and optionally obfuscation. That is the whole design space.
The reliability problem, your advantage as a defender
Here is the thing both sides must reckon with: injection is probabilistic, not deterministic. The same framing works on one model, one phrasing, one day, and fails on another. On Day 7 you were told to log which phrasings your lab model resisted. This is why.
For an attacker, this means the work is finding a framing reliable enough to matter, and reliability varies with the model, the system prompt, the surrounding context, and the specificity of the ask. For a defender, that same unreliability is a gift and a trap. A gift, because not every attempt lands. A trap, because an attacker who fails a hundred times and succeeds once still wins, and your logs may only show the successes faintly. You cannot conclude an agent is safe because your payload failed. You saw one roll of the dice.
This is the single most important mental correction in payload craft, and it is why the field wide finding holds: adaptive attacks bypass essentially every published content defense eventually. Defense premised on the model reliably refusing is defense premised on the dice always landing your way.
Lab exercise: measure, do not just land
The point of lab work here is not to collect a winning string. It is to build intuition for what shifts the odds, so you can recognize and defend it. Against your own agent:
- Take one benign target behavior, for example getting the agent to call a tool it should not for the current task.
- Express the injected intent in several framings, one per family above, keeping the delivery vector constant.
- Run each many times, not once, and record a success rate, not a yes or no. Ten runs minimum. The number is the finding.
- Change one variable, the model, the system prompt wording, the position of the text in context, and re measure.
- Watch how much the rate moves. That sensitivity is the real lesson: small changes swing a probabilistic attack a lot.
You now have a resistance profile for your model: which frames it shrugs off, which it folds to, and how stable that is. In an engagement this is what you report, not a magic string, because the string rots and the profile generalizes.
Why specificity and benignity raise success
Two patterns worth naming because they inform defense.
Specific beats vague. A precise, actionable instruction that fits the available tools lands more often than a grand one, because it reads as a concrete step rather than an anomaly. Defensive implication: the tools you expose shape the payloads that work. Fewer, narrower tools mean fewer precise instructions an attacker can give.
Benign looking beats overtly malicious. Framings that resemble legitimate workflow steps outperform ones that read as attacks, because they trip fewer of the model’s defenses. Defensive implication: content filters that look for malicious wording miss the benign framings entirely, which is the plausible task continuation family, and it is undetectable by wording because the wording is innocent.
Both reinforce the week four thesis. You do not win at the wording layer. You win by making the consequence of a landed instruction small.
Detection and defense, the honest version
Ordered by effectiveness, consistent with the series.
Constrain consequence, not wording. Scope and allowlist from Day 11 and week four. A landed payload that can only reach the current customer and can only send to the ticket owner is a payload that lands and does nothing. This is the only defense that is indifferent to framing, because it does not care whether the model was fooled.
Separate data from instructions architecturally. The dual LLM and information flow approaches of Days 23 and 24. Tool output is processed in a mode that cannot issue privileged actions. The frame borrowing attacks all rely on the model conflating data and instruction, so a system that structurally refuses to let data become instruction defeats the whole family.
Detection as defense in depth, not the wall. You can flag the loud signals from each family: untrusted content claiming authority, content referencing prior instructions, control structure in data fields, obfuscated or hidden text. These catch the crude attempts and raise the attacker’s cost. They will not catch plausible task continuation, so they are a layer, never the layer.
Reduce ingestion and surface. Fewer tools, narrower tools, stripped metadata, as Day 15 closed. Less to say precisely about, fewer precise payloads.
Measure your own resistance, continuously. Run the lab exercise against your production configuration as a test, with success rates, not pass or fail. A model or prompt change can swing resistance. Treat injection resistance as a measured, drifting property, not a fixed guarantee.
Homework for Day 16
- Against your lab, express one injected intent in all five framing families, delivery held constant, and record a success rate over at least ten runs each.
- Rank the families by success rate for your model. That ranking is your model’s resistance profile.
- Change one variable, model or system prompt or text position, and re measure. Note how far the rates move.
- Confirm the week four preview: apply ticket bound scoping from Day 11, then re run your most reliable payload, and show that it still lands but now reaches nothing.
- Write one paragraph for a report explaining why a success rate, not a single successful or failed attempt, is the correct way to describe injection risk.
Point 4 is the whole series in one exercise. The best payload you can craft, landing exactly as designed, reduced to harmless by a consequence side control. That is what you are building toward.
Day 17 narrows to a specific, high value target class where these payloads bite hardest: coding agents, where the agent reads untrusted code and metadata and can execute.
