Memory and RAG Poisoning
Every attack so far has been live. The attacker plants text, the agent reads it, the payload fires, all in one session the attacker is watching. Today we break that assumption. Memory and RAG poisoning separate the planting from the firing in time. The attacker writes the payload once, walks away, and the agent detonates it later, in a session the attacker is not present for, triggered by a victim who had nothing to do with the original input.
This is the stealthiest vector in the series, and the hardest to trace, because by the time the damage happens the cause is long gone from view. It is also increasingly common, because persistent memory and retrieval are exactly the features teams are racing to add to make agents feel smart.
The shift: delivery and detonation are separated
Go back to the loop from Day 2. Almost everything so far has lived at the ingest stage, tool output entering context during the current run. Memory and RAG poisoning live at the assemble stage, the very first step, where context is built before the user even speaks.
That is the whole difference. If the attacker can get text into something that gets loaded at assemble time, persistent memory or a retrieval store, the payload is already in the context window before the current session begins. The agent does not have to be tricked into reading it during the run. It is pre loaded, with the standing of established context, as if it had always been there.
Two properties fall out, both nasty for defenders.
Time separation. The write and the trigger can be hours, days, or weeks apart. Incident response that looks at the session where damage occurred finds nothing, because the malicious input arrived in a different session entirely.
Victim separation. The person who planted it and the person who triggers it can be different people. An attacker poisons a shared store, and a completely innocent colleague’s session fires the payload. The victim did nothing wrong and sees no cause.
Memory poisoning
Persistent memory is the feature where an agent remembers things across sessions: your preferences, past context, facts it learned. Useful, and a perfect dead drop.
The attack: get malicious content written into memory in one session, so it loads into context in a later one. The content is phrased as a durable instruction or a false fact the agent will treat as established truth going forward.
Why it is dangerous beyond normal injection: memory is trusted by design. The whole point of memory is that the agent relies on it without re verifying. So a poisoned memory is not questioned the way a suspicious ticket might be. It arrives wearing the badge of the agent’s own established knowledge.
The planting routes are the Day 15 vectors, aimed at whatever the agent chooses to remember. If the agent summarizes a conversation into memory, an attacker who controls part of that conversation influences what gets stored. If it remembers facts from documents, a poisoned document seeds memory. The write can even be indirect: the attacker never touches memory directly, they just ensure the agent decides to remember their text.
RAG and knowledge base poisoning
RAG, retrieval augmented generation, is where the agent pulls relevant documents from a store, a vector database or a document set, and loads them into context as reference. It is how agents answer from company knowledge.
The attack: plant content in the retrieval store so it gets pulled into context when a relevant query is made. Now the poison arrives dressed as the organization’s own trusted reference material, which is a stronger disguise than any ticket. The agent and the user both treat retrieved knowledge as authoritative.
Two sub cases worth separating.
Direct write. The attacker can write to the store, a wiki anyone can edit, a shared drive that gets indexed, a docs folder with loose permissions. They add a document crafted to be retrieved for target queries, carrying the payload.
Ingestion poisoning. The attacker cannot write to the store directly, but they can influence a source the store ingests. If the pipeline scrapes public pages, support tickets, or external feeds into the vector DB, the attacker poisons the source and the pipeline carries it in. This is the supply chain shape from week two applied to knowledge.
The retrieval twist: the attacker wants their document to actually get retrieved, so they craft it to rank for the queries they are targeting, stuffing it with the terms a victim query would match. It is SEO for the vector database, aimed at the agent instead of a search engine.
Why these defeat the usual thinking
Three reasons teams miss these specifically.
Session scoped review misses them. Most monitoring looks at a session as a unit. These attacks span sessions, so nothing in the damaging session looks wrong. The malicious write happened elsewhere, maybe to a different user.
The source is trusted by design. Memory and RAG exist precisely so the agent does not re verify them. Defenses that treat tool output as suspicious often treat memory and retrieval as clean, because they are internal. That trust is the vulnerability.
The trigger looks innocent. A victim asks a normal question. Retrieval pulls the poisoned doc because it matches. The victim sees a normal answer shaped by a payload they never saw. There is no suspicious input in their session at all.
Reproduce both in your lab
Safe local versions, using your existing lab.
Memory poisoning:
- Add a simple per user or shared memory file the agent loads at the start of each run and treats as established context.
- In session one, through normal interaction, get a malicious instruction or false fact written into that memory file, phrased as a durable note.
- Start a fresh session, a new run, with no attacker input at all. Just a benign task.
- Observe whether the poisoned memory, loaded at assemble, shapes behavior in the clean session. Time and session separated, same as real.
RAG poisoning:
- Give the agent a retrieval tool over a small local document folder, loaded into context on matching queries.
- Plant a document crafted to be retrieved for a target query, carrying a payload dressed as reference material.
- In a separate session, ask the target query as an innocent user would.
- Observe the poisoned doc get retrieved and shape the answer, with no attacker present in that session.
Both exercises make the time and victim separation concrete. The session where damage happens contains no attacker input, which is exactly why these are hard to catch.
Detection and defense, the honest version
Ordered by effectiveness. Same thesis, adapted to the persistent stores.
Treat memory and retrieval as untrusted input, not trusted context. The core mindset fix. The content came from somewhere, and somewhere includes attacker influenceable sources. Memory and RAG content should be handled with the same suspicion as a web page, not the reverence of the system prompt. This single reframing undoes the trusted by design weakness.
Constrain consequence, still. The week four thesis holds even here. A poisoned memory or document that fires still has to reach something valuable and exit somewhere. Scope and allowlist from Day 11 cap what a detonation can do, no matter when it was planted. Time separation does not help the attacker if the payload reaches nothing.
Control writes to the stores. For memory, be deliberate about what the agent is allowed to persist, and do not auto remember raw untrusted content as durable fact. For RAG, lock down who can write to the store and audit the ingestion pipeline, since ingestion poisoning rides in through loose sources. Fewer write paths, fewer dead drops.
Attribute and provenance tag stored content. Where possible, track where a memory or document came from, so a later retrieval carries its origin. Content from an untrusted source stays marked untrusted even after it is stored, which lets you apply the right suspicion at retrieval time and makes incident tracing possible across sessions.
Monitor the stores, not just sessions. Because the attack spans sessions, watch the stores themselves. Diff memory for unexpected durable instructions, monitor the RAG corpus for added or changed documents, the Day 9 pinning idea pointed at knowledge. A new document that is crafted to rank for sensitive queries is a signal.
Expire and review memory. Durable does not mean forever. Aging out old memory and periodically reviewing what the agent has stored limits how long a dead drop stays live.
The through line, adapted: you cannot keep every poisoned fact out of memory or retrieval, because the agent exists to ingest and trust them. So you treat those stores as untrusted, constrain what a fired payload can reach, and monitor the stores across sessions, not just the session in front of you.
Homework for Day 20
- Build the memory poisoning exercise. Plant in session one, trigger in a clean session two, and confirm the poisoned memory shapes behavior with no attacker input in session two.
- Build the RAG poisoning exercise. Plant a document crafted to be retrieved, then trigger it from an innocent query in a separate session.
- For both, note what incident response would see if it only looked at the damaging session. Write down that the cause is invisible there.
- Apply the top defenses: treat retrieved and remembered content as untrusted, and apply Day 11 scoping so the fired payload reaches nothing valuable. Confirm the detonation now does no harm.
- Audit a real agent you use with memory or RAG: what can it persist, who can write to its knowledge store, and does anything mark stored content by origin. Note the biggest dead drop.
Point 3 is the one that lands with incident responders. Show them a compromised session with no malicious input in it, then reveal the write happened a week earlier in someone else’s session. That is why these are feared.
Day 21 closes week three by hunting the other half of every chain: exfiltration channels, the outbound paths an agent hands you for free, including the ones that masquerade as harmless output.
