The Attacker's Playbook: How Agent Systems Get Owned
Every lesson in this track is one layer of defense — this lesson shows what happens when a layer is missing, by walking the full kill chain an attacker uses against an agent stack from first reconnaissance to persistent memory poisoning.
How It Actually Happens
Security discussions often describe attacks in the abstract — "an attacker gains access and exfiltrates data." This lesson is more specific. It walks the kill chain an attacker uses against an agent stack, stage by stage, mapping each step to the agent-specific surface it exploits and the defense layer that stops it.
The goal is not to provide a blueprint for attacks. The goal is to make the defensive investment concrete: every lesson in this track is one layer of defense against one or more stages of this chain. When you see the chain in full, you understand why each layer matters and why a single missing layer can make the rest irrelevant.
Stage 1: Reconnaissance
The attacker starts by building a map. They are not yet inside your system — they are learning everything they can from outside it.
Agent stacks leak topology through multiple channels, all covered in the infrastructure exposure lesson: published documentation that contains real internal service URLs, GitHub repositories with path names or hostname references in code comments, error pages that echo stack traces with internal addresses, READMEs that list agent names and their roles, content pipeline output that reproduces whatever context the agent saw.
A reconnaissance pass against a typical development team's public artifacts will often yield: the agent platform being used, the names and roles of deployed agents, the internal service naming pattern, the directory structure on the development machine, and sometimes the approximate credential type in use (from error messages or code examples).
None of this requires breaking anything. It is all public. The sanitization gate — a CI check that scans outbound content for internal patterns before publication — is the defense that prevents reconnaissance from yielding a useful map.
What reconnaissance enables: a targeted initial access attempt aimed at the most likely entry point, with prior knowledge of what tools are available once inside.
Stage 2: Initial Access
With a map, the attacker pursues initial access. For agent systems, there are two primary vectors:
Injection via ingested content. The attacker publishes content they know the agent will read — a webpage in a topic the agent monitors, an email in a pattern the agent processes, a document in a repository the agent audits. The content contains an injection payload that, when read by the agent, attempts to override its instructions. This is the indirect injection pattern from the prompt injection lessons.
Leaked credential. The secrets hygiene lesson documents how keys end up in git history, agent transcripts, PR diffs, and error messages. A leaked key does not require any sophistication to exploit: the attacker authenticates as the agent and exercises whatever that credential permits. A console API key found in a transcript grants direct API access. A service token found in a commit history may grant access to internal services.
These two paths have different characteristics. Injection via content requires the attacker to be patient — they plant the payload and wait for the agent to read it. A leaked credential provides immediate, direct access. Both lead to the same next stage.
Stage 3: Escalation
This is the stage that is unique to agent systems and the most important to understand.
In a traditional system, escalation means exploiting a privilege to gain a higher privilege — a local user account used to access kernel space, a database query used to execute shell commands. It requires exploiting a technical vulnerability.
In an agent system, the agent's own tool privileges do the lateral movement. An injected agent with filesystem write access can write to paths it should not touch. An over-privileged agent can read the secrets directory that was not in its intended scope. An agent with outbound network access can make requests to attacker-controlled infrastructure. The tools you granted are the attack tools.
This is why the least privilege lesson is not philosophical. An agent with narrow tool grants has a narrow escalation surface. An agent with broad tool grants provides the attacker with broad capabilities as soon as initial access is achieved — no additional exploitation required.
The defense at this stage is the tool policy matrix: explicit allowlists that limit what any agent can do, path-level denylists for secrets and infrastructure, separate credentials that isolate blast radius.
Stage 4: Exfiltration
Exfiltration requires two things the agent already has: data worth stealing and a channel to send it through. When escalation succeeds, the attacker has read access to whatever the agent can reach. The exfiltration channel is whatever outbound capability the agent has.
The indirect injection lesson introduced the lethal trifecta: an agent with (a) access to private data, (b) exposure to untrusted content, and (c) an exfiltration channel is a breach waiting to happen. Exfiltration is the stage where the trifecta completes.
The channels are not exotic. A git commit with a carefully crafted message. An outbound fetch to a URL the agent is allowed to reach. An email send in a platform that includes email tools. A write to the knowledge layer that another agent (or a user) will read. Each of these is a legitimate capability the agent was granted; the injection payload redirects it toward an attacker-controlled endpoint. The qualifying test for any write channel is attacker reachability: a knowledge-layer write completes exfiltration only when its destination eventually propagates to something the attacker can read — published content, an externally visible surface, a downstream agent that publishes. A write that stays inside a store no outsider can ever reach is a persistence vector, not an exfiltration channel.
Audit logs on tool calls, combined with anomaly detection for new tool combinations, are the detection-layer defense at this stage. An agent that has never made a POST to an external URL suddenly making one is a high-confidence exfiltration signal — if you are collecting the logs.
Stage 5: Persistence via Memory Poisoning
This is the stage that makes agent security categorically different from web application security, and the one that is most commonly underestimated.
After a successful exfiltration, a sophisticated attacker does not just take the data and leave. They use their temporary control of the agent to write a re-injection payload into the agent's memory store. The payload is designed to look like legitimate stored knowledge — a summary of a process, a note about a configuration, a remembered preference — but it contains embedded instructions that re-execute on every recall of a matching query.
Three weeks later, long after the original breach has been forgotten, a new session loads relevant memories and the injected payload re-fires. The attacker no longer needs to maintain access. The memory store carries their payload forward indefinitely.
This is what makes memory an injection surface with a longer lifetime than a prompt. A prompt-level injection operates for the duration of one session. A memory-level injection operates until the memory entry is audited and removed — which, in a system without memory auditing, is never.
The defenses are: treat memory writes as untrusted writes (content provenance tracking), periodically scan memory entries for instruction-shaped content, and include memory poisoning in the scope of any security incident review.
Detection: The Layer Beneath All Others
Detection is not a defense in the traditional sense — it does not prevent any stage from occurring. It catches the chain in progress, before it completes or before a completed breach can be leveraged further.
The signal to watch for in agent audit logs:
New tool combinations. An agent that has called web_fetch for 30 days has never called memory_write followed by an outbound POST. If that combination appears, it is anomalous. The baseline of normal behavior makes abnormal behavior visible.
Spend spikes. A compromised agent being used to drive cost-DoS against your AI endpoints will appear as an unexpected spike in API usage and associated cost. Billing is a security surface — unusual spend patterns in off-hours windows are a reliable indicator.
Off-hours activity. Most legitimate agent jobs have predictable schedules or predictable triggers. Autonomous activity at 3am in a system that normally runs during business hours is worth investigating.
Unusual memory write patterns. Memory entries that appear to contain instructions rather than knowledge — identified by scanning for instruction-shaped language — are a persistence signal.
None of these signals requires sophisticated ML or a dedicated security team. They require collecting structured logs from tool calls and running basic comparisons against historical baselines. The infrastructure observability lessons cover the logging patterns. The security application is: audit logs on tool calls, surfaced to whoever is responsible for the fleet.
The Chain Is a System
The kill chain is not a menu of options — it is a sequence where each stage enables the next. Reconnaissance enables targeted initial access. Initial access provides the foothold escalation requires. Escalation provides the data and channels exfiltration needs. Persistence makes the compromise self-sustaining.
Defense in depth works because an attacker who is stopped at any stage cannot proceed to the next. They can attempt multiple initial access paths, but if all of them require either undetected content injection or a leaked credential and neither is available, the chain does not start. If the chain starts but escalation is blocked by narrow tool grants, exfiltration has no data to send. If exfiltration is blocked by removing the outbound channel leg of the trifecta, persistence has no opportunity to write to memory.
Every lesson in this track — secrets hygiene, topology sanitization, PII gates, injection defense, least privilege, bot defense — is one defense layer against one or more stages of this chain. That is not coincidence. The track was designed around this chain. The capstone lesson ties all of them together into a repeatable audit of your own stack.
Understanding the chain is what makes the defense investment rational. Security practitioners who understand attack mechanics invest in the right layers. Security practitioners who do not end up building walls in front of open doors.