Debugging Context & Prompt Failures

The Model Gets the Blame

"Claude stopped following the branching rules." "The agent is approving bad PRs now." "The summary quality dropped off last week." In 90% of these cases, the model is innocent.

Context and prompt problems are the most common failure class in AI systems, and the most misdiagnosed. They look like model problems — the behavior changed, the output is wrong, something degraded. But the model is the last thing that changed. The system prompt, the conversation history, the CLAUDE.md, the tools configuration — these are what changed.

The first debugging move for any "the model got worse" report is not to look at the model. It is to read the context as the model sees it.

The Four Pollution Scenarios

The four scenarios cover the most common context pollution patterns. Notice that in every case, the model is behaving correctly given the context it sees — the problem is that the context is wrong.

The code review agent approving bad PRs because prior conversation history included examples of approving similar PRs. The content agent writing off-brand copy because 15 turns of user corrections have accumulated into an in-context persona drift. The tool call format degrading because an earlier malformed response set a bad precedent.

In each case, the model is following in-context patterns — as it is designed to do. The debugging target is the context, not the model.

The Root Cause Decision Tree

The tree has two branches at the top: does a fresh session reproduce it? This single check splits the problem space into prompt bugs (fresh session fails) and context pollution (fresh session succeeds).

For prompt bugs, the second branch: is the instruction present in the prompt? If present but not followed, it is a positioning bug (lost-in-the-middle or low-salience wording). If absent, it was never specified — add it explicitly.

For context pollution, the fix is conversation management: compress the history using /compact in Claude Code, start a fresh session with /clear, or inject a "reset" instruction that overrides the accumulated in-context drift.

Diagnosing the CLAUDE.md

Knox's CLAUDE.md contains the most important behavioral constraints for Claude Code sessions. It is a living document — updated regularly, sometimes by multiple agents in parallel. The risk: a git conflict resolution or an agent edit removes a constraint while adding a feature.

The diagnostic:

# When: Claude Code session stops following a rule you thought was in CLAUDE.md
# Run from the repo root — CLAUDE.md is version-controlled there
git log --oneline -20 CLAUDE.md
git diff HEAD~5 HEAD -- CLAUDE.md

The diff shows what changed. The commit message shows who changed it and why. Nine times out of ten, there is a specific edit that removed or weakened the constraint.

Project instructions live in CLAUDE.md at the repo root (and in nested directories that override it) — distinct from user-level config and memory under ~/.claude/. This distinction matters: an agent that reads the wrong file loads system configuration instead of the project instruction file, and silently operates on the wrong constraints.

Lost-in-the-Middle in Practice

A system prompt has 3,000 tokens. The instruction "always create a feature branch before committing — never commit to main directly" is at token position 1,200 — the middle third.

Transformer attention is highest at the start and end of the context. The middle 30-40% is the low-attention zone. An instruction in that zone will be followed most of the time — and ignored some of the time. The failure appears intermittent and "random," which makes it look like model non-determinism when it is actually a positioning problem.

The fix:

Move critical constraints to the first 200 tokens or the last 100 tokens of the system prompt.
Use <instructions> XML tags to mark constraints as mandatory — this increases salience.
For Claude Code: critical rules go at the top of CLAUDE.md, under a ## HARD RULES section.

Context Compression in Claude Code

Claude Code sessions accumulate context over time. After 30+ turns, the conversation history is long, potentially polluted, and expensive. Three tools:

@file reference: Injects the content of a file directly into the conversation. Use this to re-inject the project instructions when you suspect context drift: @CLAUDE.md

/compact: Compresses the conversation history into a summary, freeing context while preserving the key facts. For pollution that survives compaction, /clear resets the session entirely — the agent loses conversation history but gains a clean context.

# memory shortcut: Quick-adds a fact to memory (e.g., CLAUDE.md) for persistence across sessions; /memory opens the memory files for editing. Unlike conversation history, memory is explicitly authored and does not accumulate noise.

The choice: if the task requires conversation continuity, compress. If the conversation is polluted past recovery, reset. If the agent needs to remember something across sessions, write to memory.

Apply the fix in code: here is an agent driver that accumulates one unbounded context across unrelated tasks. Find the pollution source and make each independent task run against clean context.

Distinguishing Model Non-Determinism From Context Pollution

Both produce inconsistent behavior. The test:

Run the failing prompt 5 times in a fresh session. If it fails consistently: prompt bug. If it fails intermittently: model non-determinism.
Run the failing prompt in the existing session. If it fails consistently: context pollution. If intermittently: could be either.
If fresh session is intermittent: the prompt is ambiguous or the model samples from multiple valid responses. Fix the prompt to narrow the solution space.
If existing session is consistent but fresh session works: context pollution is confirmed.

The Reread Discipline

The most reliable debugging technique for context and prompt problems: read the full context as the model sees it.

Open the conversation. Read it from the top. By turn 20, you have an in-context persona, in-context examples, in-context patterns. Ask yourself: given this context, what would be the natural next response?

If the bad output is the natural response to the context — you have found the pollution. The fix is to remove or override the polluting turns, not to change the model or the base system prompt.

This "reread/pollution lens" is the fastest route to context failure diagnosis. It takes two minutes and surfaces 80% of context pollution bugs without any instrumentation.