Indexing & Recall: Progressive Disclosure

The Token Budget Problem

Every token in the context window serves one of two masters: task context or recalled knowledge. These are not additive — they compete. A context window filled with recalled memories has less room for the tool definitions, conversation history, and task description the agent needs to actually do the work. A context window filled with task context has no memory to draw on.

The naive solution is to load everything: dump all memories into context at session start. This fails catastrophically at scale. An agent that has been running for three months with a healthy write path might have hundreds of live memories. At 300 tokens of average content each, dumping everything consumes tens of thousands of tokens before the first tool call. The model starts ignoring earlier context, attention quality degrades, and task performance drops.

The correct solution is progressive disclosure: stage the recall to match what the agent actually needs, and never fetch more than confirmed-relevant content.

The funnel has three stages, and the discipline is that each stage is triggered only when justified.

Stage 1: Semantic Search Returns Pointers

The entry point for every recall operation is a compact semantic search. The query is embedded with the same text-embedding-3-small model used at write time — vector space consistency is critical for accurate similarity matching. The query runs against the live memories in the relevant namespace, filtered to exclude soft-deleted rows and expired working memories:

SELECT
  memory_id,
  heading,
  LEFT(content, 120) AS snippet,
  memory_type,
  confidence,
  embedding <=> $1::vector AS distance
FROM memories
WHERE namespace = $2
  AND deleted_at IS NULL
  AND (expires_at IS NULL OR expires_at > NOW())
ORDER BY embedding <=> $1::vector ASC
LIMIT 8

The key discipline is LEFT(content, 120). The search returns a snippet, not the full content. Each result costs roughly 30 tokens: the heading, the snippet, the memory_type, and the distance score. At 8 results, that is 240 tokens — cheap enough to run at session start for any namespace.

The agent uses the heading and snippet to confirm relevance. If the heading for a deploy-related memory is "Deploy pipeline: use gh workflow run deploy.yml" and the current task is debugging a failed deploy, that is confirmation. If the heading is "Naming convention: use kebab-case for filenames" and the task is debugging a deploy, that memory can be skipped.

Stage 2: Timeline Context (Optional)

For time-sensitive queries — "what did we change about X recently?" — a second-stage timeline query provides temporal ordering without full content. The input is the memory_ids from Stage 1 that look relevant by heading:

SELECT memory_id, heading, created_at, supersedes
FROM memories
WHERE memory_id = ANY($1)
ORDER BY created_at DESC

This stage adds ~15 tokens per result and surfaces whether any of the Stage 1 results are part of an active supersede chain — useful when you want the most recent version of a fact without accidentally acting on an older one.

Stage 2 is optional. Skip it when recency does not matter for the query.

Stage 3: Full Content Fetch

Only after Stage 1 (or Stage 2) confirms that a specific memory is relevant does the agent fetch its full content:

// Single fetch
const memory = await fetchMemory("mem_d4e5f6")
// → returns the full MemoryRecord with complete content

// Bulk fetch — multiple confirmed-relevant memories
const memories = await fetchMemories(["mem_d4e5f6", "mem_g7h8i9"])

Full content per memory ranges from 200 tokens for a brief procedural rule to 600+ tokens for a detailed episodic retro. This is why Stage 3 is gated on Stage 1 confirmation. Fetching the full content for all 8 Stage 1 results would cost 1,600–4,800 tokens. Fetching only the 1-2 confirmed relevant results costs 200–1,200 tokens — a 4x to 24x reduction.

The economics diagram above makes the tradeoff concrete. The index-line approach (30 tokens per memory) scales to 100+ memories in a reasonable budget. The dump-everything approach (300+ tokens per memory) exhausts the context window before the task even begins. The efficient path is not choosing between knowing everything and knowing nothing — it is using the funnel to know precisely what matters.

Hybrid Recall: Semantic + Type + Recency

Semantic similarity alone is not always the right filter. Two additional dimensions improve precision:

Memory type filtering. For an operational question like "what is the current deploy process?", filtering to memory_type = 'procedural' ensures the result is a standing rule rather than a historical observation. For a debugging task where you want to know what went wrong in similar situations before, filtering to memory_type = 'episodic' gets you the event history.

Recency weighting. For semantic memories that may have been superseded, sorting by created_at DESC within the semantic-similarity results ensures the most recent (and therefore most likely current) version surfaces first. The supersede chain guarantees that old rows have deleted_at set, but within live rows, recency is a useful tie-breaker.

// Hybrid recall: semantic + type filter
const procedural = await searchMemories(
  "how to deploy to production",
  "project-alpha",
  { limit: 5, memoryType: "procedural" }
)

The memoryType option adds a single AND memory_type = $N clause to the cosine query. The cost is negligible; the precision improvement is real.

The Anti-Pattern: Dump-Everything Recall

The most tempting recall mistake is loading all live memories into context "to be safe." The reasoning is intuitive: more context means the agent can draw on more knowledge. The reality is that more context without relevance filtering means more noise competing with the signal.

A 24/7 agent platform that ran for six months had accumulated over 400 live memories by the time the dump-everything pattern was identified. Sessions were consistently underperforming on new tasks — the agent would correctly recall the most recent events but would apply outdated procedural rules from much earlier. Investigation revealed that the context window was over 80% full at session start, leaving minimal room for task context. The fix was implementing progressive disclosure: a compact index at start, full fetches on demand. Session performance recovered within a week.

The principle is not new. It is the same progressive loading pattern used in every performant read-heavy system: load a list first, fetch detail on demand. For memory systems, the currency is tokens rather than bytes — but the discipline is identical.