Prompt Caching: The 90% Discount

The 100× Resend Problem

In the conversations-and-stop-reasons lesson you learned that Claude's API is stateless by design: your code owns history, and every turn re-sends the full conversation including the system prompt. That is a deliberate architectural decision — it keeps the server simple and makes the API composable.

It is also expensive at scale.

A 24/7 agent platform might have a 15,000-token system prompt defining its behavior, knowledge, and tools. If that agent handles 500 conversations per day with an average of 20 turns each, it re-sends 15,000 tokens roughly 10,000 times per day — paying full base input price every time. Without caching, the system prompt alone can represent 80-90% of your total token cost.

Prompt caching is the mechanism that makes this economical. One breakpoint on the system prompt, and every repeat read costs 10% of the base input price.

The Core Invariant: Prefix Match

Everything about caching follows from one rule.

Caching is a prefix match. Any byte change in the cached prefix invalidates everything after it.

This means you need to understand the render order:

Tools array — rendered first
System prompt — rendered second
Messages (conversation history) — rendered last

A cache breakpoint caches everything from the start of the request up to and including that breakpoint. If you put a breakpoint on the last block of your system prompt, you cache the tools array plus the entire system prompt together. The messages vary per request and are never cached by that breakpoint.

The breakpoint is the cache_control field on a content block:

import Anthropic from "@anthropic-ai/sdk"

const client = new Anthropic()

const response = await client.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  system: [
    {
      type: "text",
      text: LARGE_SYSTEM_PROMPT, // 12,000 tokens
      cache_control: { type: "ephemeral" }, // breakpoint here
    },
  ],
  messages: conversationHistory, // dynamic — not cached
})

// Verify caching is working
console.log("Cache read tokens:", response.usage.cache_read_input_tokens)
console.log("Cache write tokens:", response.usage.cache_creation_input_tokens)
console.log("Non-cached input:", response.usage.input_tokens)

You can place up to 4 breakpoints per request. Use them strategically: one on the system prompt, optionally one on the last assistant turn in a long conversation history. Do not place breakpoints inside dynamic content — any variation in that content wastes the write cost without hitting the cache.

The Economics

The numbers are straightforward but the implications take a moment to absorb.

Cache reads: 0.1× base input price. This is the 90% discount. It applies regardless of TTL — 5-minute and 1-hour reads cost the same.

Cache writes: 1.25× base input price (5-minute TTL) or 2× base input price (1-hour TTL). You pay a premium on the first call that populates the cache.

Break-even: with 5-minute ephemeral caching, the second request in the window covers the write premium and you start saving. With 1-hour caching, break-even is the third request.

The minimum cacheable prefix is a critical detail: 4096 tokens on Claude Opus 4.8 and Haiku 4.5, 2048 tokens on Claude Sonnet 4.6. A system prompt below this threshold silently does not cache — you get no error, no warning, just zero in cache_read_input_tokens. Always verify.

The Silent Invalidator Audit

A cache that appears to be set up correctly but consistently shows zero cache_read_input_tokens has a silent invalidator. Here is the standard audit list:

Timestamps in the system prompt. The most common mistake. Any call to Date.now(), new Date().toISOString(), or even a date string that changes per day embedded in the system prompt invalidates the cache every request. The fix: inject time context as a message, not in the system prompt.

// WRONG — invalidates cache every millisecond
const systemPrompt = `You are a helpful assistant. Current time: ${new Date().toISOString()}.`

// RIGHT — frozen system prompt; dynamic context as a message
const systemPrompt = `You are a helpful assistant.`
const messages = [
  { role: "user", content: `[Context: ${new Date().toISOString()}]\n\nUser question: ${userInput}` }
]

Unsorted JSON serialization. If your system prompt contains JSON-serialized configuration, serialize with deterministic key ordering. JSON.stringify(obj) does not guarantee key order across Node.js versions. Use a sorted serializer or manually construct the string.

Per-user tool sets. If you generate a different tools array per user (adding or removing tools based on permissions), the tools portion of the render order changes per user and the cache is effectively per-user rather than shared. Either use a fixed, maximally-permissive tool set with authorization enforced at execution time, or accept per-user cache isolation.

Model changes mid-conversation. Different models have separate caches. If you route different turns to different models, each model switch is a full cache miss.

Placement Patterns

Three scenarios cover most production use cases.

Large system prompt: place one breakpoint at the end of the system block. The entire system prompt plus tools are cached. This is the default starting point and covers the majority of the cost savings.

Multi-turn conversation with long history: place a second breakpoint on the last assistant turn in the history. This caches the stable conversation history prefix. As conversation grows, move the breakpoint forward. Keep it on the last stable (fully completed) turn, not the one currently being extended.

Shared prefix + varying suffix: if multiple users share the same core system prompt but each has a small personal context block, place the breakpoint after the shared section and inject personal context as messages. The shared prefix (the expensive part) caches across all users.

// Multi-turn pattern with history caching
const messages: Anthropic.Messages.MessageParam[] = [
  ...previousTurns.slice(0, -1), // all but last assistant turn without cache
  {
    ...previousTurns[previousTurns.length - 1],
    content: previousTurns[previousTurns.length - 1].content.map((block, i, arr) =>
      i === arr.length - 1
        ? { ...block, cache_control: { type: "ephemeral" } } // breakpoint on last stable block
        : block
    ),
  },
  { role: "user", content: currentUserMessage }, // volatile — not cached
]

Pre-Warming

For predictable workloads, pre-warm the cache with a max_tokens: 0 request before the first real user request arrives — the API runs prefill, writes the cache at your breakpoint, and returns immediately with empty content and zero output tokens billed. This pays the write cost during off-peak time so the first user request hits the cache.

// Pre-warm during initialization, not during the request path
async function prewarmCache() {
  await client.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 0,
    system: [{ type: "text", text: SYSTEM_PROMPT, cache_control: { type: "ephemeral" } }],
    messages: [{ role: "user", content: "warmup" }],
  })
}

Mission Control's AI-powered status summaries use this exact pattern: the system prompt is pre-warmed at startup, and every real query hits the cache within the first 5 minutes of operation.

What's Next

The next lesson covers adaptive thinking and the effort dial — the controls that let you tune reasoning depth without guessing how many tokens the model needs to think through a problem.