Prompt Caching — Cutting Your AI Bill by 50%+

The 90% Discount You Are Not Taking

Anthropic offers a 90% discount on cached input tokens. On a cache hit, you pay roughly 10% of the normal input token price for the cached portion. For exact current rates, see anthropic.com/pricing.

That is a 90% discount on every token in your system prompt for every request where the cache is warm.

Clarity — the AI decision-making product Knox runs — has a 2,608-token system prompt. Before implementing prompt caching, every request processed all 2,608 tokens at full price. After implementing caching with an 83% hit rate, 83% of requests process those 2,608 tokens at 10% of normal cost. Total cost reduction: 53%. The implementation took two hours. The savings are permanent.

This is not a micro-optimization. At scale, prompt caching is the highest-leverage cost reduction available in production AI products.

How Prompt Caching Works

When you make a request to the Anthropic API, you can mark specific content blocks with cache_control: { type: 'ephemeral' }. Anthropic processes that block, computes a cache key from its exact content, and stores the processed result on their servers.

On the next request with the same content in that block, Anthropic serves the cached version. You pay 10% of normal input token price for the cached portion. The model still runs inference on your user message — only the prefix processing is cached.

The cache lives for approximately 5 minutes of inactivity. Frequently accessed prompts stay warm indefinitely. Low-traffic products may experience more cache misses simply because the cache expires between requests.

One constraint silently kills caching for small prompts: there is a minimum cacheable prefix — roughly 1,024 to 4,096 tokens depending on the model. A prefix below that threshold is silently not cached: no error, no warning, just cache_creation_input_tokens: 0 and cache_read_input_tokens: 0 on every response. If your system prompt is a few hundred tokens, the cache_control marker does nothing. This is rarely a problem for production products — real system prompts with instructions, rules, and examples easily clear the minimum — but it bites toy examples and early prototypes.

The implementation in the Anthropic SDK looks like this:

const response = await client.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  system: [
    {
      type: 'text',
      text: STATIC_SYSTEM_PROMPT,
      cache_control: { type: 'ephemeral' },
    },
  ],
  messages: [{ role: 'user', content: userMessage }],
})

The response includes usage.cache_creation_input_tokens (tokens processed and cached on this request) and usage.cache_read_input_tokens (tokens served from cache). Track these to measure your hit rate.

The Two-Block System

Effective prompt caching requires separating your context into two distinct blocks:

Block 1: The static system prompt. This is everything that is identical across all requests — your product's instructions, persona, rules, behavioral constraints, and guidelines. This block must be byte-for-byte identical on every request. No timestamps. No per-user context. No dynamic values of any kind. This block gets cached.

Block 2: The dynamic user content. This is the user's message, plus any dynamic context that varies per user or per request — the user's name, their current tier, their location, the current time if the AI needs it. This block is never cached. It changes on every request.

The key discipline is strict separation. Everything that must be the same goes in Block 1. Everything that can change goes in Block 2.

The Cache-Busting Mistakes That Kill Your Hit Rate

A 0% cache hit rate means something is making every request unique. These are the four most common causes:

Timestamp injection. Adding "Current time: " + new Date().toISOString() to your system prompt makes it unique on every request. The prefix never matches. Cache hit rate: 0%. The fix is simple: remove timestamps from the system prompt entirely. If your AI needs time context, pass it in the user message: "Context: the current time is 2:00 PM EST. User question: ..."

Per-user preferences in the prefix. Injecting user preferences like "User's preferred language is Spanish" into the system prompt means every user has a unique system prompt. Cache hit rate: proportional to how many times the same user makes a request. For most products, this is low. The fix: inject user preferences into the user message or a dynamic context block after the cached prefix.

Dynamic prompt construction. Building system prompts from database values at request time — "Available products: " + await getProductList() — produces variable content depending on the database state. If product availability changes frequently, the prefix changes frequently. Pre-compile your system prompt to a static string at server startup and reload it only when you intentionally update it.

Multiple prompt variants. Running A/B tests with multiple system prompt variants splits your traffic. Each variant has its own cache. Instead of one hot cache with 5,000 requests/day, you have two lukewarm caches with 2,500 requests/day each. This roughly halves cache efficiency. Consolidate variants into a single system prompt with conditional logic where possible, or accept that A/B testing temporarily reduces cache efficiency.

The Real Numbers From Clarity

Clarity's implementation demonstrates what production prompt caching looks like at meaningful traffic. The examples below use $0.003/1K input tokens as an illustrative rate — see anthropic.com/pricing for current figures.

System prompt: 2,608 tokens. User message average: 82 tokens. Daily requests at production volume: approximately 1,000. Cache hit rate: 83%.

Without caching: (2,608 + 82) * 1,000 * $0.003/1K = $8.07/day.

With caching: 170 requests at full price = (2,608 + 82) * 170 * $0.003/1K = $1.37. 830 requests with a cache hit on the system prompt = (2,608 * 0.10 + 82) * 830 * $0.003/1K = $0.65 for the cached system prompt + $0.20 for the user messages = $0.85. Total: $1.37 + $0.85 = approximately $2.22/day.

That is a 72.5% daily cost reduction. One caveat the worked example glosses over: cache writes are not free. Anthropic bills cache_creation_input_tokens at roughly 1.25x the base input rate — a 25% surcharge on the request that first warms the cache — so miss-heavy days cost slightly more than the 1.0x math above suggests. The break-even is roughly one cache hit per write, which the 5-minute TTL and steady traffic comfortably clear at this volume. The 53% figure Knox cites is the all-in number: it also accounts for output tokens (which are never cached) and variation in request size. At larger traffic volumes — 5,000 requests/day — the same hit rate produces proportionally larger savings because the fixed cost of cache warming amortizes across more hits.

Measuring and Monitoring Cache Performance

Prompt caching is invisible if you do not instrument it. Add tracking to every AI call:

const response = await client.messages.create({ ... })

const cacheHit = (response.usage.cache_read_input_tokens ?? 0) > 0
const cacheCreation = response.usage.cache_creation_input_tokens ?? 0
const cacheRead = response.usage.cache_read_input_tokens ?? 0

// Log for your metrics system
logger.info('ai_request', {
  cacheHit,
  cacheCreation,
  cacheRead,
  inputTokens: response.usage.input_tokens,
})

Track the daily hit rate as cache_read / (cache_read + cache_creation). If your hit rate drops below 70%, investigate for cache-busting patterns. Common triggers: a code deploy that changed the system prompt, a configuration change that injected a dynamic value, or traffic patterns shifting to a time-of-day where the cache expires between sessions.

At 70-85% hit rate, you are capturing most of the available savings. Above 85% — Clarity territory — you have a highly optimized prompt architecture with consistent traffic patterns. Below 40%, you likely have a structural issue: the system prompt is not stable, or the traffic volume is too low to keep the cache warm.

The trading systems Knox runs on Tesseract approach prompt caching differently — the trading decision prompts are small (under 300 tokens) and vary more with market context, so caching provides less benefit. The larger, more stable system prompts in Semantic Memory Layer and Mission Control are the primary targets for cache optimization. Architecture matters as much as implementation.

For the complete production architecture of a live AI ecosystem and how prompt caching fits into cost management across the full stack, jeremyknox.ai covers the implementation details. For AI-assisted decision-making under real cost constraints, indecision.io applies these principles to the product layer.