Production Hardening: Errors, Retries & Cost Control

The Gap Between Demo and Production

A Claude integration that works in a demo is easy to build. You call the API, get a response, display it. The model is impressive. Everyone is excited.

The gap between that demo and a system that runs reliably for months — processing millions of tokens, handling API pressure, staying inside budget, recovering gracefully when things go wrong — is almost entirely in the infrastructure around the call, not in the call itself.

This lesson covers that infrastructure: typed error handling, retry discipline, cost observability, and model routing. These are the pieces that make the difference between a demo and a production system.

The Error Taxonomy

Every error from the Claude API has a status code and a type. The most important axis is retryability: some errors are permanent (fix your code), others are transient (try again with backoff). Treating a permanent error as transient wastes requests and delays the fix. Treating a transient error as permanent drops work that would have succeeded on retry.

Three errors deserve specific attention because they are commonly mishandled:

404 is a config bug. In most APIs, a 404 means a resource does not exist. In the Claude API, a 404 almost always means the model ID is wrong — typically a guessed date suffix that does not exist. claude-sonnet-4-6 is correct. claude-sonnet-4.6 (dot notation) 404s. claude-sonnet-4-6-20250101 or any date-suffixed variant 404s. A 404 is not an outage. It is a hard stop: fix the model ID before doing anything else.

529 is not 500. Both are retryable, but they mean different things. api_error (500) is a server-side error — retry with exponential backoff. overloaded_error (529) means the platform is under capacity pressure. Retry intervals need to be longer, and for latency-critical paths you should consider falling back to a different model class (Haiku, which may have capacity when Sonnet is pressured). Backoff strategies that work for transient errors often fail at overload scale.

400 is a development bug. Invalid request errors should not appear in production if you have tested your code. The most common causes: illegal constraints in a structured output schema (the SDK stripped them during development, but a raw schema sent from another client path hit the validator), sampling parameters on Opus 4.8/4.7, or a prefill in the message array. A 400 in production means your deploy pipeline did not include a live smoke call.

Typed Exceptions: Most-Specific-First

The SDK provides typed exception classes for each error category. Use them — never string-match on error messages. Error message text is undocumented and can change. Exception class names are a stable API contract.

The load-bearing rule is catch order. RateLimitError, OverloadedError, and BadRequestError all extend APIError. If you catch APIError first, the subclass handlers are unreachable. Catch most-specific first:

import Anthropic from "@anthropic-ai/sdk"

const client = new Anthropic()

async function callWithTypedErrors(prompt: string): Promise<string> {
  try {
    const response = await client.messages.create({
      model: "claude-sonnet-4-6",
      max_tokens: 1024,
      messages: [{ role: "user", content: prompt }],
    })
    const block = response.content[0]
    if (block.type !== "text") throw new Error("Unexpected non-text block: " + block.type)
    return block.text

  } catch (e) {
    if (e instanceof Anthropic.RateLimitError) {
      // Honor retry-after. SDK auto-retries (default maxRetries: 2), but custom logic
      // may need to handle queue-and-resume for sustained throughput.
      throw new Error("Rate limited — check x-ratelimit-remaining headers: " + e.message)
    }
    if (e instanceof Anthropic.OverloadedError) {
      // Longer backoff than 5xx. Consider Haiku fallback on latency-critical paths.
      throw new Error("API overloaded — backoff and retry: " + e.message)
    }
    if (e instanceof Anthropic.BadRequestError) {
      // Do not retry. This is a code defect — fix the request.
      throw new Error("Bad request (fix your params): " + e.message)
    }
    if (e instanceof Anthropic.APIError) {
      // General API error — retry with backoff
      throw new Error("API error " + e.status + ": " + e.message)
    }
    throw e // Not an Anthropic error
  }
}

The SDK applies exponential backoff automatically on 429 and 5xx errors with maxRetries defaulting to 2. You can configure this per-client: new Anthropic({ maxRetries: 0 }) disables automatic retries if you want full control. For high-throughput systems, honoring the retry-after header explicitly is worth implementing — it tells you exactly how long to wait, rather than guessing with backoff.

Cost Observability: The Usage Ledger

Every response from client.messages.create() and stream.finalMessage() includes a usage object with token counts. This is operational data. Log it.

interface UsageRecord {
  model: string
  inputTokens: number
  outputTokens: number
  cacheReadTokens: number
  timestamp: number
}

function recordUsage(model: string, usage: Anthropic.Messages.Usage): void {
  const record: UsageRecord = {
    model,
    inputTokens: usage.input_tokens,
    outputTokens: usage.output_tokens,
    cacheReadTokens: (usage as Record<string, number>)["cache_read_input_tokens"] ?? 0,
    timestamp: Date.now(),
  }
  // In production: emit to your metrics system (DataDog, CloudWatch, custom)
  console.log("[usage]", record)
}

The fields that matter for cost tracking: input_tokens and output_tokens at base rates, plus cache_read_input_tokens at 0.1× and cache_creation_input_tokens at 1.25× (5-minute TTL) or 2× (1-hour TTL). Full cost requires all four fields.

For token pre-flight sizing, use POST /v1/messages/count_tokens. Never use tiktoken — Claude uses a different tokenizer and tiktoken undercounts by 15-20% or more, which means your context-window checks and cost estimates are systematically wrong.

const tokenCount = await client.messages.countTokens({
  model: "claude-sonnet-4-6",
  messages: [{ role: "user", content: longDocument }],
})
console.log("Pre-flight tokens:", tokenCount.input_tokens)

Model Routing: Architecture, Not Afterthought

The three-model family is not a ladder where you use Opus for everything and pay the premium. It is a routing architecture where you match model capability to task complexity.

The routing logic is simpler than it looks: ask what the task actually requires.

Haiku is the right model for high-volume tasks that do not need deep reasoning — classification, extraction, simple routing decisions. At $1.00/1M input tokens and $5.00/1M output tokens, you can run Haiku continuously at a cost that is nearly invisible. The constraint is capability, not cost.

Sonnet is the production default for most generation and tool-use workloads. It handles balanced reasoning, structured output, and multi-turn conversations competently at reasonable cost. Most production Claude integrations should default to Sonnet and upgrade to Opus only when Sonnet demonstrably falls short.

Opus is for tasks that require deep reasoning, complex agentic work, or where the quality gap between Sonnet and Opus is measurable and matters. At $5.00/1M input and $25.00/1M output, Opus is roughly 1.7 times more expensive than Sonnet on input — and five times more expensive than Haiku. Applying it to classification or simple extraction is a five-times overpay versus Haiku for no gain.

Routing by task type in code:

type TaskType = "classification" | "generation" | "reasoning" | "agentic"

function routeModel(task: TaskType): string {
  switch (task) {
    case "classification": return "claude-haiku-4-5"
    case "generation":     return "claude-sonnet-4-6"
    case "reasoning":      return "claude-sonnet-4-6"
    case "agentic":        return "claude-opus-4-8"
  }
}

Note the absence of hardcoded beliefs about model capabilities. The Models API (client.models.retrieve(modelId)) provides live capability flags including context window size and supported features. In long-running systems, querying the Models API at startup for the current spec is more reliable than hardcoding values that may change.

The usage ledger and model routing are not independent concerns. Tracking per-model token usage is how you validate routing decisions. If your "classification" tasks are producing 50K+ output tokens on Haiku, something is wrong — either the routing is off, or the task classification is wrong. Usage telemetry surfaces these mismatches before they appear as a surprise line item in the monthly bill.

Rate Limits and Throughput Architecture

Rate limits in the Claude API operate on two dimensions simultaneously: requests per minute (RPM) and tokens per minute (TPM, broken into input and output separately) per model class. The response headers x-ratelimit-remaining-requests, x-ratelimit-remaining-input-tokens, and x-ratelimit-remaining-output-tokens tell you where you stand in real time. The retry-after header tells you exactly how long to wait when you hit a 429.

The SDK's default behavior (maxRetries: 2) handles occasional 429s automatically with exponential backoff. For sustained high-throughput workloads — an eval harness processing thousands of requests, a bulk enrichment job — the defaults are not enough. You need a request queue with concurrency controls, explicit rate tracking, and retry logic that can sustain throughput over minutes rather than handling occasional single retries.

For the highest-volume use cases, the Batch API is the right escape valve. A batch of 10,000 requests bypasses the synchronous rate limit entirely — the batch API has its own higher-volume limits that are designed for bulk offline work. If you find yourself building a sophisticated concurrent-request queue to stay under RPM limits, that is often a signal the workload belongs in a batch, not in synchronous concurrency.

The Production Readiness Gap

Most Claude integrations reach a working state in an afternoon. They become production systems over weeks, as each of the failure modes in this lesson gets discovered and handled. The error taxonomy, retry logic, usage ledger, and model routing are not extras that get added later — they are the difference between a demo and a system that operates for months without incident.

The hardening checklist for any Claude integration before it handles real traffic: typed exception handlers for every error class, verification that the most-specific-first catch order is correct, a per-call usage log emitting to monitoring, a model routing table that matches task types to model capabilities, and a token count pre-flight check before any request that might approach context limits. These are the pieces that convert API access into production infrastructure.