Conversations, System Prompts & Stop Reasons
Statelessness is a design decision — your code owns history, and every stop reason is a signal your integration must handle.
Statelessness as a Design Decision
The Claude API is stateless. Between one call and the next, the API retains nothing about your conversation. There is no session ID, no server-side history, no automatic context accumulation. Every request is a fresh HTTP call that must carry everything the model needs to give a coherent response.
This sounds like a limitation. It is actually an architectural gift.
Because the API is stateless, YOUR code owns the conversation. You decide what history to include in each turn, how to summarize long conversations, when to inject new context, and how to compact before hitting context limits. A stateful API that managed history for you would make those decisions opaque — you would be a passenger. Owning the messages array means you are the driver.
The pattern is simple: maintain an array, append to it each turn, pass the whole array on every request.
const messages: Anthropic.MessageParam[] = []
// Turn 1
messages.push({ role: "user", content: userInput })
const response1 = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
system: "You are a helpful assistant.",
messages,
})
messages.push({ role: "assistant", content: response1.content })
// Turn 2
messages.push({ role: "user", content: nextInput })
const response2 = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
system: "You are a helpful assistant.",
messages, // includes turn 1
})
The system prompt is passed every turn. The messages array grows every turn. You're re-sending everything each time — which is why prompt caching (the prompt-caching lesson) matters so much. For now, understand the pattern. The cost optimization comes later.
The sequence diagram shows three turns. Notice the history size growing in the label for each turn. By turn 20, you're sending twenty user messages and nineteen assistant responses on every API call. That is the context accumulation reality every production chat integration must plan for.
The System Prompt is the Constitution
The system prompt (system top-level param) is the standing instruction that governs all of Claude's responses in the conversation. It is not a message. It does not belong in the messages array. It is its own param.
await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 2048,
system: `You are the AI Tutor for the academy at jeremyknox.ai.
Your role: explain technical concepts with precision and encourage
active learning. Use TypeScript examples. Never invent API facts —
if you are not certain, say so and suggest the learner verify.`,
messages,
})
Think of the system prompt as the constitution and each user message as a bill. The constitution stays fixed (mostly — you can change it between turns, but changing it invalidates caching). User messages operate within the constitution's constraints.
The system prompt is the right place for: role definition, behavioral constraints, format instructions, domain context that does not change per user, and organizational rules. It is the wrong place for: per-request dynamic data, user-specific context, or anything that changes frequently — put those in messages.
The First Message Rule
One strict constraint from the API: the first message in the messages array must have role: "user". The API does not allow starting with an assistant message. This means you cannot bootstrap a conversation with a fake assistant message to prime a specific context — use the system prompt instead.
This rule simplifies the append pattern: you always push a user message first, get an assistant response back, push that, and alternate. Any time you're about to push an assistant message first, you probably want the system prompt.
Stop Reasons: Your Control Flow Signal
Every production integration switches on stop_reason. Not optionally — exhaustively. The decision tree diagram shows why: each stop reason requires a different response from your code.
Let's walk through each:
end_turn — the happy path. Model finished normally. Read the content blocks.
max_tokens — the response was truncated. Your max_tokens cap was hit before the model finished. Options: raise max_tokens, switch to streaming (required when max_tokens exceeds ~16,000), or redesign the prompt to elicit a shorter response. Do not silently deliver a truncated response to users.
stop_sequence — you configured a stop sequence and the model hit it. Intentional. Consume the content up to the sequence.
tool_use — the model is requesting tool calls. Do not read content as text. Handle each tool_use block, return results, and continue the loop. This is the tool-loop lesson's entire topic.
pause_turn — server-side tool loop paused. Append the paused response's content to your messages array as an assistant turn, then re-send the request to resume — that hands the loop's progress back to the server. Do not discard the turn or re-send the original request without the paused content; that restarts the turn from scratch. This is the API telling you it is not done yet.
refusal — the model declined to respond. A stop_details object carries structured information about the refusal. Surface this to the user — do not retry verbatim. A refusal is not an error in the technical sense; it is the model making a content policy decision.
model_context_window_exceeded — this is the critical one that catches developers off guard. It is NOT the same as max_tokens. max_tokens caps output length. model_context_window_exceeded means the total input (system + history + current message) does not fit in the model's context window. The fix is different: compact the conversation history, summarize earlier turns, or implement a sliding window. Raising max_tokens does nothing — the problem is the input, not the output.
async function chat(messages: Anthropic.MessageParam[], userInput: string) {
messages.push({ role: "user", content: userInput })
const response = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 2048,
system: "You are a helpful assistant.",
messages,
})
switch (response.stop_reason) {
case "end_turn":
const text = response.content
.filter(b => b.type === "text")
.map(b => b.text)
.join("")
messages.push({ role: "assistant", content: response.content })
return text
case "max_tokens":
throw new Error("Response truncated — raise max_tokens or enable streaming")
case "refusal":
throw new Error("Model declined to respond — review the request")
case "model_context_window_exceeded":
// compact history and retry — pop the just-pushed user message first,
// because chat() will push userInput again on the recursive call
messages.pop()
messages = compactHistory(messages)
return chat(messages, userInput)
default:
throw new Error(`Unhandled stop_reason: ${response.stop_reason}`)
}
}
The default case in the switch matters. It catches tool_use, pause_turn, and any future stop reasons the API might add. A missing case is a silent bug — the integration silently delivers nothing, or worse, delivers wrong content.
Sampling Parameters on Current Models
A quick note on sampling parameters: temperature, top_p, and top_k are removed on Opus 4.8 and Opus 4.7. Sending any of them returns a 400 error. On the 4.6 family (Sonnet, Haiku), you can use at most one of temperature or top_p.
The reason for removing them is instructive: Anthropic found that hand-tuning sampling parameters was a cargo cult. Developers set temperature: 0 hoping for determinism and got inconsistency anyway. They set temperature: 0.9 hoping for creativity and got worse results than they could achieve through prompting. The current Opus family simply removed the knobs. If you want focused responses, prompt for them. If you want creative variation, prompt for it.
This does not mean sampling parameters are wrong in principle — they can be useful on models where they exist. It means that on current top-tier models, the steering mechanism is the prompt, not the sampling config.
The Accumulation Problem (Preview)
The conversation model re-sends full history every turn. Ten turns in, you are re-sending ten user messages and ten assistant responses on every API call. Across hundreds of active users, you are paying input token costs for the same tokens repeatedly.
This is the problem that the prompt-caching lesson solves. A cache_control breakpoint on your system prompt means you pay for it once and get cache reads (at 0.1× input price) on every subsequent turn. Understanding why the problem exists — the stateless re-send loop — makes the solution obvious when you get there.
For now, log response.usage on every call. If you see input_tokens growing steadily across a multi-turn conversation with the same user, you are looking at the accumulation problem in real numbers.