The Classifier-Router-Specialist Pattern

The Problem With Trying to Do Everything

Every production AI product eventually runs into the same wall. The team has built a system prompt that handles questions, tasks, emotional support, research requests, and casual conversation. The prompt is 1,800 tokens. It works — mostly. But tuning it feels like surgery. Change the tone for emotional support and the question-answering becomes warmer but less precise. Tighten the task-execution instructions and the casual conversation starts sounding robotic.

The product is fighting with itself. One prompt is trying to be five different products simultaneously.

This is not a prompting problem. It is an architecture problem. The solution is not a better monolithic prompt. The solution is to stop routing all inputs through the same prompt.

The Three-Part Pattern

The classifier-router-specialist pattern is conceptually simple: before sending user input to the main LLM, determine what kind of input it is, then send it to the correct specialist configuration rather than a generalist one.

The classifier is a lightweight LLM call that reads the user's input and returns a category label. Nothing more. Its output is a single word: question, task, feedback, vent, research. The classifier prompt is under 100 tokens. It uses the cheapest model available. It runs in under 50ms and costs fractions of a cent.

The router is not an LLM at all. It is a switch statement. The classifier said task — look up the task specialist configuration. Retrieve the system prompt for task mode. Retrieve the model, temperature, and max token settings for task mode. Pass everything to the specialist call.

The specialist is a focused LLM call with a system prompt tuned for one mode only. The task specialist's system prompt is 400 tokens of carefully crafted task-execution instructions. It does not carry a word about emotional support or research behavior. It is good at exactly one thing.

Why Specialists Beat Generalists at Their Own Tasks

This result surprises people who have not run the comparison. Surely a more powerful model with more instructions is better than a narrower model with fewer? In practice, the opposite is true for specific task types.

The mechanism is instruction competition. A generalist prompt must hedge across modes: "Be precise when answering questions, but warm when someone is expressing frustration." These instructions conflict. The model weights them against each other on every response, producing a compromise. The specialist has no such conflict. Its entire context is dedicated to one behavioral profile. It is not splitting attention between being an empathetic listener and being a precise answer engine.

The practical upside is not just quality. Specialists are independently tunable. When the feedback mode needs adjustment because users are responding negatively to how the system acknowledges their input, you edit the feedback specialist's system prompt. The question-handling mode is completely unaffected. This separation is worth as much as the quality improvement, because it means iteration is safe.

The Cost Arithmetic

The economics of this pattern favor it over a generalist at almost every scale.

Clarity — Knox's AI decision-making product — uses a seven-mode classifier with specialist prompts. The "everything" system prompt before the refactor was 2,100 tokens. Each specialist prompt averages 480 tokens. At Anthropic's pricing, that is a 77% reduction in input token cost per specialist call. The classifier costs very little per call. Net result: each routed call costs less than the original generalist call while producing higher-quality output.

The numbers are not exotic: a 500-token specialist prompt costs significantly less than a 2,000-token generalist prompt. The classifier adds minimal overhead. For every 1,000 requests, the specialist pattern delivers substantial savings in input token costs alone — before accounting for the output token savings from shorter, more focused responses.

At 10,000 requests per day, that adds up fast, from one architectural change.

Building the Classifier Correctly

The classifier prompt should be as short as possible. It has one job. Here is the full prompt Knox uses in production for a five-mode classifier:

Classify this user input as exactly one of: question, task, vent, feedback, research.
Respond with only the single word, lowercase. No punctuation. No explanation.

That is 32 tokens. It never needs to be longer. The classifier does not need to understand nuance — it needs to reliably assign one of N labels. Short, unambiguous prompts do this better than elaborate ones.

Error handling matters here. The classifier might return "question." (with punctuation) or "Question" (capitalized) or an entirely unexpected word if the model has a bad day. Your routing logic must trim, lowercase, and validate the output. If the label is not in your valid set, default to a reasonable fallback mode rather than crashing. In most products, question is a safe default.

The router is just logic. Do not overthink it. It is a lookup table from mode labels to specialist configurations:

const CONFIGS = {
  question:  { systemPrompt: QUESTION_PROMPT,  model: 'claude-haiku-4-5',   temperature: 0.3, maxTokens: 500 },
  task:      { systemPrompt: TASK_PROMPT,       model: 'claude-sonnet-4-6',  temperature: 0.1, maxTokens: 2000 },
  feedback:  { systemPrompt: FEEDBACK_PROMPT,   model: 'claude-haiku-4-5',   temperature: 0.7, maxTokens: 800 },
}

Note that the router is also where model routing happens. The task specialist uses claude-sonnet-4-6 because task execution benefits from the stronger model. Question and feedback modes use claude-haiku-4-5 because the quality difference does not justify the cost. The classifier-router pattern gives you per-mode model routing for free.

When to Add the Synthesis Layer

The synthesis layer exists for one situation: you routed the same input to multiple specialists in parallel, and now you need to combine their outputs into a coherent response.

This is rarer than it sounds. Most user inputs belong to one mode. The synthesis layer is not needed for single-specialist routing — add it only when you have a genuine use case for parallel specialist execution and output merging.

For the products Knox runs, the synthesis layer appears in the research pipeline: a search specialist, a database specialist, and a calculation specialist run simultaneously, and a fourth agent synthesizes their outputs into a structured response. The extra latency of the synthesis call is offset by the parallel execution of the three specialists.

For most AI SaaS products, the pattern is: classify, route, return. Clean, cheap, independently tunable.

The decision-making platform indecision.io uses the classifier-router pattern in its core inference loop. Architecture documentation and production case studies are at jeremyknox.ai.