Single Agent vs. Multi-Agent — Choosing the Right Shape

Course 2 Begins Here

Course 1 taught you the business layer: how to validate an AI product idea, understand unit economics, avoid the five failure modes, and measure the metrics that actually matter. You know what to build and whether it will survive contact with real users.

Course 2 is about how to build it correctly. The business layer and the technical layer are not sequential — they run in parallel. But the technical decisions you make in the first week of architecture have a compounding effect on everything that follows: your cost structure, your debugging experience, your ability to iterate when something goes wrong.

The first decision is deceptively simple: how many LLM calls does this feature need?

Single Agent: The Default That Deserves Respect

A single-agent architecture is exactly what it sounds like: one system prompt, one user message, one LLM call, one response. The user sends a request. Your code assembles a context window with a system prompt and the user's message. The model processes it and returns a response. Done.

This pattern is laughed at in AI conference talks and treated as the naive starting point that serious engineers graduate from. That framing is wrong. Single-agent is not a beginner's tool — it is the correct tool for most production AI features.

Here is what single-agent gives you that multi-agent cannot match:

Debuggability. When something goes wrong — and it will — you have exactly one prompt and one response to inspect. There is no ambiguity about where the failure happened. The system prompt produced the wrong output. Fix the system prompt. This sounds trivial until you have spent four hours trying to determine which specialist in a three-agent chain produced the hallucination.

Latency. A well-optimized single-agent call on a modern model returns its first token in roughly 300-800ms, with a full short response landing in about 1-3 seconds. Multi-agent architectures require at minimum two sequential calls — the orchestrator plus the specialist — and often three or four, multiplying that wall-clock time 2-4x. At several seconds of round-trip latency, you are well past the threshold where users perceive a response as slow.

Cost predictability. Single-agent cost scales linearly with traffic. Multi-agent cost scales linearly but with a multiplier of 3-5x. At low traffic that difference is irrelevant. At 10,000 calls per day, it is $6,000 per month.

Prompt coherence. A single system prompt can hold a consistent persona, tone, and set of constraints across the entire response. Multi-agent systems require each specialist to independently maintain coherence — and the synthesizer to harmonize outputs that may have taken different stylistic approaches.

Multi-Agent: When It Earns Its Cost

Multi-agent is not bad. It is expensive, and it is complex. The question is whether what you get is worth what you pay.

There are exactly three situations where multi-agent earns its cost:

Parallel work. If your feature requires three genuinely independent analyses that do not depend on each other, you can run three specialist agents simultaneously. A research tool that queries a database, searches the web, and analyzes a document in parallel will return results faster than doing them sequentially in a single prompt. The wall-clock time of the longest call replaces the sum of all calls.

Specialist quality. Some tasks have a ceiling on single-agent quality. A general-purpose agent asked to write, test, and debug production code will produce mediocre output across all three. A dedicated coding agent with a 2,000-token system prompt optimized specifically for code generation, given the same task, will consistently outperform it. When quality benchmarks consistently show a 15-20%+ improvement from specialization — and that improvement maps to business outcomes — specialists are worth the cost.

Independent verification. A second agent checking the first catches a class of errors that no amount of self-reflection catches. If your product involves consequential outputs — legal text, financial calculations, medical information, security configurations — a verification agent is not a luxury. It is a product requirement. The first agent produces. The second agent audits. Users trust the result.

The Pseudo-Multi-Agent Trap

The most common architecture mistake in production AI is building multi-agent systems for tasks that a single well-crafted prompt handles better and cheaper. This happens for a specific reason: multi-agent feels more sophisticated, and engineers default to what feels like the "right" solution when the product requirements do not give clear constraints.

The tell: you have a multi-agent architecture where the orchestrator passes the user's input to a single specialist, that specialist calls the LLM once, and the result is returned directly. You have added two hops — the orchestrator and the routing logic — and bought nothing. This is a single-agent system wearing multi-agent clothes.

A related failure: chaining agents sequentially when the output of each step is consumed by the next. Agent A produces an outline. Agent B writes the draft based on the outline. Agent C edits the draft. This is three sequential LLM calls. A single agent with a three-phase system prompt — "first outline, then draft based on your outline, then edit your draft" — accomplishes the same task in one call. The sequential chain adds latency and cost, and makes debugging dramatically harder.

The Cost Math You Cannot Skip

Before choosing multi-agent, run this calculation:

Single agent at $0.01/call, 10,000 calls/day = $100/day, $3,000/month.

Three-specialist multi-agent at $0.03/call, same traffic = $300/day, $9,000/month.

Delta: $6,000/month. That is your break-even threshold. The multi-agent architecture needs to generate at least $6,000/month of additional value — through higher retention, fewer support escalations, higher conversion, or reduced churn — to justify the switch.

Often it does not. "The outputs are slightly more polished" is not worth $6,000/month. "The specialist catches errors that cost us a customer refund every three days" might be.

Knox's trading bots demonstrate both sides of this tradeoff. Foresight and Sports Prediction Agent use single-agent decision loops — a carefully tuned system prompt evaluates market signals and produces a trading decision. Speed matters more than marginal quality improvement. A 100ms decision that is 94% accurate is better than a 400ms decision that is 96% accurate when latency directly affects fill prices. Semantic Memory Layer, which handles complex knowledge retrieval and synthesis, uses multi-agent flows because users are asking questions where the quality delta matters more than the response speed. Neither system is wrong. Both systems made the calculation.

Run the numbers before you commit to the architecture. The architecture that earns the most revenue is not the most sophisticated one — it is the one that matches the actual cost structure of the problem it solves.

For external reading on AI decision-making under uncertainty and what the research actually says about AI confidence, Rewired Minds covers cognitive architecture. For production AI architecture patterns and deployment case studies from a live production ecosystem, jeremyknox.ai has the full documentation.