Tool Calling as Capability Design

Tools Are a Product Decision Before They Are a Technical One

Most engineers approach tool calling as a technical question: which API should I expose to the model? That framing misses the more important question: what does this tool let the AI do that it cannot do without it?

A search tool does not make your AI better at answering questions it already knows. It makes your AI capable of answering questions about things that happened after its training cutoff — today's news, current prices, live system status. That is a different product. A code execution tool does not make your AI better at explaining code. It makes your AI capable of actually running code and returning real output. That is a different product.

Before writing a single line of tool implementation, write one sentence: "This tool turns my AI from [X] to [Y]." If you cannot complete that sentence with a meaningful before-and-after, you are not ready to implement the tool.

The Three Principles of Tool Design

The design principles for tools are not about implementation complexity. They are about how tools behave when integrated into a live AI system.

Atomic. A tool does one job and returns one shape. searchWeb(query: string): {results: SearchResult[]} is an atomic tool. searchAndSummarize(query: string): string is not — it combines two operations with different failure modes, different caching requirements, and different return types. Atomic tools are independently testable. When a combined tool fails, you do not know which operation failed. When an atomic tool fails, the failure is immediate and specific.

Reliable. A tool must always return a value the AI can use. This means handling errors internally and returning a structured result in both success and failure cases. If the underlying API times out, the tool should not throw an uncaught exception — it should return {error: 'timeout', fallback: null}. The AI's reasoning loop must always receive a tool result it can work with. If it receives nothing, it will fill the gap. Filled gaps are hallucinations.

Cheap. A tool adds latency and cost to every call that uses it. Good tools run in under 100ms and cost less than the LLM call they support. A tool that queries three external APIs sequentially before returning adds 400-600ms to every user turn. Over a day of traffic, that is a latency penalty that compounds into degraded user experience. Cache aggressively, parallelize where you can, and set timeouts so a slow external service does not block the entire response.

Tools vs. Context Injection vs. RAG

These three patterns solve overlapping problems, and choosing the wrong one for the job produces either poor quality or unnecessary cost.

Tools are right when you need fresh external data at request time or when you need side effects. Checking a live stock price, triggering a system event, sending an email, reading a user's current account status — these are tool calls. The data does not exist until the call is made.

Context injection is right when the data is small, stable, and always relevant. If every user request needs to know the current user's name, subscription tier, and preferences, inject that at the start of every request in the system message. Context injection is zero latency and zero cost — you already have the data. Do not make a tool call to retrieve information you could have included in the prompt.

RAG (Retrieval Augmented Generation) is right when you have a large stable corpus that users query selectively. A documentation search system, a knowledge base, a product catalog with thousands of items. RAG retrieves only what is relevant to the current query rather than injecting the entire corpus into every prompt. Semantic Memory Layer uses RAG to index 2,968 chunks of knowledge across 43 repositories — retrieving relevant context on demand rather than including everything in every prompt.

The failure mode is using tools for data that should be injected, or injecting large corpora that should be in RAG. A user's subscription tier should be in the system message, not fetched by a tool on every request. A 500-page documentation corpus should be in RAG, not injected as context.

The Tool Registry Pattern

As your product grows from two tools to ten, the question of which tools the AI can call — and under what conditions — becomes a governance problem.

The tool registry is a central configuration that answers three questions for every tool:

1. What does this tool do? Name, description, and input schema. The AI uses this to decide whether to call the tool.
2. Which models can use this tool? Your task-specialist model gets the full tool set. Your question-specialist model, which never needs to execute code, should not be offered the code execution tool. Offering tools the AI should never use creates noise in the model's tool selection reasoning.
3. Which user tiers can trigger this tool? A free-tier user asking the AI to browse the web should get a helpful message about upgrading, not a free call to an expensive external API. Implement authorization at the tool registry level, not scattered through individual tool implementations.

Mission Control's tool registry follows this pattern. The tools for fetching trading bot status, querying portfolio metrics, and triggering system events are each registered with their access tier. A read-only session cannot trigger system events. A guest user cannot access portfolio data. The AI does not make the authorization decision — the registry makes it before the tool is offered.

Error Handling in the Tool Loop

Tool errors are a first-class concern, not an edge case. In production, external APIs fail, rate limits are hit, network calls time out, and data formats change without notice.

The AI receives the tool result and must reason about it. If the tool throws an uncaught exception, the AI either hallucinates or the request fails entirely. Neither is acceptable.

The correct approach: every tool wraps its core logic in a try/catch. On success, return the data in a consistent shape. On failure, return a structured error object that the AI can interpret:

async function getStockPrice(ticker: string) {
  try {
    const price = await externalAPI.fetchPrice(ticker)
    return { price, timestamp: Date.now(), source: 'live' }
  } catch (err) {
    return { error: 'price service unavailable', fallback: null, ticker }
  }
}

The AI receives this error result and can respond: "I tried to get the current price for AAPL but the service is temporarily unavailable. Based on recent training data, AAPL was trading around $185 in late 2024, but I'd recommend checking a live source for current pricing."

That is a useful response. A crash or a hallucinated price is not.

The trading bot ecosystem Knox runs on Tesseract handles this with a strict rule: every external API call has a 30-second asyncio.wait_for timeout, and every tool wrapper returns a structured result regardless of outcome. The bots never hang on a failed API call, and the decision logic always receives a result it can act on.

For production infrastructure patterns and how tools integrate into a complete production AI stack, the detailed architecture documentation is at jeremyknox.ai. For AI decision-making systems and how tools support better outcomes under uncertainty, indecision.io covers the applied research.