Function Calling and Tool Use

Language models are word predictors by default. Function calling is what turns them into action takers. It is the mechanism that lets a model reach outside its context window and interact with real systems — databases, APIs, file systems, external services.

The model does not execute code. It identifies what code should be called and provides arguments. Your code does the execution. This separation is important to understand deeply.

How It Works

The full sequence:

1. You define one or more tools as JSON Schema objects
2. You include those tool definitions in the API call
3. The model reads the user's message and decides whether to call a tool
4. If yes, the model returns a tool_calls object instead of (or in addition to) content
5. Your code reads the tool_calls, executes the named function with the provided arguments
6. Your code sends the result back to the model via a tool role message
7. The model synthesizes the function result into a final natural language response

Defining Tools

Tools are defined using JSON Schema inside a tools array:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city. Use this when the user asks about weather conditions.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name and state/country, e.g. 'Raleigh, NC'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

The description fields are critical. The model uses them to decide when to call a tool. Vague descriptions produce unreliable routing. Specific descriptions with usage examples in the description text produce consistent behavior.

Handling the tool_calls Response

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Raleigh right now?"}],
    tools=tools
)

choice = response.choices[0]

if choice.finish_reason == "tool_calls":
    for tool_call in choice.message.tool_calls:
        func_name = tool_call.function.name
        func_args = json.loads(tool_call.function.arguments)

        # YOUR code executes the function
        if func_name == "get_weather":
            result = get_weather(func_args["city"], func_args.get("unit", "fahrenheit"))

        # Return result to model
        messages.append(choice.message)  # append the assistant's tool_call message
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result)
        })

    # Get final response
    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools
    )
    print(final_response.choices[0].message.content)

Parallel Function Calling

GPT-4o can invoke multiple tools in a single response when the information gathering is parallelizable. If a user asks "What's the weather in Raleigh and NYC?" the model may return two tool_calls in one response.

if choice.finish_reason == "tool_calls":
    # May be multiple tool calls
    for tool_call in choice.message.tool_calls:
        # Handle each one — order matters for the messages array
        result = dispatch_tool(tool_call.function.name,
                               json.loads(tool_call.function.arguments))
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result)
        })

You must return a tool message for every tool_call before making the next API call. Missing tool results cause API errors.

tool_choice Parameter

By default, the model decides whether to call a tool. You can override this:

# Force a specific tool call
tool_choice={"type": "function", "function": {"name": "get_weather"}}

# Prevent any tool calls
tool_choice="none"

# Let model decide (default)
tool_choice="auto"

# Force a tool call (any tool)
tool_choice="required"

Use tool_choice="required" when you need guaranteed structured output — the model will always return a tool invocation, never plain text.

Error Handling When Tools Fail

Your function can fail. The API response should still include a tool result message — include the error in it:

try:
    result = get_weather(city)
    tool_content = json.dumps({"success": True, "data": result})
except Exception as e:
    tool_content = json.dumps({"success": False, "error": str(e)})

messages.append({
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": tool_content
})

The model handles failure gracefully when you pass it the error message — it will typically explain the failure to the user rather than hallucinating a result.

Real-World Tool Patterns

Database queries: The model identifies what data to fetch, calls a query_database tool with filters as arguments, and synthesizes the returned rows into a response. The model never sees your database credentials or schema — it only sees the tool definition and the results you return.

Calculator/computation: For precise arithmetic, a calculate tool that evaluates math expressions prevents the model from hallucinating numerical results.

API integrations: Weather, stock prices, flight data, CRM records — any external API becomes available to the model as a tool. The model becomes the user-facing interface; your tool definitions determine what it can access.

Write operations: File creation, database inserts, email sending. Be careful with destructive operations — require confirmation before write tools execute.

Bottom Line

Function calling is the pattern that turns a language model into an agent. The model handles natural language interpretation and decision-making. Your code handles execution and data access. The boundary is clean, secure, and composable.

The next lesson covers structured outputs — how to force the model to return exactly the JSON schema you need, with zero hallucinated fields.