Mastering Function Calling & Tools

Dec 29, 2025 • 30 min read

Function Calling (or "Tools") is the bridge between the LLM's brain and the outside world. It turns a passive text generator into an Agent that can query databases, send emails, or control software.

1. The Core Mental Model: The API Glue

Crucially, the LLM does not execute the code. It generates the JSON arguments for you to execute the code.

2. Defining Tools with Zod (The Modern Way)

Writing raw JSON schemas is painful and error-prone. Modern AI engineering uses Zod to define schemas in TypeScript.

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// 1. Define the Schema with Zod
const WeatherSchema = z.object({
  location: z.string().describe("City and state, e.g. San Francisco, CA"),
  unit: z.enum(["celsius", "fahrenheit"]).optional().default("celsius")
});

// 2. Convert to OpenAI format
const tools = [{
  type: "function",
  function: {
    name: "get_weather",
    description: "Get current temperature", // CRITICAL: The model reads this!
    parameters: zodToJsonSchema(WeatherSchema)
  }
}];

Why "Description" is the Most Code

The description field is not a comment. It is part of the prompt. If your tool is meant to "Search the user's email", but you describe it as "Search", the model might confuse it with "Search Google".Be Verbose.

3. The "Human-in-the-Loop" Pattern

What if the tool is dangerous? (e.g., transfer_money). You cannot let the AI run this autonomously. You need a confirmation step.

// Server-Side Logic
if (toolCall.name === "transfer_money") {
  // DO NOT EXECUTE.
  // Return a special UI state to the client.
  return {
    ui: <ConfirmationCard amount={500} to="Bob" />,
    status: "waiting_for_approval"
  };
}

4. Parallel Function Calling

A common mistake is assuming the model calls one tool at a time. It can call multiple.

If a user asks: "What's the weather in Tokyo, Paris, and London?", the model will return an array of 3 tool calls in a single response.

// The Model Response (simplified)
{
  tool_calls: [
    { id: "call_1", name: "get_weather", args: '{ "location": "Tokyo" }' },
    { id: "call_2", name: "get_weather", args: '{ "location": "Paris" }' },
    { id: "call_3", name: "get_weather", args: '{ "location": "London" }' }
  ]
}

// Your Job: Execute ALL of them in parallel
const promises = tool_calls.map(call => executeTool(call));
const results = await Promise.all(promises);

5. Structured Outputs (Strict Mode)

OpenAI recently introduced strict: true. This guarantees that the output matches your schema 100% of the time, solving the "missing bracket" or "wrong type" issues.

This is essential for Data Extraction pipelines.

const response = await openai.chat.completions.create({
  model: "gpt-4o-2024-08-06",
  messages: [...],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "extraction",
      schema: MyZodSchema,
      strict: true // <--- The Limit Saver
    }
  }
});

6. Real World Use Case: The Refund Agent

Let's build a Customer Support Agent that can issue refunds.

The Toolset

• look_up_order(order_id): Read-only. Safe.
• issue_refund(order_id): Definite Side Effect. Dangerous.

The Prompt Strategy

"You are a helpful support agent. First, ALWAYS look up the order to verify eligibility. Only if the order is eligible (less than 30 days old), call the refund tool."

This is "Chain of Thought" via Tooling. The model effectively "checks its work" before taking action.

7. FAQ: Common Pitfalls

8. Conclusion

Tools are what separate "Chatbots" from "Agents". By mastering Zod schemas, parallel execution, and approval workflows, you can build systems that do real work in the real world.