How tool calling works
By the end: You can give a model tools and handle the full round trip correctly.
- What actually happens during a tool call
- Describing a tool so the model uses it correctly
- Executing the call and returning the result
- The fact that the model never runs anything
New to tool calling? A model can only produce text, so by itself it cannot look anything up, do arithmetic you would trust, or change anything in the world. Tool calling is the arrangement that fixes this. You describe the functions available to it — their names, what they do, what arguments they take — and when the model wants one, its reply is a structured request to call it. Your code runs the function and sends the result back as another message. Why it matters
Everything so far has ended with the model producing an answer. Tool calling is how it gets to do more than that: you describe the actions available, the model asks for one, and your own program carries it out and reports back. This section follows that round trip a step at a time, because the division of labour within it is what people misread, and misreading it leads to designs that assume the model can reach out into the world by itself.
The model asks, your code acts#
This is the point people misunderstand. The model cannot run code, call an API or touch a database. It can only produce text. Tool calling is a convention where part of that text is a structured request, which your program reads and acts on.
1. You send: messages + a list of available tools
2. Model replies: "call get_weather with {city: 'Berlin'}"
3. YOUR code runs get_weather("Berlin") -> "18C, cloudy"
4. You send: everything again, plus that result
5. Model replies: "It is 18 degrees and cloudy in Berlin."Note steps 4 and 5. A tool call always needs a second request, because the model has to see the result before it can use it. One user question becomes at least two API calls.
Describing a tool#
A tool definition is a name, a description and a schema for its arguments. The description is not documentation for you. It is the only thing the model uses to decide whether this tool is the right one, so it should say when to use it, not just what it does.
{
"name": "search_orders",
"description": (
"Find a customer's orders by email address. "
"Use this when the user asks about order status, delivery "
"or refunds. Returns at most 10 of the most recent orders."
),
"parameters": {
"type": "object",
"properties": {
"email": {
"type": "string",
"description": "The customer's email address",
},
"status": {
"type": "string",
"enum": ["pending", "shipped", "delivered"],
"description": "Optional filter by order status",
},
},
"required": ["email"],
},
}Vague descriptions cause tool confusion
If two tools have similar descriptions, the model picks the wrong one and you will blame the model. Say explicitly when each should be used, and when it should not. Of everything you can edit in an agent, this does the most good for the least effort.
The round trip in code#
messages = [{"role": "user", "content": "Where is my order?"}]
response = client.chat(messages, tools=TOOLS)
if response.stop_reason == "tool_use":
for call in response.tool_calls:
result = TOOL_FUNCTIONS[call.name](**call.arguments)
messages.append(response.message) # what the model asked
messages.append({ # what it got back
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(result),
})
response = client.chat(messages, tools=TOOLS) # ask again, with results
print(response.text)Both appends are required. The model's request has to stay in the history, or the result arrives with nothing to attach to and the model loses track of what it asked.
Where people go wrong
Forgetting to append the model's tool request to the history before appending the result. The conversation then contains a result with no matching request, and the model either repeats the call or gets confused about what it was doing.
If this comes up in an interview
Walk me through what happens when a model calls a tool.
You send messages plus tool definitions. The model responds with a structured request naming a tool and arguments rather than a final answer. Your code executes that function, appends both the request and the result to the history, and calls the model again. The model then answers using the result. It never executes anything itself.
Then they push further
Where does the tool actually run?
In your application. The model only produces a structured request naming a tool and its arguments. Your code validates it, decides whether to run it, runs it, and sends the result back in the next request.
What do you check before running it?
That the tool exists and is allowed for this user, that the arguments pass the schema, and that the values make sense, such as an order id that belongs to this customer. The model's request is untrusted input.
What if the model asks for a tool that does not exist?
Return an error result that says so and lists what is available, and let it try again. If it keeps happening, the tool names or descriptions are probably confusing.
What gets marked up. Marked up for treating the model's tool request as untrusted input that must be validated and authorised. Weak answers imply the model runs the tool itself.
What makes a good tool description?
It says when to use the tool, not only what it does, and it says what the tool returns. The model uses the description to choose between tools, so vague or overlapping descriptions cause it to pick wrongly.
How many API calls does one tool-using question take?
At least two. One where the model requests the tool, one where it sees the result and answers. More if it chains several tools. This is why agents are slower and more expensive than a single completion.
Why use an enum in a parameter schema?
It constrains the argument to valid values at generation time, so an invalid status or category cannot be produced. It also removes a whole class of validation code from your handler.
Build this: Give a model two tools with deliberately similar descriptions and watch it pick the wrong one. Then rewrite the descriptions to say when each applies, and confirm the choice becomes reliable.
Watch it explained
This is one section of Tool calling, 7 hours across 2 sections. Read it in full to get the whole topic in order.
Stop 15 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.