Building with tool-calling APIs

The short version

Tool calling turns a chatbot into software: you describe your functions as JSON schemas, the model decides when to call them and with what arguments, your code executes, and the results go back into the conversation. Every agent you've used is this loop with better marketing. Getting it right is mostly about API design, not model cleverness.

Designing tools the model can actually use

Names and descriptions are the UI. The model never sees your implementation — only the name, description, and schema. get_customer_orders with "Returns a customer's recent orders, most recent first" beats query_db_v2 every time. Write descriptions like documentation for a smart intern.

Fewer, chunkier tools beat many tiny ones. Ten overlapping tools create decision paralysis and misfires. Five well-separated tools with clear boundaries get called correctly. If two tools are often confused, merge them and add a parameter.

Schemas are contracts. Required vs optional matters enormously — models treat "optional" as "probably skip." Enums beat free strings for anything with a fixed set of values. And validate server-side anyway: the model will eventually pass you a date like "next Tuesday."

The loop, done right

1. System prompt sets the role and the available tools.
2. Model responds with either text or tool calls (possibly several in parallel — design for it).
3. You execute the calls, with timeouts and error handling. A tool that hangs hangs your agent.
4. Results go back as tool messages. Keep them compact — a 50KB JSON dump eats the context window that the next reasoning step needed.
5. Repeat until the model answers directly or hits your max-iteration cap. Always have a cap.

Failure modes

• The model won't call the tool. Usually a description problem, or the tool wasn't obviously relevant. Also check: some models need an explicit nudge ("use the search tool for current events").
• Argument hallucination. IDs and enums get invented. Constrain with enums, validate, and return clear errors — models self-correct well when the error message says what was wrong.
• Tool result overload. Returning everything "just in case" drowns the reasoning. Return what the tool was asked for, paginate the rest.

Rule of thumb: design your tools as if the caller is brilliant but has never seen your codebase — because that's exactly what it is. And cap the loop; an uncapped agent is a billing incident waiting to happen.