Tool use: how the model gets eyes and hands

A tool is just a function plus a description the model can read. The model decides when to call it; you handle what happens when it does.

In this notebook: 1. The anatomy of a tool (name + description + JSON schema + callable) 2. Why JSON Schema specifically 3. Building a ToolRegistry and dispatching calls 4. The full tool-use round trip with the model 5. Trade-offs: tool granularity, security, error reporting

References: - Anthropic tool use docs - Toolformer (Schick et al., 2023) — first demonstration that LMs can learn to call APIs - See also: docs/PAPERS.md

⏱️ Session S6.2

One session (epilogue): read and run, then register a tool of your own design. Cards, prerequisites, and done-when tests: the Session Guide.

1. Anatomy of a tool

Four things:

field	what for
`name`	how the model refers to it
`description`	what it does — the model reads this every turn, so be specific and short
`input_schema`	JSON Schema for the arguments — the model’s fine-tuning teaches it to emit args matching this shape
`func`	the actual callable; receives kwargs from the model’s `input`

Code

import sys
sys.path.insert(0, '../../src')

from ai_playground.agents import Tool, ToolRegistry

weather = Tool(
    name='get_weather',
    description='Get current weather for a city. Returns temperature in Celsius and conditions.',
    input_schema={
        'type': 'object',
        'properties': {
            'city': {'type': 'string', 'description': 'City name, e.g. "Paris"'},
        },
        'required': ['city'],
    },
    func=lambda city: f'15°C, cloudy in {city}',  # stub — replace with a real API
)

print(weather.to_anthropic_schema())

2. Why JSON Schema

Every major model provider settled on the same convention: tool arguments described as JSON Schema, model emits a JSON object that matches.

Why this won:

Self-documenting. The schema tells the model what’s required, types, defaults, enums — no separate docs to fall out of sync.
Universal. Same format works for OpenAI, Anthropic, Gemini, Mistral. Write your tools once.
Constrained generation. Modern inference engines can mask logits to force valid JSON. The model literally cannot emit an invalid argument shape.

Good schemas have: - Specific description fields per property (the model reads these too, not just the top-level description) - required arrays so it knows what’s mandatory - Sensible enums where applicable — cuts model error rate

Bad schemas: - One giant string arg called "input" (just call the function with that string yourself) - No required (model omits args, you crash) - Vague descriptions (“does stuff with files”) — model picks wrong tool

3. The registry and dispatch

A ToolRegistry is a name→Tool dict with a dispatch method that handles exceptions. The agent loop calls dispatch for every tool_use block the model emits.

Code

registry = ToolRegistry([weather])

# Successful call
r = registry.dispatch('get_weather', {'city': 'Paris'}, call_id='abc')
print(f'success: content={r.content!r}, is_error={r.is_error}')

# Unknown tool — handled gracefully
r = registry.dispatch('get_stocks', {'ticker': 'AAPL'}, call_id='def')
print(f'unknown: content={r.content!r}, is_error={r.is_error}')

# Tool raises — wrapped, not propagated
broken = Tool(name='broken', description='', input_schema={}, func=lambda: 1/0)
registry.register(broken)
r = registry.dispatch('broken', {}, call_id='ghi')
print(f'crashed: content={r.content!r}, is_error={r.is_error}')

Why catch all exceptions? A misbehaving tool shouldn’t kill the agent. We hand the error back to the model with is_error=True, and let it decide: retry with different args, try a different tool, or apologize to the user. This is how real agents stay robust — failure is just another tool result.

4. The builtin tool set

We ship five capability primitives: file_read, file_write, calculator, shell_exec, web_fetch. These are deliberately low-level — a real domain agent would have task-specific tools (search_jira, send_slack_message, query_users_table) instead of raw shell access.

Code

from ai_playground.agents import builtin_tools

for t in builtin_tools(include_shell=False, include_web=False):
    print(f'- {t.name}: {t.description}')

Trade-off: tool granularity

Should you give the model shell_exec (one tool, infinite power) or list_files / read_file / write_file / git_status / git_diff / ... (many tools, narrow each)?

	one big tool	many small tools
Model errors	model misuses shell syntax	model picks wrong tool
Auditability	one log line per action	clear, structured trace
Security	full shell access	each tool can be sandboxed
Capability	anything possible	only what you exposed

Production agents almost always go narrow. The shell tool here is for quick experiments — if you build something real, replace it with the specific verbs your task needs.

5. The full round trip with a real agent

A two-turn agent task: call a tool, get a result, answer. We’ll script the model again so this runs offline.

Code

from ai_playground.agents import Agent
from ai_playground.agents.llm import LLMResponse, ToolCall

class ScriptedLLM:
    def __init__(self, responses):
        self.responses = list(responses)
    def complete(self, messages, tools=None, system=None, max_tokens=1024):
        return self.responses.pop(0)

llm = ScriptedLLM([
    LLMResponse(
        text="I'll check the weather.",
        tool_calls=[ToolCall(id='c1', name='get_weather', arguments={'city': 'Tokyo'})],
        stop_reason='tool_use',
    ),
    LLMResponse(
        text="It's 15°C and cloudy in Tokyo right now.",
        stop_reason='end_turn',
    ),
])

agent = Agent(llm=llm, tools=ToolRegistry([weather]))
result = agent.run("what's the weather in Tokyo?")
print('FINAL:', result.final_text)
print('\nTrace:')
for step in result.steps:
    print(f'  iter {step.iteration}: tools={[c.name for c in step.response.tool_calls]}, results={[r.content for r in step.tool_results]}')

6. Designing tools that the model will actually use well

A few rules of thumb that come up repeatedly:

Tool names should be verbs. search_users, send_email. Models call verb-shaped tools more reliably than noun-shaped ones.
Descriptions tell the model when to call. Not just what the tool does. “Use this when the user asks about prices” beats “queries the price table”.
Return concrete error messages. "user_id 42 not found" lets the model recover. "Error" doesn’t.
Keep arguments flat. Models are noticeably worse at deeply nested JSON. If you have nested objects, consider splitting into multiple tools or flattening.
Limit output size. Truncate large tool outputs — they eat context window and bury the answer. Our file_read defaults to 50KB for this reason.
Idempotency is forgiveness. If a tool is safe to call twice, model retries are free. If not, the model can corrupt state on its first error.

Key takeaways

A tool = name + description + JSON Schema + callable. Nothing more.
The model’s job is to pick tools and supply args; your job is to dispatch and return results.
Always wrap tool execution in try/except and report errors as data; never let them crash the loop.
Narrow, verb-shaped tools beat one big shell.

02_memory.ipynb — make the agent remember across turns and across sessions.

1. Anatomy of a tool

2. Why JSON Schema

3. The registry and dispatch

4. The builtin tool set

Trade-off: tool granularity

5. The full round trip with a real agent

6. Designing tools that the model will actually use well

Key takeaways

Next