Tool router
A tool router is a layer in an agent that decides which tool to call (or which sub-agent to delegate to) for a given step — distinct from a model router which picks the underlying LLM.
As agent tool counts grow past ~20, frontier models start mis-routing at higher rates. A tool router solves this by classifying intent first (using a small cheap model) and exposing only the relevant tools to the executor for that step. The pattern composes with model routing: tool router picks the tool set, model router picks the model that handles it best. Frameworks like LangGraph, OpenAI Swarm, and Claude Agent SDK ship variants. Best practice in 2026 is to design agents as small specialised sub-agents rather than monolithic ones — natural tool routing falls out of architecture.
When to use tool router
- Agents with more than ~20 tools.
- Multi-agent architectures.
Common mistakes
- Tool router that runs on every step — overhead can dominate.
- Skipping evals on the routing layer — routing accuracy is its own quality dimension.
FAQ
What is tool router?
A tool router is a layer in an agent that decides which tool to call (or which sub-agent to delegate to) for a given step — distinct from a model router which picks the underlying LLM.
When should I use tool router?
Agents with more than ~20 tools. Multi-agent architectures.
What are the most common mistakes with tool router?
Tool router that runs on every step — overhead can dominate. Skipping evals on the routing layer — routing accuracy is its own quality dimension.
Related terms
- AI agent — An AI agent is a system where a language model autonomously plans and executes a sequence of tool calls to accomplish a goal.
- Model router — A model router picks which language model handles each request based on cost, latency, or task type — the standard production pattern in 2026.
- Function calling (tool use) — Function calling lets a language model emit a structured request to invoke a developer-defined tool, enabling reliable JSON output and agent workflows.
- Tool use (LLM) — Tool use is the umbrella term for any LLM mechanism that lets the model invoke external functions, APIs, or services — function calling, code interpreter, MCP servers, browser actions.
Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/tool-router.md.