Router LLM

A router LLM is a small fast language model whose only job is to classify or rewrite an incoming request — deciding which downstream model, agent, or tool should handle it.

Router LLMs (typically GPT-4o-mini, Claude Haiku, Gemini Flash, or even a small open-weight model) handle the lightweight "what is this" step before the heavyweight "answer this" step runs. They classify intent, rewrite queries for retrieval, decide which expert agent to call, or pick the cheapest model that will meet quality. In 2026 router LLMs are the production default for cost-sensitive multi-skill apps because they cut frontier-model calls by 50-90% with minimal quality loss when the routing layer is evaluated and tuned.

When to use router llm

• Cost-sensitive production with diverse query types.
• Multi-agent stacks with skill routing.

Common mistakes

• Skipping evals on the router — routing errors are invisible without explicit monitoring.
• Using a router for tasks the frontier model could handle directly in fewer calls.

FAQ

Related terms

• Model router — A model router picks which language model handles each request based on cost, latency, or task type — the standard production pattern in 2026.
• Semantic routing — Semantic routing classifies an incoming query by meaning — via embedding similarity to predefined route prototypes — and dispatches it to the right model, agent, or sub-system.
• Tool router — A tool router is a layer in an agent that decides which tool to call (or which sub-agent to delegate to) for a given step — distinct from a model router which picks the underlying LLM.
• AI agent — An AI agent is a system where a language model autonomously plans and executes a sequence of tool calls to accomplish a goal.

Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/router-llm.md.