concept

Function router LLM

A function router LLM is a small fast model whose only job is to classify the incoming request and emit a structured tool call — pre-deciding which function the downstream pipeline should run.

When agent tool counts grow past ~30, frontier models start mis-routing tools. A function router LLM specialises a small cheap model (GPT-4o-mini, Claude Haiku, Gemini Flash, or a fine-tuned open-weight) on the single task of routing — given a query, emit the correct function name + arguments. The strong executor model handles the actual response. This pattern combines well with semantic routing and is the production default in 2026 for cost-sensitive multi-tool agents. Best practice: fine-tune the router on your specific tool surface with synthetic queries + labelled tools for a measurable accuracy lift.

When to use function router llm

Cost-sensitive agents with diverse tool sets.
Multi-skill assistants where wrong-tool routing degrades quality fast.

Common mistakes

Using a too-weak router — mis-routes silently.
No fallback when the router is uncertain — pick a safe default or escalate.

FAQ

What is function router llm?

A function router LLM is a small fast model whose only job is to classify the incoming request and emit a structured tool call — pre-deciding which function the downstream pipeline should run.

When should I use function router llm?

Cost-sensitive agents with diverse tool sets. Multi-skill assistants where wrong-tool routing degrades quality fast.

What are the most common mistakes with function router llm?

Using a too-weak router — mis-routes silently. No fallback when the router is uncertain — pick a safe default or escalate.

Model router — A model router picks which language model handles each request based on cost, latency, or task type — the standard production pattern in 2026.
Tool router — A tool router is a layer in an agent that decides which tool to call (or which sub-agent to delegate to) for a given step — distinct from a model router which picks the underlying LLM.
Semantic routing — Semantic routing classifies an incoming query by meaning — via embedding similarity to predefined route prototypes — and dispatches it to the right model, agent, or sub-system.
Router LLM — A router LLM is a small fast language model whose only job is to classify or rewrite an incoming request — deciding which downstream model, agent, or tool should handle it.

Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/function-router-llm.md.