Tree of Thought (ToT)

Tree of Thought prompts a model to branch into multiple reasoning paths, evaluate them, and search for the best one.

Tree of Thought (ToT) generalizes chain-of-thought: instead of one linear reasoning trace, the model generates several candidate "thoughts" at each step, scores or votes on them, and explores the most promising branches — backtracking when a path looks weak. It frames reasoning as a search over a tree, which suits problems with many viable intermediate steps (planning, puzzles, multi-constraint writing, math). ToT trades extra tokens and latency for higher reliability on hard tasks where a single greedy chain often commits early to a wrong step. In practice it is implemented with a generator prompt, an evaluator prompt, and a controller that decides which branches to expand.

When to use tree of thought (tot)

• Hard reasoning or planning tasks where one-shot chain-of-thought is unreliable.
• Problems with a checkable intermediate state (the evaluator can score partial progress).
• When accuracy matters more than cost or speed.

When not to use tree of thought (tot)

• Simple lookups or single-step answers — the extra branches waste tokens.
• Latency-sensitive UX where each query must return fast.

Example

Input: Solve a scheduling puzzle: generate 3 candidate first moves, evaluate each, expand the best two.
Output: The model discards the dead-end branch early and returns a valid schedule from the surviving path.

Common mistakes

• Using ToT for trivial tasks where chain-of-thought (or no reasoning) already works.
• Skipping the evaluator step — branching without scoring is just expensive sampling.
• Letting the tree grow unbounded; cap breadth and depth to control cost.

FAQ

Related terms

• Chain-of-thought prompting — Chain-of-thought (CoT) prompting tells a language model to write its reasoning steps before its final answer, increasing accuracy on multi-step problems.
• Few-shot prompting — Few-shot prompting supplies 2–10 input–output examples inside the prompt so the model imitates the pattern on a new input.
• System prompt — A system prompt is the high-priority instruction block that defines a model's role, constraints, and default behaviors for an entire conversation.
• Prompt engineering — Prompt engineering is the practice of designing input text that reliably steers a large language model toward a specific output.

Sources

• Tree of Thoughts (Yao et al., 2023)

Last updated: 2026-06-02. Raw markdown: https://promtable.com/glossary/tree-of-thought.md.