Self-consistency
Self-consistency runs the same prompt multiple times at non-zero temperature and picks the most common final answer, raising accuracy on reasoning tasks.
Introduced by Wang et al. (2022), self-consistency exploits the fact that reasoning chains often disagree along the way but converge on the same final answer if the prompt is good. You sample N reasoning chains, extract the final answer from each, and take the majority vote (or a weighted average for numeric answers). It is most useful on math, logic, and multi-step planning tasks where there's a discrete right answer. Cost scales linearly with N — production deployments usually pick N=3–5.
When to use self-consistency
- Math, logic, multi-step QA where there's one correct answer.
- Cost-tolerant inference where 3–5× compute is acceptable.
Common mistakes
- Using it on open-ended generation — no notion of 'majority'.
- Setting temperature to 0 (then all samples are identical).
FAQ
What is self-consistency?
Self-consistency runs the same prompt multiple times at non-zero temperature and picks the most common final answer, raising accuracy on reasoning tasks.
When should I use self-consistency?
Math, logic, multi-step QA where there's one correct answer. Cost-tolerant inference where 3–5× compute is acceptable.
What are the most common mistakes with self-consistency?
Using it on open-ended generation — no notion of 'majority'. Setting temperature to 0 (then all samples are identical).
Related terms
- Chain-of-thought prompting — Chain-of-thought (CoT) prompting tells a language model to write its reasoning steps before its final answer, increasing accuracy on multi-step problems.
- Temperature — Temperature is a sampling parameter that controls randomness in a language model's output, where 0 is fully deterministic and higher values introduce more variety.
Sources
Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/self-consistency.md.