# Scaling law (LLM)

**Source:** https://promtable.com/glossary/scaling-law

> A scaling law is an empirical relationship — typically a power law — between a language model's loss and inputs like parameter count, training compute, or training data size.

---
A scaling law is an empirical relationship — typically a power law — between a language model's loss and inputs like parameter count, training compute, or training data size.

Kaplan et al. (2020) and Chinchilla (Hoffmann et al., 2022) established that LLM loss scales predictably with parameters, data, and compute. The Chinchilla finding — that for a given compute budget there's an optimal balance of parameters and tokens (~20 tokens per parameter) — reshaped how labs trained models. In 2026 scaling laws still drive frontier model design, plus newer laws describe how reasoning capability scales with extra inference compute (test-time scaling). The practical implication: "bigger model" alone isn't the answer — it's bigger model, more tokens, and more inference compute together.

## Common mistakes

- Comparing models on parameter count alone — token count and inference compute matter as much.

## Related terms

- [mixture-of-experts](https://promtable.com/glossary/mixture-of-experts)
- [fine-tuning](https://promtable.com/glossary/fine-tuning)
- [reasoning-model](https://promtable.com/glossary/reasoning-model)

## Sources

- [Chinchilla (Hoffmann et al., 2022)](https://arxiv.org/abs/2203.15556)

*Last updated: 2026-06-01*
---

Original page: https://promtable.com/glossary/scaling-law
Maintained by Promtable (https://promtable.com). Content: CC BY 4.0. Cite as "Promtable — https://promtable.com/glossary/scaling-law".
Contact: info@vibecodingturkey.com.