# LoRA (Low-Rank Adaptation)

**Source:** https://promtable.com/glossary/lora

> LoRA is a fine-tuning method that trains a small set of low-rank adapter weights on top of a frozen base model — cheaper to train and store than full fine-tuning.

---
LoRA is a fine-tuning method that trains a small set of low-rank adapter weights on top of a frozen base model — cheaper to train and store than full fine-tuning.

LoRA (Hu et al., 2021) inserts trainable rank-decomposition matrices into transformer layers while keeping the original weights frozen. The result: you can fine-tune a 70B-parameter model on a single GPU and store the adapter (a few MB) instead of a full checkpoint (140 GB). LoRA adapters can be hot-swapped at inference time, so one base model can serve many specialised tasks. QLoRA adds 4-bit quantisation, making fine-tuning a 70B model viable on a single 24 GB GPU. LoRA is the default fine-tuning technique in 2026 for open-weight LLMs and image diffusion models.

## When to use

- Customising open-weight models on small datasets (500–10,000 examples).
- Training a character or art style on Stable Diffusion / Flux.
- Multi-tenant deployments where many adapters share one base.

## Common mistakes

- Setting LoRA rank too low (under 4) — under-fits on complex tasks.
- Forgetting to merge LoRA into base weights for production latency-critical paths.

## Related terms

- [fine-tuning](https://promtable.com/glossary/fine-tuning)
- [diffusion-model](https://promtable.com/glossary/diffusion-model)
- [embeddings](https://promtable.com/glossary/embeddings)

## Sources

- [Hu et al. 2021 (arXiv)](https://arxiv.org/abs/2106.09685)

*Last updated: 2026-06-01*
---

Original page: https://promtable.com/glossary/lora
Maintained by Promtable (https://promtable.com). Content: CC BY 4.0. Cite as "Promtable — https://promtable.com/glossary/lora".
Contact: info@vibecodingturkey.com.