# LoRA fine-tune

**Source:** https://promtable.com/glossary/lora-fine-tune

> LoRA (Low-Rank Adaptation) fine-tune is the parameter-efficient method that trains small adapter matrices on top of frozen base weights — 10-100× cheaper than full fine-tune, swappable per task, easy to serve many LoRAs from one base model.

---
LoRA (Low-Rank Adaptation) fine-tune is the parameter-efficient method that trains small adapter matrices on top of frozen base weights — 10-100× cheaper than full fine-tune, swappable per task, easy to serve many LoRAs from one base model.

Full fine-tune updates all base model parameters — expensive in compute + memory + storage. LoRA freezes the base + trains low-rank matrices (`W' = W + AB^T` where A, B are tiny) — typically < 1% of base params. Benefits: 10-100× cheaper training, tiny adapter files (10-100 MB vs 100+ GB full model), easy to swap LoRAs at serve time, lets one base model serve many specialized variants (per-tenant, per-language, per-task). 2026 production patterns: train per-customer LoRAs in minutes-to-hours, serve dozens of LoRAs from one base on multi-LoRA inference engines (Predibase LoRAX, vLLM multi-LoRA). Trade-offs: LoRA quality usually approaches but doesn't quite match full fine-tune on hardest tasks; merging multiple LoRAs is non-trivial.

## When to use

- Per-customer / per-task model specialization.
- Cost-sensitive fine-tunes.

## Common mistakes

- Picking too-low rank — quality cap on hard tasks.
- Over-training — LoRA overfits faster than full fine-tune.

## Related terms

- [lora](https://promtable.com/glossary/lora)
- [lora-stacking](https://promtable.com/glossary/lora-stacking)
- [lora-hot-swap](https://promtable.com/glossary/lora-hot-swap)

*Last updated: 2026-06-01*
---

Original page: https://promtable.com/glossary/lora-fine-tune
Maintained by Promtable (https://promtable.com). Content: CC BY 4.0. Cite as "Promtable — https://promtable.com/glossary/lora-fine-tune".
Contact: info@vibecodingturkey.com.