Fine-tuning

Fine-tuning updates a pretrained model's weights on task-specific data, baking the new behaviour into the model rather than relying on prompts.

Fine-tuning takes a foundation model and trains it further on domain-specific examples (often via LoRA or QLoRA adapters rather than full weight updates) so the new behaviour is reproducible without long prompts. It is the right call when prompting plateaus on a high-volume narrow task, when latency budget excludes few-shot examples, or when you need to ship a tone/style consistently. It is overkill when the task is rare, when data is small, or when system-prompt + few-shot already meets quality. In 2026 most teams fine-tune open-weight models (Llama 4, Mistral, Qwen) rather than closed APIs because the cost is lower and ownership matters.

When to use fine-tuning

• High-volume narrow task where prompting plateaus.
• Strict latency budget excluding long prompts.
• Custom domain language (legal, medical, code).

Common mistakes

• Fine-tuning before exhausting prompt + few-shot options.
• Tiny datasets (under ~500 examples) — usually noise.

FAQ

Related terms

• Prompt engineering — Prompt engineering is the practice of designing input text that reliably steers a large language model toward a specific output.
• Few-shot prompting — Few-shot prompting supplies 2–10 input–output examples inside the prompt so the model imitates the pattern on a new input.
• System prompt — A system prompt is the high-priority instruction block that defines a model's role, constraints, and default behaviors for an entire conversation.

Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/fine-tuning.md.