Prompt Tuning, Fine-Tuning & Reasoning
Lesson 3 of 5Intermediate 9 min

Fine-tuning efficiently with LoRA and QLoRA

Fine-tuning every weight is expensive. LoRA trains a tiny add-on instead; QLoRA does it on a quantised model to fit on one GPU.

In this lesson you'll

  • Explain how low-rank adaptation works
  • Explain what quantisation adds in QLoRA
  • Decide when fine-tuning is worth it

Full fine-tuning updates every weight, so each task needs its own full copy of the model. LoRA freezes the pretrained weights and trains a pair of small low-rank matrices beside selected layers. For GPT-3 175B, the authors reported 10,000× fewer trainable parameters and 3× less GPU memory than full fine-tuning, with quality on par or better and no added inference latency, since the matrices can be merged into the weights [1].

QLoRA backpropagates through a base model quantised to 4 bits into LoRA adapters. It introduced the 4-bit NormalFloat data type and paged optimisers, and made it possible to fine-tune a 65B-parameter model on a single 48GB GPU [2]. Libraries such as Hugging Face PEFT package these methods for everyday use [3].

When is fine-tuning worth it?

  • Prompt tuning has plateaued below your ship threshold on held-out data.
  • You need a fixed style or format that prompts can't hold consistently.
  • You want a smaller, cheaper model to match a larger one on a narrow task.
  • You have hundreds to thousands of clean, representative examples [4].

In Kojen

LoRA, QLoRA and full fine-tuning for text models are previews. You can configure them in the Forge, but the metrics are simulated and no text model weights are trained yet. Use prompt tuning or model search for real results today.

Key takeaways

  • LoRA trains small add-on matrices; the base model stays frozen.
  • QLoRA adds 4-bit quantisation to fit big models on one GPU.
  • Fine-tune only after prompting has measurably plateaued.

Practise in the dojo

Look at the LoRA and QLoRA settings in the Forge to see the knobs (rank, learning rate, epochs). Remember these runs are previews with simulated metrics.

Explore the Forge

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. What does LoRA train?
2. What made QLoRA's single-GPU fine-tuning of a 65B model possible?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Hu, E. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022 · arXiv:2106.09685.
  2. [2]Dettmers, T. et al. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023 · arXiv:2305.14314.
  3. [3]Hugging Face (2025). PEFT: Parameter-Efficient Fine-Tuning documentation. huggingface.co.
  4. [4]Zhou, C. et al. (2023). LIMA: Less Is More for Alignment. NeurIPS 2023 · arXiv:2305.11206.