Skip to content

Glossary · Training

QLoRA

QLoRA is LoRA on top of a base model whose frozen weights are stored in 4-bit, cutting fine-tuning memory roughly four-fold compared with LoRA on 16-bit weights.

Also called Quantized LoRA · Reviewed 15 November 2026

Dettmers et al. introduced the 4-bit NormalFloat (NF4) data type, double quantisation of the scaling constants (about 4.127 bits per weight in total) and paged optimisers, and fine-tuned a 65B model on a single 48 GB GPU.

Weights are dequantised on the fly for each matrix multiplication, so QLoRA is somewhat slower than LoRA on the same GPU; it trades speed for memory.

Learn more on Kojen

Sources

  1. [1]Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)

All 31 terms

Put the definitions to work.

Kojen evaluates, tunes and red-teams your agents with these methods built in: Wilson intervals, pass^k, LLM judges with visible components. Free to start.

Start training free