Skip to content

VRAM reference

How much GPU memory each open model needs

Memory for fine-tuning and running 25 popular open-weights models, worked out from each model's published architecture. The figures use 2,048-token training sequences, LoRA rank 16 and an 8K-token inference context; each model's page has the full breakdown, a GPU-by-GPU verdict and a calculator for other settings.

Checked 15 November 2026 · Prices as of November 2026
ModelParametersQLoRALoRAFull fine-tuneRun, 16-bitRun, 4-bit
Llama 3.1 8B8.03B9.4 GB20 GB132 GB17 GB5.6 GB
Llama 3.1 70B70.6B48 GB150 GB1,134 GB144 GB42 GB
Llama 3.2 1B1.24B3.7 GB5.1 GB22 GB2.7 GB1.0 GB
Llama 3.2 3B3.21B5.3 GB9.5 GB54 GB7.4 GB2.7 GB
Llama 3.3 70B70.6B48 GB150 GB1,134 GB144 GB42 GB
Qwen3 0.6B0.75B3.5 GB4.4 GB15 GB2.4 GB1.4 GB
Qwen3 1.7B2.03B4.7 GB7.2 GB35 GB5.0 GB2.1 GB
Qwen3 4B4.02B6.3 GB12 GB67 GB9.3 GB3.5 GB
Qwen3 8B8.19B10 GB21 GB135 GB18 GB5.8 GB
Qwen3 14B14.8B15 GB34 GB240 GB31 GB9.6 GB
Qwen3 32B32.8B26 GB72 GB529 GB68 GB21 GB
Qwen3 30B-A3B30.5B (3.3B active)20 GB64 GB492 GB62 GB18 GB
Qwen2.5 7B7.62B9.5 GB19 GB125 GB16 GB4.8 GB
Qwen2.5 72B72.7B51 GB155 GB1,169 GB148 GB44 GB
Mistral 7B7.25B6.3 GB17 GB117 GB16 GB5.2 GB
Mixtral 8x7B46.7B (12.9B active)26 GB95 GB749 GB94 GB27 GB
Gemma 3 4B4.30B8.6 GB14 GB74 GB8.9 GB2.7 GB
Gemma 3 12B12.2B14 GB31 GB200 GB25 GB7.7 GB
Gemma 3 27B27.4B24 GB63 GB445 GB56 GB17 GB
Phi-414.7B13 GB33 GB238 GB31 GB9.9 GB
DeepSeek-R1-Distill-Qwen-7B7.62B9.5 GB19 GB125 GB16 GB4.8 GB
DeepSeek-R1-Distill-Qwen-32B32.8B26 GB72 GB529 GB68 GB21 GB
DeepSeek-R1-Distill-Llama-70B70.6B48 GB150 GB1,134 GB144 GB42 GB
gpt-oss-20b20.9B (3.6B active)17 GB46 GB339 GB42 GB14 GB
gpt-oss-120b116.8B (5.1B active)66 GB238 GB1,873 GB234 GB66 GB

By GPU

RTX 3090 (24 GB)

Llama 3.1 8B · Llama 3.2 1B · Llama 3.2 3B · Qwen3 4B · Qwen3 8B · Qwen3 14B · Mistral 7B · Gemma 3 12B · Phi-4 · gpt-oss-20b

RTX 4090 (24 GB)

Llama 3.1 8B · Llama 3.1 70B · Llama 3.2 1B · Llama 3.2 3B · Llama 3.3 70B · Qwen3 4B · Qwen3 8B · Qwen3 14B · Qwen3 32B · Qwen3 30B-A3B · Qwen2.5 7B · Mistral 7B · Mixtral 8x7B · Gemma 3 4B · Gemma 3 12B · Gemma 3 27B · Phi-4 · DeepSeek-R1-Distill-Qwen-7B · DeepSeek-R1-Distill-Qwen-32B · gpt-oss-20b

RTX 5090 (32 GB)

Llama 3.1 8B · Llama 3.1 70B · Qwen3 8B · Qwen3 14B · Qwen3 32B · Qwen3 30B-A3B · Qwen2.5 7B · Mistral 7B · Mixtral 8x7B · Gemma 3 12B · Gemma 3 27B · Phi-4 · DeepSeek-R1-Distill-Qwen-7B · DeepSeek-R1-Distill-Qwen-32B · gpt-oss-20b · gpt-oss-120b

L40S (48 GB)

Llama 3.1 8B · Qwen3 8B · Qwen3 14B · Qwen3 32B · Qwen3 30B-A3B · Gemma 3 27B

A100 40GB (40 GB)

Llama 3.1 8B

A100 80GB (80 GB)

Llama 3.1 8B · Llama 3.1 70B · Qwen3 32B · Qwen2.5 72B · Mixtral 8x7B

H100 SXM (80 GB)

Llama 3.1 8B · Llama 3.1 70B · Llama 3.3 70B · Qwen3 8B · Qwen3 14B · Qwen3 32B · Qwen3 30B-A3B · Qwen2.5 72B · Mixtral 8x7B · Gemma 3 27B · DeepSeek-R1-Distill-Qwen-32B · DeepSeek-R1-Distill-Llama-70B · gpt-oss-20b · gpt-oss-120b

H200 (141 GB)

Llama 3.1 70B · Llama 3.3 70B · Qwen2.5 72B · DeepSeek-R1-Distill-Llama-70B · gpt-oss-120b

B200 (180 GB)

Llama 3.1 70B · gpt-oss-120b

How we estimate it

Every figure comes from the model's published architecture (layers, hidden size, attention and KV heads, vocabulary) rather than a rule of thumb, using the same functions as the calculator above. Sizes are in GB (10⁹ bytes).

  • Full fine-tuning: 16 bytes per parameter for mixed-precision Adam (bf16 weights and gradients, fp32 master weights and two moments), as in the ZeRO paper.
  • LoRA: frozen bf16 weights (2 bytes per parameter) plus 16 bytes per adapter parameter, rank 16 on every linear layer. For mixture-of-experts models, adapters go on attention only and the experts stay frozen.
  • QLoRA: NF4 weights with double quantisation (4.127 bits per parameter); embeddings and the output head stay in bf16, as bitsandbytes leaves them.
  • Activations: gradient checkpointing and FlashAttention. Each layer's bf16 input is kept, one layer is recomputed at a time, and the output logits take 8 bytes per vocabulary entry per token (fp32 values and gradients). Chunked cross-entropy, as in Unsloth or Liger kernels, cuts that last term.
  • Inference: 2, 1 or about 0.56 bytes per weight (4-bit with group-wise scales), plus an fp16 KV cache of 2 × layers × KV heads × head dim × 2 bytes per token; sliding-window layers only cache their window.
  • Fits means the estimate is within 90% of the card's listed memory; the rest covers the CUDA context, kernels and fragmentation. Multi-GPU counts assume memory splits evenly.

More tools

Sources

  1. [1]Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)
  2. [2]Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)
  3. [3]Rajbhandari et al., 2019. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054)
  4. [4]Korthikanti et al., 2022. Reducing Activation Recomputation in Large Transformer Models (arXiv:2205.05198)
  5. [5]Chen et al., 2016. Training Deep Nets with Sublinear Memory Cost (arXiv:1604.06174)

Most agents don't need a bigger GPU.

Kojen's prompt tuning and model search run for real on hosted models against your own test cases. Free to start, then pay as you go.

Start training free