Skip to content

VRAM reference

Llama 3.2 1B VRAM requirements

Fine-tuning Llama 3.2 1B takes about 3.7 GB of GPU memory with QLoRA, 5.1 GB with LoRA on 16-bit weights and 22 GB for full fine-tuning, at 2,048 tokens per sequence. Running it takes 2.7 GB in 16-bit or 1.0 GB in 4-bit with a 8K-token context.

By Will Schulz · Architecture and prices checked 15 November 2026 · Prices as of November 2026

Fine-tuning memory

At 2,048 tokens per sequence, batch 1, LoRA rank 16, gradient checkpointing on
MethodWeightsTraining stateActivationsTotalSmallest single GPU80 GB GPUs needed
QLoRA (4-bit base)1.0 GB0.2 GB2.4 GB3.7 GBRTX 3090 (24 GB)1
LoRA (16-bit base)2.5 GB0.2 GB2.4 GB5.1 GBRTX 3090 (24 GB)1
Full fine-tuning2.5 GB17 GB2.4 GB22 GBRTX 5090 (32 GB)1

On a 24 GB card such as the RTX 4090: QLoRA: yes, about 3.7 GB at 2K tokens, batch 1, rank 16; up to 16K-token sequences at batch 1 or batch 8 at 2K, LoRA rank up to 1024.

Inference memory

One sequence, fp16 KV cache
PrecisionWeightsKV cache at 8KTotalAt 128K contextSmallest single GPU
16-bit (BF16/FP16)2.5 GB0.3 GB2.7 GB6.8 GBRTX 3090 (24 GB)
8-bit1.2 GB0.3 GB1.5 GB5.5 GBRTX 3090 (24 GB)
4-bit0.7 GB0.3 GB1.0 GB5.0 GBRTX 3090 (24 GB)

Which GPUs fit Llama 3.2 1B

One GPU unless a count is shown. Follow a GPU's name for its maximum sequence length, batch size, LoRA rank and rental cost.

GPUQLoRALoRAFull fine-tuningRun, fp16Run, int8Run, int4
RTX 3090 24 GBYesYes2×YesYesYes
RTX 4090 24 GBYesYes2×YesYesYes
RTX 5090 32 GBYesYesYesYesYesYes
L40S 48 GBYesYesYesYesYesYes
A100 40GB 40 GBYesYesYesYesYesYes
A100 80GB 80 GBYesYesYesYesYesYes
H100 SXM 80 GBYesYesYesYesYesYes
H200 141 GBYesYesYesYesYesYes
B200 180 GBYesYesYesYesYesYes

What's specific to Llama 3.2 1B

  • Small enough for full fine-tuning on one 24 GB card, which is rare: most models need several GPUs for that.
  • A fifth of its parameters are the 128K-entry embedding table, so the logits dominate training activations.
ArchitectureLlama 3.2 1B
Parameters1.24B
Layers16
Hidden size2,048
MLP width8,192
Attention heads / KV heads32 / 8 (head dim 64)
Vocabulary128,256 tokens, tied embeddings
Context128K tokens
LicenseLlama 3.2 Community License
ReleasedSeptember 2024

Try other settings

Pre-filled for Llama 3.2 1B. Change the sequence length, batch, LoRA rank, context or GPU.

JobMemoryRTX 4090 (24 GB)Smallest single GPU
QLoRA (4-bit base)2K tokens × batch 1, rank 163.7 GBFitsRTX 3090 (24 GB)
LoRA (16-bit base)2K tokens × batch 1, rank 165.1 GBFitsRTX 3090 (24 GB)
Full fine-tuning2K tokens × batch 122 GB2 GPUsRTX 5090 (32 GB)
Inference, 16-bit (BF16/FP16)8K-token context × 1 sequence2.7 GBFitsRTX 3090 (24 GB)
Inference, 8-bit8K-token context × 1 sequence1.5 GBFitsRTX 3090 (24 GB)
Inference, 4-bit8K-token context × 1 sequence1.0 GBFitsRTX 3090 (24 GB)

A job fits when the estimate is within 90% of the card's memory. Training assumes gradient checkpointing and FlashAttention; multi-GPU counts assume the memory splits evenly (FSDP, ZeRO-3 or tensor parallelism).

How we estimate it

Every figure comes from the model's published architecture (layers, hidden size, attention and KV heads, vocabulary) rather than a rule of thumb, using the same functions as the calculator above. Sizes are in GB (10⁹ bytes).

  • Full fine-tuning: 16 bytes per parameter for mixed-precision Adam (bf16 weights and gradients, fp32 master weights and two moments), as in the ZeRO paper.
  • LoRA: frozen bf16 weights (2 bytes per parameter) plus 16 bytes per adapter parameter, rank 16 on every linear layer. For mixture-of-experts models, adapters go on attention only and the experts stay frozen.
  • QLoRA: NF4 weights with double quantisation (4.127 bits per parameter); embeddings and the output head stay in bf16, as bitsandbytes leaves them.
  • Activations: gradient checkpointing and FlashAttention. Each layer's bf16 input is kept, one layer is recomputed at a time, and the output logits take 8 bytes per vocabulary entry per token (fp32 values and gradients). Chunked cross-entropy, as in Unsloth or Liger kernels, cuts that last term.
  • Inference: 2, 1 or about 0.56 bytes per weight (4-bit with group-wise scales), plus an fp16 KV cache of 2 × layers × KV heads × head dim × 2 bytes per token; sliding-window layers only cache their window.
  • Fits means the estimate is within 90% of the card's listed memory; the rest covers the CUDA context, kernels and fragmentation. Multi-GPU counts assume memory splits evenly.

Frequently asked questions

Can I fine-tune Llama 3.2 1B on a 24 GB GPU?
On an RTX 3090 or 4090 (24 GB): QLoRA: yes, about 3.7 GB at 2K tokens, batch 1, rank 16; up to 16K-token sequences at batch 1 or batch 8 at 2K, LoRA rank up to 1024. LoRA: yes, about 5.1 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 7 at 2K, LoRA rank up to 1024. Estimates use gradient checkpointing and leave 10% of memory free.
How much VRAM do you need to run Llama 3.2 1B?
About 2.7 GB in 16-bit and 1.0 GB in 4-bit, at 8K tokens of context. The smallest single GPU here for 4-bit is the RTX 3090 (24 GB); for 16-bit, the RTX 3090 (24 GB).
How many GPUs does full fine-tuning Llama 3.2 1B take?
About 22 GB with mixed-precision Adam at 2K tokens, so at least 1 × 80 GB H100 with the training state sharded (FSDP or ZeRO-3). LoRA needs 5.1 GB and QLoRA 3.7 GB.
How long a context can Llama 3.2 1B handle on an 80 GB GPU?
Its KV cache costs 4.3 GB at the full 128K tokens (fp16 cache). On one 80 GB GPU: 128K tokens with 16-bit weights, 128K tokens with 4-bit weights.

Terms used on this page: LoRA, QLoRA, KV cache, gradient checkpointing, quantization. For any model size, use the fine-tuning VRAM calculator; for the method in depth, the Academy lesson Memory first: what hardware can train what.

Sources

  1. [1]Hugging Face. meta-llama/Llama-3.2-1B-Instruct config.json
  2. [2]Meta. meta-llama/Llama-3.2-1B-Instruct model card
  3. [3]Meta. The Llama 3 Herd of Models (arXiv:2407.21783)
  4. [4]Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)
  5. [5]Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)
  6. [6]Rajbhandari et al., 2019. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054)
  7. [7]Korthikanti et al., 2022. Reducing Activation Recomputation in Large Transformer Models (arXiv:2205.05198)
  8. [8]Chen et al., 2016. Training Deep Nets with Sublinear Memory Cost (arXiv:1604.06174)

Architecture read from config.json and parameter count from the Hugging Face safetensors metadata on 15 November 2026.

Find out if you need to fine-tune at all.

Kojen's prompt tuning and model search run for real on your own test cases, on hosted models. Measure the cheaper levers before you rent a GPU.

Start training free