Skip to content

VRAM reference

Llama 3.3 70B VRAM requirements

Fine-tuning Llama 3.3 70B takes about 48 GB of GPU memory with QLoRA, 150 GB with LoRA on 16-bit weights and 1,134 GB for full fine-tuning, at 2,048 tokens per sequence. Running it takes 144 GB in 16-bit or 42 GB in 4-bit with a 8K-token context.

By Will Schulz · Architecture and prices checked 15 November 2026 · Prices as of November 2026

Fine-tuning memory

At 2,048 tokens per sequence, batch 1, LoRA rank 16, gradient checkpointing on
MethodWeightsTraining stateActivationsTotalSmallest single GPU80 GB GPUs needed
QLoRA (4-bit base)40 GB3.3 GB5.6 GB48 GBA100 80GB (80 GB)1
LoRA (16-bit base)141 GB3.3 GB5.6 GB150 GBB200 (180 GB)3
Full fine-tuning141 GB988 GB5.6 GB1,134 GBNone16

On a 24 GB card such as the RTX 4090: QLoRA: no, it needs about 48 GB at 2K tokens, more than the 24 GB available.

Inference memory

One sequence, fp16 KV cache
PrecisionWeightsKV cache at 8KTotalAt 128K contextSmallest single GPU
16-bit (BF16/FP16)141 GB2.7 GB144 GB184 GBB200 (180 GB)
8-bit71 GB2.7 GB73 GB114 GBH200 (141 GB)
4-bit40 GB2.7 GB42 GB83 GBL40S (48 GB)

Which GPUs fit Llama 3.3 70B

One GPU unless a count is shown. Follow a GPU's name for its maximum sequence length, batch size, LoRA rank and rental cost.

GPUQLoRALoRAFull fine-tuningRun, fp16Run, int8Run, int4
RTX 3090 24 GB3×7×53×7×4×2×
RTX 4090 24 GB3×7×53×7×4×2×
RTX 5090 32 GB2×6×40×5×3×2×
L40S 48 GB2×4×27×4×2×Yes
A100 40GB 40 GB2×5×32×4×3×2×
A100 80GB 80 GBYes3×16×2×2×Yes
H100 SXM 80 GBYes3×16×2×2×Yes
H200 141 GBYes2×9×2×YesYes
B200 180 GBYesYes8×YesYesYes

What's specific to Llama 3.3 70B

  • Same architecture as Llama 3.1 70B, so the memory figures are identical; only the post-training differs.
  • QLoRA fits on one 48 GB card at short sequences, the setup the QLoRA paper made famous for 65B models.
ArchitectureLlama 3.3 70B
Parameters70.6B
Layers80
Hidden size8,192
MLP width28,672
Attention heads / KV heads64 / 8 (head dim 128)
Vocabulary128,256 tokens
Context128K tokens
LicenseLlama 3.3 Community License
ReleasedDecember 2024

Try other settings

Pre-filled for Llama 3.3 70B. Change the sequence length, batch, LoRA rank, context or GPU.

JobMemoryRTX 4090 (24 GB)Smallest single GPU
QLoRA (4-bit base)2K tokens × batch 1, rank 1648 GB3 GPUsA100 80GB (80 GB)
LoRA (16-bit base)2K tokens × batch 1, rank 16150 GB7 GPUsB200 (180 GB)
Full fine-tuning2K tokens × batch 11,134 GB53 GPUsNone: needs several
Inference, 16-bit (BF16/FP16)8K-token context × 1 sequence144 GB7 GPUsB200 (180 GB)
Inference, 8-bit8K-token context × 1 sequence73 GB4 GPUsH200 (141 GB)
Inference, 4-bit8K-token context × 1 sequence42 GB2 GPUsL40S (48 GB)

A job fits when the estimate is within 90% of the card's memory. Training assumes gradient checkpointing and FlashAttention; multi-GPU counts assume the memory splits evenly (FSDP, ZeRO-3 or tensor parallelism).

How we estimate it

Every figure comes from the model's published architecture (layers, hidden size, attention and KV heads, vocabulary) rather than a rule of thumb, using the same functions as the calculator above. Sizes are in GB (10⁹ bytes).

  • Full fine-tuning: 16 bytes per parameter for mixed-precision Adam (bf16 weights and gradients, fp32 master weights and two moments), as in the ZeRO paper.
  • LoRA: frozen bf16 weights (2 bytes per parameter) plus 16 bytes per adapter parameter, rank 16 on every linear layer. For mixture-of-experts models, adapters go on attention only and the experts stay frozen.
  • QLoRA: NF4 weights with double quantisation (4.127 bits per parameter); embeddings and the output head stay in bf16, as bitsandbytes leaves them.
  • Activations: gradient checkpointing and FlashAttention. Each layer's bf16 input is kept, one layer is recomputed at a time, and the output logits take 8 bytes per vocabulary entry per token (fp32 values and gradients). Chunked cross-entropy, as in Unsloth or Liger kernels, cuts that last term.
  • Inference: 2, 1 or about 0.56 bytes per weight (4-bit with group-wise scales), plus an fp16 KV cache of 2 × layers × KV heads × head dim × 2 bytes per token; sliding-window layers only cache their window.
  • Fits means the estimate is within 90% of the card's listed memory; the rest covers the CUDA context, kernels and fragmentation. Multi-GPU counts assume memory splits evenly.

Frequently asked questions

Can I fine-tune Llama 3.3 70B on a 24 GB GPU?
On an RTX 3090 or 4090 (24 GB): QLoRA: no, it needs about 48 GB at 2K tokens, more than the 24 GB available. LoRA: no, it needs about 150 GB at 2K tokens, more than the 24 GB available. Estimates use gradient checkpointing and leave 10% of memory free.
How much VRAM do you need to run Llama 3.3 70B?
About 144 GB in 16-bit and 42 GB in 4-bit, at 8K tokens of context. The smallest single GPU here for 4-bit is the L40S (48 GB); for 16-bit, the B200 (180 GB).
How many GPUs does full fine-tuning Llama 3.3 70B take?
About 1,134 GB with mixed-precision Adam at 2K tokens, so at least 16 × 80 GB H100s with the training state sharded (FSDP or ZeRO-3). LoRA needs 150 GB and QLoRA 48 GB.
How long a context can Llama 3.3 70B handle on an 80 GB GPU?
Its KV cache costs 43 GB at the full 128K tokens (fp16 cache). On one 80 GB GPU: 16-bit weights don't fit, 64K tokens with 4-bit weights.

Terms used on this page: LoRA, QLoRA, KV cache, gradient checkpointing, quantization. For any model size, use the fine-tuning VRAM calculator; for the method in depth, the Academy lesson Memory first: what hardware can train what.

Sources

  1. [1]Hugging Face. meta-llama/Llama-3.3-70B-Instruct config.json
  2. [2]Meta. meta-llama/Llama-3.3-70B-Instruct model card
  3. [3]Meta. The Llama 3 Herd of Models (arXiv:2407.21783)
  4. [4]Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)
  5. [5]Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)
  6. [6]Rajbhandari et al., 2019. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054)
  7. [7]Korthikanti et al., 2022. Reducing Activation Recomputation in Large Transformer Models (arXiv:2205.05198)
  8. [8]Chen et al., 2016. Training Deep Nets with Sublinear Memory Cost (arXiv:1604.06174)

Architecture read from config.json and parameter count from the Hugging Face safetensors metadata on 15 November 2026.

Find out if you need to fine-tune at all.

Kojen's prompt tuning and model search run for real on your own test cases, on hosted models. Measure the cheaper levers before you rent a GPU.

Start training free