Llama 3.2 3B on an RTX 3090
Yes, you can fine-tune Llama 3.2 3B on one RTX 3090 (24 GB) with QLoRA or LoRA. 4-bit inference: yes, 1.8 GB of weights, room for 128K tokens of context or 42 concurrent 4K-token sequences.
Verdict by method
| Fine-tuning | Memory at 2K tokens | One RTX 3090? | Longest sequence (batch 1) | Largest batch at 2K | Largest LoRA rank |
|---|---|---|---|---|---|
| QLoRA (4-bit base) | 5.3 GB | Yes | 8K tokens | 7 | 512 |
| LoRA (16-bit base) | 9.5 GB | Yes | 8K tokens | 5 | 512 |
| Full fine-tuning | 54 GB | No | — | — | n/a |
| Inference | Memory at 8K | One RTX 3090? | Longest context | Concurrent 4K sequences |
|---|---|---|---|---|
| 16-bit (BF16/FP16) | 7.4 GB | Yes | 128K tokens | 32 |
| 8-bit | 4.2 GB | Yes | 128K tokens | 39 |
| 4-bit | 2.7 GB | Yes | 128K tokens | 42 |
- QLoRA: yes, about 5.3 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 7 at 2K, LoRA rank up to 512.
- LoRA: yes, about 9.5 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 5 at 2K, LoRA rank up to 512.
- Full fine-tuning: no, it needs about 54 GB at 2K tokens, more than the 24 GB available.
Where the memory goes
| Method | Weights | Training state | Activations | Total | On one RTX 3090 | 80 GB GPUs needed |
|---|---|---|---|---|---|---|
| QLoRA (4-bit base) | 2.2 GB | 0.4 GB | 2.7 GB | 5.3 GB | Fits | 1 |
| LoRA (16-bit base) | 6.4 GB | 0.4 GB | 2.7 GB | 9.5 GB | Fits | 1 |
| Full fine-tuning | 6.4 GB | 45 GB | 2.7 GB | 54 GB | Doesn't fit | 1 |
| Precision | Weights | KV cache at 8K | Total | At 128K context | On one RTX 3090 |
|---|---|---|---|---|---|
| 16-bit (BF16/FP16) | 6.4 GB | 0.9 GB | 7.4 GB | 21 GB | Fits |
| 8-bit | 3.2 GB | 0.9 GB | 4.2 GB | 18 GB | Fits |
| 4-bit | 1.8 GB | 0.9 GB | 2.7 GB | 17 GB | Fits |
The cheapest way to run it
Renting an RTX 3090 costs $0.50 an hour on Runpod Secure Cloud ($0.22 on Runpod Community Cloud). LoRA on it processes about 2,216 tokens a second by our estimate, so 10 million training tokens take 1.3 hours and cost about $0.63.
| Job (default settings) | Memory | Cheapest rental that fits | Per hour |
|---|---|---|---|
| QLoRA fine-tuning | 5.3 GB | RTX 3090 (this GPU) | $0.50 |
| LoRA fine-tuning | 9.5 GB | RTX 3090 (this GPU) | $0.50 |
| 4-bit inference | 2.7 GB | RTX 3090 (this GPU) | $0.50 |
| 16-bit inference | 7.4 GB | RTX 3090 (this GPU) | $0.50 |
On-demand rates seen on 15 November 2026: Runpod Secure Cloud, Lambda for the A100 40GB. Consumer cards count as single GPUs only; data-centre GPUs up to eight per node.
Try other settings
| Job | Memory | RTX 3090 (24 GB) | Smallest single GPU |
|---|---|---|---|
| QLoRA (4-bit base)2K tokens × batch 1, rank 16 | 5.3 GB | Fits | RTX 3090 (24 GB) |
| LoRA (16-bit base)2K tokens × batch 1, rank 16 | 9.5 GB | Fits | RTX 3090 (24 GB) |
| Full fine-tuning2K tokens × batch 1 | 54 GB | 3 GPUs | A100 80GB (80 GB) |
| Inference, 16-bit (BF16/FP16)8K-token context × 1 sequence | 7.4 GB | Fits | RTX 3090 (24 GB) |
| Inference, 8-bit8K-token context × 1 sequence | 4.2 GB | Fits | RTX 3090 (24 GB) |
| Inference, 4-bit8K-token context × 1 sequence | 2.7 GB | Fits | RTX 3090 (24 GB) |
A job fits when the estimate is within 90% of the card's memory. Training assumes gradient checkpointing and FlashAttention; multi-GPU counts assume the memory splits evenly (FSDP, ZeRO-3 or tensor parallelism).
How we estimate it
Every figure comes from the model's published architecture (layers, hidden size, attention and KV heads, vocabulary) rather than a rule of thumb, using the same functions as the calculator above. Sizes are in GB (10⁹ bytes).
- Full fine-tuning: 16 bytes per parameter for mixed-precision Adam (bf16 weights and gradients, fp32 master weights and two moments), as in the ZeRO paper.
- LoRA: frozen bf16 weights (2 bytes per parameter) plus 16 bytes per adapter parameter, rank 16 on every linear layer. For mixture-of-experts models, adapters go on attention only and the experts stay frozen.
- QLoRA: NF4 weights with double quantisation (4.127 bits per parameter); embeddings and the output head stay in bf16, as bitsandbytes leaves them.
- Activations: gradient checkpointing and FlashAttention. Each layer's bf16 input is kept, one layer is recomputed at a time, and the output logits take 8 bytes per vocabulary entry per token (fp32 values and gradients). Chunked cross-entropy, as in Unsloth or Liger kernels, cuts that last term.
- Inference: 2, 1 or about 0.56 bytes per weight (4-bit with group-wise scales), plus an fp16 KV cache of 2 × layers × KV heads × head dim × 2 bytes per token; sliding-window layers only cache their window.
- Fits means the estimate is within 90% of the card's listed memory; the rest covers the CUDA context, kernels and fragmentation. Multi-GPU counts assume memory splits evenly.
Speed estimates use RTX 3090's dense BF16 peak of 71.2 TFLOPS, about 4 FLOPs per active parameter per token for LoRA and 40% utilisation; see fine-tuning and training costs for the method.
Frequently asked questions
- Can I fine-tune Llama 3.2 3B on an RTX 3090?
- QLoRA: yes, about 5.3 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 7 at 2K, LoRA rank up to 512. LoRA: yes, about 9.5 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 5 at 2K, LoRA rank up to 512. Full fine-tuning: no, it needs about 54 GB at 2K tokens, more than the 24 GB available.
- Can I run Llama 3.2 3B on an RTX 3090?
- 16-bit (BF16/FP16) inference: yes, 6.4 GB of weights, room for 128K tokens of context or 32 concurrent 4K-token sequences. 8-bit inference: yes, 3.2 GB of weights, room for 128K tokens of context or 39 concurrent 4K-token sequences. 4-bit inference: yes, 1.8 GB of weights, room for 128K tokens of context or 42 concurrent 4K-token sequences.
- What does fine-tuning Llama 3.2 3B on an RTX 3090 cost?
- With LoRA at our planning rate of 40% utilisation, about 1.3 hours for 10 million training tokens, around $0.63 at $0.50 an hour (Runpod Secure Cloud, November 2026).
Related
- Llama 3.2 3B VRAM requirementsEvery GPU and method
- Llama 3.2 3B on RTX 409024 GB
- Fine-tuning Llama 3.2 3B: costPer million tokens
- Llama 3.1 8B on RTX 3090Meta
- Llama 3.2 1B on RTX 3090Meta
- Qwen3 4B on RTX 3090Alibaba Qwen
- Qwen3 8B on RTX 3090Alibaba Qwen
- Qwen3 14B on RTX 3090Alibaba Qwen
- Mistral 7B on RTX 3090Mistral AI
Sources
- Hugging Face. meta-llama/Llama-3.2-3B-Instruct config.json
- Meta. meta-llama/Llama-3.2-3B-Instruct model card
- Meta. The Llama 3 Herd of Models (arXiv:2407.21783)
- NVIDIA. RTX 3090 specifications
- Runpod. GPU pricing (updated 12 November 2026)
- Lambda. GPU cloud pricing
- Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)
- Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)
- Rajbhandari et al., 2019. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054)
- Korthikanti et al., 2022. Reducing Activation Recomputation in Large Transformer Models (arXiv:2205.05198)
Measure before you rent the GPU.
Prompt tuning and model search in Kojen run on hosted models against your own test cases, so you know whether fine-tuning Llama 3.2 3B is worth the GPU time.
Start training free