Skip to content

VRAM reference

Qwen3 8B on an RTX 4090

Yes, you can fine-tune Qwen3 8B on one RTX 4090 (24 GB) with QLoRA or LoRA. 4-bit inference: yes, 4.6 GB of weights, room for 64K tokens of context or 28 concurrent 4K-token sequences.

By Will Schulz · Checked 15 November 2026 · Prices as of November 2026

Verdict by method

Fine-tuningMemory at 2K tokensOne RTX 4090?Longest sequence (batch 1)Largest batch at 2KLargest LoRA rank
QLoRA (4-bit base)10 GBYes8K tokens4256
LoRA (16-bit base)21 GBYes2K tokens132
Full fine-tuning135 GBNo——n/a
InferenceMemory at 8KOne RTX 4090?Longest contextConcurrent 4K sequences
16-bit (BF16/FP16)18 GBYes32K tokens8
8-bit9.4 GBYes64K tokens22
4-bit5.8 GBYes64K tokens28
  • QLoRA: yes, about 10 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 4 at 2K, LoRA rank up to 256.
  • LoRA: yes, about 21 GB at 2K tokens, batch 1, rank 16; up to 2K-token sequences at batch 1, LoRA rank up to 32.
  • Full fine-tuning: no, it needs about 135 GB at 2K tokens, more than the 24 GB available.

Where the memory goes

At 2,048 tokens per sequence, batch 1, LoRA rank 16, gradient checkpointing on
MethodWeightsTraining stateActivationsTotalOn one RTX 409080 GB GPUs needed
QLoRA (4-bit base)6.1 GB0.7 GB3.4 GB10 GBFits1
LoRA (16-bit base)16 GB0.7 GB3.4 GB21 GBFits1
Full fine-tuning16 GB115 GB3.4 GB135 GBDoesn't fit2
One sequence, fp16 KV cache
PrecisionWeightsKV cache at 8KTotalAt 128K contextOn one RTX 4090
16-bit (BF16/FP16)16 GB1.2 GB18 GB36 GBFits
8-bit8.2 GB1.2 GB9.4 GB28 GBFits
4-bit4.6 GB1.2 GB5.8 GB24 GBFits

The cheapest way to run it

Renting an RTX 4090 costs $0.89 an hour on Runpod Secure Cloud ($0.34 on Runpod Community Cloud). LoRA on it processes about 2,017 tokens a second by our estimate, so 10 million training tokens take 1.4 hours and cost about $1.23.

Job (default settings)MemoryCheapest rental that fitsPer hour
QLoRA fine-tuning10 GBRTX 3090$0.50
LoRA fine-tuning21 GBRTX 3090$0.50
4-bit inference5.8 GBRTX 3090$0.50
16-bit inference18 GBRTX 3090$0.50

On-demand rates seen on 15 November 2026: Runpod Secure Cloud, Lambda for the A100 40GB. Consumer cards count as single GPUs only; data-centre GPUs up to eight per node.

Try other settings

JobMemoryRTX 4090 (24 GB)Smallest single GPU
QLoRA (4-bit base)2K tokens × batch 1, rank 1610 GBFitsRTX 3090 (24 GB)
LoRA (16-bit base)2K tokens × batch 1, rank 1621 GBFitsRTX 3090 (24 GB)
Full fine-tuning2K tokens × batch 1135 GB7 GPUsB200 (180 GB)
Inference, 16-bit (BF16/FP16)8K-token context × 1 sequence18 GBFitsRTX 3090 (24 GB)
Inference, 8-bit8K-token context × 1 sequence9.4 GBFitsRTX 3090 (24 GB)
Inference, 4-bit8K-token context × 1 sequence5.8 GBFitsRTX 3090 (24 GB)

A job fits when the estimate is within 90% of the card's memory. Training assumes gradient checkpointing and FlashAttention; multi-GPU counts assume the memory splits evenly (FSDP, ZeRO-3 or tensor parallelism).

How we estimate it

Every figure comes from the model's published architecture (layers, hidden size, attention and KV heads, vocabulary) rather than a rule of thumb, using the same functions as the calculator above. Sizes are in GB (10⁹ bytes).

  • Full fine-tuning: 16 bytes per parameter for mixed-precision Adam (bf16 weights and gradients, fp32 master weights and two moments), as in the ZeRO paper.
  • LoRA: frozen bf16 weights (2 bytes per parameter) plus 16 bytes per adapter parameter, rank 16 on every linear layer. For mixture-of-experts models, adapters go on attention only and the experts stay frozen.
  • QLoRA: NF4 weights with double quantisation (4.127 bits per parameter); embeddings and the output head stay in bf16, as bitsandbytes leaves them.
  • Activations: gradient checkpointing and FlashAttention. Each layer's bf16 input is kept, one layer is recomputed at a time, and the output logits take 8 bytes per vocabulary entry per token (fp32 values and gradients). Chunked cross-entropy, as in Unsloth or Liger kernels, cuts that last term.
  • Inference: 2, 1 or about 0.56 bytes per weight (4-bit with group-wise scales), plus an fp16 KV cache of 2 × layers × KV heads × head dim × 2 bytes per token; sliding-window layers only cache their window.
  • Fits means the estimate is within 90% of the card's listed memory; the rest covers the CUDA context, kernels and fragmentation. Multi-GPU counts assume memory splits evenly.

Speed estimates use RTX 4090's dense BF16 peak of 165.2 TFLOPS, about 4 FLOPs per active parameter per token for LoRA and 40% utilisation; see fine-tuning and training costs for the method.

Frequently asked questions

Can I fine-tune Qwen3 8B on an RTX 4090?
QLoRA: yes, about 10 GB at 2K tokens, batch 1, rank 16; up to 8K-token sequences at batch 1 or batch 4 at 2K, LoRA rank up to 256. LoRA: yes, about 21 GB at 2K tokens, batch 1, rank 16; up to 2K-token sequences at batch 1, LoRA rank up to 32. Full fine-tuning: no, it needs about 135 GB at 2K tokens, more than the 24 GB available.
Can I run Qwen3 8B on an RTX 4090?
16-bit (BF16/FP16) inference: yes, 16 GB of weights, room for 32K tokens of context or 8 concurrent 4K-token sequences. 8-bit inference: yes, 8.2 GB of weights, room for 64K tokens of context or 22 concurrent 4K-token sequences. 4-bit inference: yes, 4.6 GB of weights, room for 64K tokens of context or 28 concurrent 4K-token sequences.
What does fine-tuning Qwen3 8B on an RTX 4090 cost?
With LoRA at our planning rate of 40% utilisation, about 1.4 hours for 10 million training tokens, around $1.23 at $0.89 an hour (Runpod Secure Cloud, November 2026).

Sources

  1. [1]Hugging Face. Qwen/Qwen3-8B config.json
  2. [2]Alibaba Qwen. Qwen/Qwen3-8B model card
  3. [3]Alibaba Qwen. Qwen3 Technical Report (arXiv:2505.09388)
  4. [4]NVIDIA. RTX 4090 specifications
  5. [5]Runpod. GPU pricing (updated 12 November 2026)
  6. [6]Lambda. GPU cloud pricing
  7. [7]Dettmers et al., 2023. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314)
  8. [8]Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)
  9. [9]Rajbhandari et al., 2019. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054)
  10. [10]Korthikanti et al., 2022. Reducing Activation Recomputation in Large Transformer Models (arXiv:2205.05198)

Skip the setup: Kojen trains Qwen3 8B LoRA.

Kojen trains LoRA adapters on Qwen3 8B from your input/expected rows on the cheapest GPU in stock, billed by the second (for accounts with GPU training enabled).

Start training free