Fine-tuning an 8B model like Llama 3.1 8B or Qwen3-8B with LoRA costs a few dollars of GPU time for a typical dataset. That's the short answer, and it surprises people who have read about seven-figure training runs. The longer answer is that the GPU time is usually the smallest line in the budget.
Here is the arithmetic, with GPU prices as listed in mid-November 2026, so you can work out your own number before renting anything.
The method
1. tokens = examples × average tokens per example (after padding) × epochs 2. FLOPs ≈ 6 × parameters × tokens 3. seconds = FLOPs ÷ (GPU peak FLOPS × utilisation) 4. hours = seconds ÷ 3,600 + fixed overhead 5. dollars = hours × hourly price
The 6 × parameters × tokens rule comes from the scaling-law literature: about two FLOPs per parameter per token for the forward pass and four for the backward pass.[1] LoRA changes the split but not the total much. It skips the gradients for the frozen weights (about a third of the work), but gradient checkpointing, which nearly every LoRA recipe enables to save memory, recomputes the forward pass. Those roughly cancel, so 6 × N × tokens remains a reasonable estimate.
Utilisation is the big unknown. Large pretraining runs reach 38–43% of peak on H100s;[2] small fine-tuning jobs with short, padded sequences often do worse. We use 35% and add a fixed 15 minutes per run for starting the machine, downloading about 16 GB of weights, evaluating and saving. Both are our assumptions.
A worked example
An A100's dense BF16 peak is 312 TFLOPS;[3] an H100 SXM's is about 989.[4] Five thousand examples of 500 tokens, trained for two epochs:
Qwen3-8B (8.2B parameters), 5,000 examples × 500 tokens × 2 epochs tokens = 5,000 × 500 × 2 = 5 million FLOPs = 6 × 8.2e9 × 5e6 = 2.46e17 A100 = 312 TFLOPS × 35% = 109 TFLOPS seconds = 2.46e17 ÷ 1.09e14 = 2,250 s ≈ 38 minutes hours = 0.63 + 0.25 overhead ≈ 0.88 hours dollars = 0.88 × $2.50 (Modal A100 80GB) ≈ $2.19
Prices as of November 2026
| Provider | GPU | On-demand price per hour |
|---|---|---|
| Modal[5] | A100 80GB / H100 | $2.50 / $3.95 (billed per second) |
| Runpod, secure cloud[6] | A100 SXM 80GB / H100 SXM | $2.19 / $4.39 |
| Runpod, community cloud[6] | A100 SXM 80GB / H100 SXM | $1.39 / $2.69 |
| Lambda, from 16 November[7] | 1 × H100 SXM | $4.89 |
Runpod's page was updated on 12 November 2026; Lambda's lists rates taking effect on 16 November. Vast.ai runs a marketplace whose prices move with supply and demand, often below these, so check its live listings.[8]
Cost by dataset size
Our estimates using the method above, at 500 tokens per example and two epochs, including the 15-minute overhead:
| Examples | Tokens | A100 time | Modal A100 | Runpod A100 (secure) | Runpod H100 (secure) |
|---|---|---|---|---|---|
| 1,000 | 1M | ≈ 8 min | $0.94 | $0.82 | $1.27 |
| 5,000 | 5M | ≈ 38 min | $2.19 | $1.92 | $1.96 |
| 20,000 | 20M | ≈ 2.5 h | $6.88 | $6.03 | $4.56 |
Two things stand out. For small jobs the fixed overhead dominates, so a cheaper, slower GPU wins. For larger jobs the H100's roughly three times higher throughput beats its higher hourly price. And at every size the answer is single-digit dollars.
At larger scale
With 100 million training tokens, a large instruction-tuning set, the comparison between LoRA and full fine-tuning becomes interesting:
100 million training tokens (e.g. 50,000 long examples × 1,000 tokens × 2 epochs) FLOPs = 6 × 8.2e9 × 1e8 = 4.9e18 LoRA, 1 × A100 4.9e18 ÷ (312e12 × 35%) ≈ 12.5 h × $2.50 ≈ $31 LoRA, 1 × H100 4.9e18 ÷ (989e12 × 35%) ≈ 3.9 h × $4.39 ≈ $17 Full, 8 × H100 4.9e18 ÷ (8 × 989e12 × 40%) ≈ 0.43 h × 8 × $4.39 ≈ $15
Full fine-tuning of an 8B model needs about 131 GB for weights, gradients and Adam state alone, so it needs several GPUs. But the FLOPs are similar, so the compute bill is too. What full fine-tuning really costs is access to a multi-GPU node, 16 GB checkpoints for every version, and a dedicated copy of the model to serve. The memory arithmetic is in LoRA vs QLoRA vs full fine-tuning.
What the GPU estimate leaves out
- Several runs, not one. Expect three to five runs while you settle the learning rate, epochs and data. Multiply accordingly.
- Padding and long examples. Count tokens after the chat template is applied and sequences are padded. A dataset that averages 500 tokens but pads to 2,048 costs four times as much.
- Data. Writing or reviewing a few thousand good examples takes days of skilled time. It's almost always the biggest cost.
- Evaluation. Each comparison runs the base and tuned model on a test set and grades every answer, often with an LLM judge.
- Serving. A fine-tuned model needs somewhere to run. Shared serving platforms charge per token; a dedicated GPU costs the same whether it's busy or idle:
One GPU running around the clock: 730 hours a month Runpod L40S, community cloud 730 × $0.79 = $577 a month Runpod L40S, secure cloud 730 × $1.09 = $796 a month Runpod A100 SXM, secure cloud 730 × $2.19 = $1,599 a month
A month of one dedicated serving GPU costs hundreds of times more than the training run. Serving many LoRA adapters on one base model is the standard way to cut that; see Serve a LoRA adapter behind an OpenAI-compatible endpoint.
For pretraining costs rather than fine-tuning, see What it really costs to train an LLM and the LLM training cost calculator.



