Skip to content

Training

What it costs to fine-tune an 8B model in 2026

The arithmetic from training tokens to GPU-hours to dollars for a LoRA fine-tune of Llama 3.1 8B or Qwen3-8B, with Modal, Runpod and Lambda prices as of November 2026, and the costs the GPU bill leaves out.

By Will Schulz8 min read
A dark metal GPU card with a stack of glowing red glass coins on top and a glass stopwatch beside it

Fine-tuning an 8B model like Llama 3.1 8B or Qwen3-8B with LoRA costs a few dollars of GPU time for a typical dataset. That's the short answer, and it surprises people who have read about seven-figure training runs. The longer answer is that the GPU time is usually the smallest line in the budget.

Here is the arithmetic, with GPU prices as listed in mid-November 2026, so you can work out your own number before renting anything.

The method

1. tokens   = examples × average tokens per example (after padding) × epochs
2. FLOPs    ≈ 6 × parameters × tokens
3. seconds  = FLOPs ÷ (GPU peak FLOPS × utilisation)
4. hours    = seconds ÷ 3,600 + fixed overhead
5. dollars  = hours × hourly price

The 6 × parameters × tokens rule comes from the scaling-law literature: about two FLOPs per parameter per token for the forward pass and four for the backward pass.[1] LoRA changes the split but not the total much. It skips the gradients for the frozen weights (about a third of the work), but gradient checkpointing, which nearly every LoRA recipe enables to save memory, recomputes the forward pass. Those roughly cancel, so 6 × N × tokens remains a reasonable estimate.

Utilisation is the big unknown. Large pretraining runs reach 38–43% of peak on H100s;[2] small fine-tuning jobs with short, padded sequences often do worse. We use 35% and add a fixed 15 minutes per run for starting the machine, downloading about 16 GB of weights, evaluating and saving. Both are our assumptions.

A worked example

An A100's dense BF16 peak is 312 TFLOPS;[3] an H100 SXM's is about 989.[4] Five thousand examples of 500 tokens, trained for two epochs:

Qwen3-8B (8.2B parameters), 5,000 examples × 500 tokens × 2 epochs

tokens   = 5,000 × 500 × 2                    = 5 million
FLOPs    = 6 × 8.2e9 × 5e6                    = 2.46e17
A100     = 312 TFLOPS × 35%                   = 109 TFLOPS
seconds  = 2.46e17 ÷ 1.09e14                  = 2,250 s ≈ 38 minutes
hours    = 0.63 + 0.25 overhead               ≈ 0.88 hours
dollars  = 0.88 × $2.50 (Modal A100 80GB)     ≈ $2.19

Prices as of November 2026

ProviderGPUOn-demand price per hour
Modal[5]A100 80GB / H100$2.50 / $3.95 (billed per second)
Runpod, secure cloud[6]A100 SXM 80GB / H100 SXM$2.19 / $4.39
Runpod, community cloud[6]A100 SXM 80GB / H100 SXM$1.39 / $2.69
Lambda, from 16 November[7]1 × H100 SXM$4.89

Runpod's page was updated on 12 November 2026; Lambda's lists rates taking effect on 16 November. Vast.ai runs a marketplace whose prices move with supply and demand, often below these, so check its live listings.[8]

Cost by dataset size

Our estimates using the method above, at 500 tokens per example and two epochs, including the 15-minute overhead:

ExamplesTokensA100 timeModal A100Runpod A100 (secure)Runpod H100 (secure)
1,0001M≈ 8 min$0.94$0.82$1.27
5,0005M≈ 38 min$2.19$1.92$1.96
20,00020M≈ 2.5 h$6.88$6.03$4.56

Two things stand out. For small jobs the fixed overhead dominates, so a cheaper, slower GPU wins. For larger jobs the H100's roughly three times higher throughput beats its higher hourly price. And at every size the answer is single-digit dollars.

At larger scale

With 100 million training tokens, a large instruction-tuning set, the comparison between LoRA and full fine-tuning becomes interesting:

100 million training tokens (e.g. 50,000 long examples × 1,000 tokens × 2 epochs)
FLOPs              = 6 × 8.2e9 × 1e8          = 4.9e18

LoRA, 1 × A100     4.9e18 ÷ (312e12 × 35%)    ≈ 12.5 h  × $2.50 ≈ $31
LoRA, 1 × H100     4.9e18 ÷ (989e12 × 35%)    ≈ 3.9 h   × $4.39 ≈ $17
Full, 8 × H100     4.9e18 ÷ (8 × 989e12 × 40%) ≈ 0.43 h × 8 × $4.39 ≈ $15

Full fine-tuning of an 8B model needs about 131 GB for weights, gradients and Adam state alone, so it needs several GPUs. But the FLOPs are similar, so the compute bill is too. What full fine-tuning really costs is access to a multi-GPU node, 16 GB checkpoints for every version, and a dedicated copy of the model to serve. The memory arithmetic is in LoRA vs QLoRA vs full fine-tuning.

What the GPU estimate leaves out

  • Several runs, not one. Expect three to five runs while you settle the learning rate, epochs and data. Multiply accordingly.
  • Padding and long examples. Count tokens after the chat template is applied and sequences are padded. A dataset that averages 500 tokens but pads to 2,048 costs four times as much.
  • Data. Writing or reviewing a few thousand good examples takes days of skilled time. It's almost always the biggest cost.
  • Evaluation. Each comparison runs the base and tuned model on a test set and grades every answer, often with an LLM judge.
  • Serving. A fine-tuned model needs somewhere to run. Shared serving platforms charge per token; a dedicated GPU costs the same whether it's busy or idle:
One GPU running around the clock: 730 hours a month
Runpod L40S, community cloud   730 × $0.79 = $577 a month
Runpod L40S, secure cloud      730 × $1.09 = $796 a month
Runpod A100 SXM, secure cloud  730 × $2.19 = $1,599 a month

A month of one dedicated serving GPU costs hundreds of times more than the training run. Serving many LoRA adapters on one base model is the standard way to cut that; see Serve a LoRA adapter behind an OpenAI-compatible endpoint.

For pretraining costs rather than fine-tuning, see What it really costs to train an LLM and the LLM training cost calculator.

Frequently asked questions

How much does it cost to fine-tune Llama 3.1 8B or Qwen3-8B?
With LoRA, a few dollars of GPU time for a typical dataset. By our estimate, 5,000 examples of 500 tokens for two epochs takes about 38 minutes on an A100 and costs about $2 at $2.50 an hour. Data preparation, repeated runs, evaluation and serving usually cost far more.
How do you estimate fine-tuning cost?
Multiply examples × tokens per example × epochs to get training tokens, multiply by 6 × parameters for FLOPs, divide by the GPU's peak FLOPS times a utilisation of roughly 30–40%, add fixed startup time, and multiply by the hourly price.
Is an H100 or A100 cheaper for fine-tuning?
For small jobs, the A100, because fixed startup time dominates. For larger jobs the H100's roughly three times higher throughput outweighs its higher price: at 20,000 examples we estimate about $4.56 on a Runpod H100 versus about $6 on an A100.
Is full fine-tuning much more expensive than LoRA?
In FLOPs, not much: both are roughly 6 × parameters × tokens. Full fine-tuning of an 8B model needs about 131 GB for weights, gradients and optimiser state, so it needs a multi-GPU node, larger checkpoints and a dedicated copy of the model to serve.
What does it cost to serve a fine-tuned 8B model?
On a dedicated GPU, it's the hourly price times 730 hours a month: about $577 for a Runpod community L40S and $1,599 for a secure-cloud A100 at November 2026 prices. Serving several LoRA adapters on one base model spreads that cost.

Go deeper in the Academy

Free lessons with quizzes, citations and a hands-on exercise.

Sources

  1. [1]Scaling Laws for Neural Language Models. Kaplan et al., OpenAI, arXiv (2020).
  2. [2]The Llama 3 Herd of Models. Llama Team, Meta, arXiv (2024).
  3. [3]NVIDIA A100 Tensor Core GPU. NVIDIA.
  4. [4]NVIDIA H100 Tensor Core GPU. NVIDIA.
  5. [5]Pricing (checked 15 November 2026). Modal.
  6. [6]GPU pricing (updated 12 November 2026). Runpod.
  7. [7]AI cloud pricing (rates effective 16 November 2026). Lambda.
  8. [8]Pricing. Vast.ai.

Put it into practice.

Kojen gives you the Forge, LLM-judged evaluations, red-team simulations and live endpoints in one place. Free to start, no card needed, then pay as you go.

Start training free

More reading

All articles →