All articles

Training

Prompt tuning or fine-tuning? Pull the cheaper lever first

Instructions, model choice and weights are three levers with very different costs. What the research says about each, and how to decide when fine-tuning is actually worth it.

By Will Schulz8 min read
A glass brush beside a red stroke on the left, a glowing ingot on an anvil on the right

When an agent underperforms, the instinct is to reach for the heaviest tool: “we need to fine-tune.” Sometimes that's right. More often, the cheaper levers haven't been pulled yet, and nobody can tell whether fine-tuning helped because there was no measurement to begin with.

OpenAI's own model-optimisation guidance frames it as a combination of three things, not a choice of one: evals, prompt engineering, and fine-tuning.[1] Here's how we think about the order you use them in, and what Kojen does at each step.

The three levers, cheapest first

  1. Change the instructions. Rewrite the system prompt, add examples, tighten the output format. This costs minutes, and you can roll it back instantly.
  2. Change the model. Swap the base model for one that's better suited, faster or cheaper for this task. This still needs no training.
  3. Change the weights. Fine-tune with LoRA, a full fine-tune or preference training. This is the most expensive option and the hardest to undo.

All three rest on the same thing, an evaluation you trust. Without one, “the new prompt feels better” is just an opinion.

Prompt tuning is optimisation, not guesswork

Hand-editing prompts is slow and biased toward whatever example you looked at last. The research community has shown you can treat the prompt as something to optimise automatically. In OPRO, Google DeepMind researchers used an LLM as the optimiser: each step proposes new instructions based on earlier ones and their scores. The best optimised prompts beat human-designed ones by up to 8% on GSM8K and up to 50% on Big-Bench Hard tasks.[2]

Stanford's DSPy goes further, compiling a pipeline of LM calls against a metric. Within minutes of compiling, DSPy programs outperformed standard few-shot prompting by over 25% with GPT-3.5 and by over 65% with Llama 2 13B Chat.[3]

How Kojen's prompt tuning works

In the Forge, a prompt-tuning run does that loop for you:

  • It reads your project's task description and your dataset's input/expected rows. With no dataset, it generates test cases from the task.
  • It holds out 30% of the cases for a final check the optimiser never sees.
  • It proposes system prompts over several rounds, scoring each one on the training cases with an LLM judge.
  • It scores the winner on the held-out cases and saves it as the project's active model version.

Model search: the lever people skip

Teams often lock in a base model on day one and never revisit it. But the right model for a narrow classification task is rarely the right one for multi-step tool use. Kojen's model search run scores your task on 3–5 models from the catalogue (Llama, Mistral, Qwen, Gemma, DeepSeek, GPT-4.1 mini, Gemini 2.5 Flash, Claude Haiku and others) and picks the most accurate, using latency to break ties. A smaller model that matches a larger one on your test set is cheaper on every call from then on.

When changing the weights is worth it

Fine-tuning earns its cost when prompting has plateaued and you need something the instructions can't buy:

  • Lower latency and cost at scale. Teach a small model what a big one does with a long prompt.
  • Strict, repeatable formats that the base model keeps drifting from.
  • Behaviour the base model resists, such as a house style, domain jargon or refusal patterns.

Parameter-efficient methods have made this much more practical. LoRA freezes the pre-trained weights and trains small low-rank matrices instead. Compared with fully fine-tuning GPT-3 175B, it cut trainable parameters by 10,000× and GPU memory by 3×, while matching or beating full fine-tuning quality on the models tested.[4] QLoRA added 4-bit quantisation, enough to fine-tune a 65B-parameter model on a single 48 GB GPU.[5]

Preference training is the other big tool. OpenAI's InstructGPT work showed that a 1.3B model fine-tuned with human feedback produced outputs people preferred over the 175B GPT-3, despite having 100× fewer parameters.[6] Size isn't destiny. Training signal matters.

A decision checklist

  1. Do you have a test set with expected answers? If not, build one first.
  2. Have you run prompt tuning against it? Note the held-out score.
  3. Have you run model search? You may already be on the wrong model.
  4. Is the remaining gap about cost, latency, format or style, rather than missing knowledge? That's when to fine-tune.
  5. Whatever you change, compare versions side by side in Compare before you switch the active one.

Go deeper in the Academy

Free lessons with quizzes, citations and a hands-on exercise.

201 · Prompt Tuning, Fine-Tuning & Reasoning

Automatic prompt optimisation

201 · Prompt Tuning, Fine-Tuning & Reasoning

Fine-tuning efficiently with LoRA and QLoRA

Sources

  1. [1]Model optimization / Fine-tuning guide. OpenAI platform docs.
  2. [2]Large Language Models as Optimizers (OPRO). Yang et al., Google DeepMind, arXiv (2023).
  3. [3]DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. Khattab et al., Stanford, arXiv (2023).
  4. [4]LoRA: Low-Rank Adaptation of Large Language Models. Hu et al., Microsoft, arXiv (2021).
  5. [5]QLoRA: Efficient Finetuning of Quantized LLMs. Dettmers et al., arXiv (2023).
  6. [6]Training language models to follow instructions with human feedback. Ouyang et al., OpenAI, arXiv (2022).

Put it into practice.

Kojen gives you the Forge, LLM-judged evaluations, red-team simulations and live endpoints in one place. Free to start, no card needed, then pay as you go.

Start training free

More reading

All articles →
Blank glass pages falling into a glass funnel, with a small obsidian stone with a red seam beneath its spout

Training

Can you just feed an LLM your documents?

Training on raw text teaches style more reliably than facts. What the research says about teaching models new knowledge, and when retrieval beats training.

Nov 5, 20269 min read
A glass scale with glowing red edges, a small obsidian stone on one pan and a large obsidian block on the other

Training

What it really costs to train an LLM, from $48 to $100 million

The arithmetic behind training budgets, estimated costs by model size, what real runs reported, and what the headline numbers leave out. Prices as of November 2026.

Nov 5, 20268 min read
A faceted obsidian computer tower split by a glowing red seam, with a circuit of red light branching from its top

Training

What LLM can you train on a $10,000 computer?

What a $10,000 machine can train from scratch, what it can fine-tune, and whether renting beats buying at November 2026 GPU prices.

Nov 5, 20267 min read