Prompt Tuning, Fine-Tuning & Reasoning
Lesson 2 of 5Intermediate 8 min

Automatic prompt optimisation

Instead of guessing at prompt edits, let a model propose candidates and keep the ones that score best.

In this lesson you'll

  • Describe how APE and OPRO search for prompts
  • Set up a prompt-tuning run with an honest held-out check
  • Read a run's results critically

Hand-tuning prompts is slow and biased toward what the author happens to try. Two lines of research automate it. Automatic Prompt Engineer (APE) has an LLM generate candidate instructions and keeps the one that scores best on a set of examples; its instructions matched or beat human-written ones on most of the tasks tested [1]. OPRO treats the LLM as an optimiser: each round, a meta-prompt lists earlier prompts with their scores, and the model proposes better ones. OPRO's prompts beat human-designed prompts by up to 8% on GSM8K and up to 50% on Big-Bench Hard tasks [2].

The loop

  1. Start from a baseline prompt and score it.
  2. Propose variations, informed by which examples failed.
  3. Score each variation with a judge or exact match.
  4. Keep the best, repeat for several rounds.
  5. Confirm the winner on held-out examples the loop never saw.

Watch for overfitting

An optimiser will happily find a prompt that games the examples it's scored on. The final held-out check is what tells you whether the gain is real.

In Kojen

Prompt tuning in the Forge runs this loop over several judge-scored rounds, holds out 30% of the test cases for the final check, and saves the result as an active model version of your project.

Key takeaways

  • LLMs can propose and score prompt edits faster than people can.
  • Feed failures back into each round.
  • Trust only the held-out score.

Practise in the dojo

Start a prompt-tuning run, then compare the baseline and final held-out scores on the run page.

Run prompt tuning

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. What does OPRO put in its meta-prompt each round?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Zhou, Y. et al. (2022). Large Language Models Are Human-Level Prompt Engineers. ICLR 2023 · arXiv:2211.01910.
  2. [2]Yang, C. et al. (2023). Large Language Models as Optimizers. ICLR 2024 · arXiv:2309.03409.