Evaluation & Trust
Lesson 1 of 5Intermediate 8 min

Test sets that tell the truth

An evaluation is only as honest as its test set. Start small, draw from real failures, and keep it away from training.

In this lesson you'll

  • Build a first test set quickly
  • Separate capability tests from regression tests
  • Keep evaluation data out of tuning loops

Teams often delay evaluation because they think they need hundreds of cases. Anthropic's 2026 guidance on agent evals says 20–50 simple tasks drawn from real failures is a great start, because early changes have large, visible effects [1]. Grow the set as the model improves.

Two kinds of suite

  • Capability evals ask "can it do this yet?" and should start with a low pass rate so there's room to climb.
  • Regression evals ask "does it still do everything it used to?" and should pass at close to 100%. Capability tasks that reach a high pass rate graduate into the regression suite [1].

Keep it clean

Any example the model was tuned on stops measuring generalisation. Hold the test split back from prompt tuning and fine-tuning, and be wary of public benchmarks whose questions may have leaked into pretraining data. Make every case unambiguous: if two experts would disagree on the expected answer, the case measures the rubric, not the model [2].

In Kojen

Any dataset with an input column and an expected-answer column can be picked as a test set on Evaluations. Without one, Kojen generates test cases from the project description.

Key takeaways

  • 20–50 real failure cases beat a large synthetic set you never read.
  • Run capability and regression suites separately.
  • Never tune on the test split.

Practise in the dojo

Run an accuracy evaluation on your project with a held-out dataset, then read every failed case.

Run an evaluation

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. What pass rate should a regression suite have?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Anthropic (2026). Demystifying evals for AI agents. Anthropic Engineering, January 2026.
  2. [2]Zinkevich, M. (2017). Rules of Machine Learning: Best Practices for ML Engineering. Google for Developers.