Evaluation & Trust
Lesson 5 of 5Intermediate 7 min

Latency, trust scores and the ship decision

Users feel the slowest responses, not the average. Combine quality and speed into one decision you can defend.

In this lesson you'll

  • Read p50, p95 and p99 latency
  • Combine metrics into a trust score
  • Write down a ship threshold

Averages hide the experience that drives complaints. Dean and Barroso showed that in systems that fan out to many calls, rare slow responses come to dominate overall latency, which is why engineers track tail percentiles like p95 and p99 [1]. Google's SRE book lists latency alongside traffic, errors and saturation as the four golden signals to monitor [2].

One score, many inputs

Model cards recommend reporting performance broken down by condition and group rather than one headline number [3]. A trust score is useful as a gate, as long as you keep the components visible underneath it.

  1. Set minimums for each component, such as accuracy ≥ 85% and no critical safety failures.
  2. Set a p95 latency budget from the product's needs.
  3. Ship only when every minimum is met, and record the evaluation alongside the model version.

In Kojen

Evaluations report p50/p95/p99 latency from real calls and a trust score from 0 to 100. For comprehensive evaluations it weights accuracy 50%, hallucination 30% and bias 20%.

Key takeaways

  • Watch p95 and p99, not the mean.
  • Gate on minimums per component, then on the overall score.
  • Keep the evaluation with the model version you ship.

Practise in the dojo

Compare two model versions side by side on trust score and p95 latency, and decide which you'd ship.

Compare versions

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. Why track p99 latency rather than the average?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Dean, J. & Barroso, L. A. (2013). The Tail at Scale. Communications of the ACM 56(2).
  2. [2]Beyer, B. et al. (eds.) (2016). Site Reliability Engineering, ch. 6: Monitoring Distributed Systems. O'Reilly / Google.
  3. [3]Mitchell, M. et al. (2019). Model Cards for Model Reporting. FAT* 2019 · arXiv:1810.03993.