AI training and evaluation glossary
31 terms used across Kojen's tools, Academy and blog, each defined in a sentence or two, then explained with sources and a link to where we cover it in depth.
Evaluation
- Agent eval
- An agent eval measures an AI agent on multi-step tasks, scoring both the final outcome and how it got there, including tool calls, turns, cost and reliability over repeated runs.
- Cohen's kappa
- Cohen's kappa measures agreement between two raters on categorical labels, corrected for the agreement expected by chance: 1 is perfect, 0 is chance level.
- Confidence interval
- A confidence interval is a range computed from a sample that, across repeated samples, would contain the true value a stated share of the time, such as 95%. It is the error bar on an eval score.
- Hallucination
- A hallucination is model output that is fluent and confident but unsupported by its source material or false about the world.
- Holdout set
- A holdout set is a portion of labelled examples kept out of training and tuning, used only to measure how well a model or prompt generalises to data it hasn't seen.
- LLM-as-a-judge
- LLM-as-a-judge is the practice of using a language model to grade another model's outputs against a rubric or a reference answer, instead of a human grader.
- McNemar test
- McNemar's test checks whether two classifiers, prompts or models differ in accuracy when both are run on the same cases, using only the cases where exactly one of them is correct.
- pass@k
- pass@k is the probability that at least one of k sampled attempts at a task is correct. It measures whether a model can solve a task given several tries.
- pass^k
- pass^k is the probability that all k attempts at the same task succeed. It measures how reliably an agent succeeds every time, and falls as k grows.
- Wilson interval
- The Wilson score interval is a confidence interval for a proportion, such as a pass rate, that stays within 0–100% and keeps close to its stated coverage even for small samples and rates near 0 or 100%.
Training
- DPO
- Direct Preference Optimization (DPO) trains a model on pairs of preferred and rejected responses with a simple classification-style loss, matching RLHF's objective without a separate reward model or reinforcement learning.
- LoRA
- LoRA (low-rank adaptation) fine-tunes a model by freezing its weights and training small low-rank matrices added to chosen layers, so only a tiny fraction of parameters is updated.
- QLoRA
- QLoRA is LoRA on top of a base model whose frozen weights are stored in 4-bit, cutting fine-tuning memory roughly four-fold compared with LoRA on 16-bit weights.
- RLHF
- Reinforcement learning from human feedback (RLHF) fine-tunes a model to maximise a reward model trained on human preference comparisons, usually with PPO and a penalty for drifting from the original model.
- RLVR
- Reinforcement learning with verifiable rewards (RLVR) trains a model with rewards from an automatic checker, such as a correct maths answer or passing unit tests, instead of a learned reward model.
Efficiency
- 6ND
- 6ND is the rule of thumb that training a Transformer takes about 6 floating-point operations per parameter per token, so compute ≈ 6 × N (parameters) × D (training tokens).
- Chinchilla scaling
- Chinchilla scaling is DeepMind's finding that, for a fixed compute budget, model size and training tokens should grow in equal proportion, which works out to about 20 tokens per parameter.
- Gradient checkpointing
- Gradient checkpointing saves memory in training by keeping only some layers' activations during the forward pass and recomputing the rest during the backward pass.
- KV cache
- The KV cache stores the attention keys and values of tokens a Transformer has already processed, so generating each new token doesn't recompute them. Its size grows linearly with context length and batch size.
- MFU
- Model FLOPs utilisation (MFU) is the share of a GPU's theoretical peak throughput that a training run spends on the model's own arithmetic. It is the main efficiency number for turning FLOPs into GPU-hours.
- Mixed precision
- Mixed-precision training runs most arithmetic in 16-bit floating point (FP16 or BF16) while keeping a 32-bit master copy of the weights for updates, roughly halving memory traffic and using fast tensor cores.
- MoE
- A mixture-of-experts (MoE) model replaces each dense feed-forward layer with many expert networks and a router that sends each token to a few of them, so only a fraction of the parameters work on any token.
- Quantization
- Quantization stores a model's weights (and sometimes activations) in fewer bits, such as 8 or 4 instead of 16, to cut memory and often speed up inference at a small cost in accuracy.
Security
- Indirect prompt injection
- Indirect prompt injection is prompt injection delivered through content an AI system reads, such as a web page, email, document or tool result, rather than typed by the user.
- Prompt injection
- Prompt injection is an attack in which text supplied to a language model overrides or subverts its intended instructions, for example "ignore previous instructions and…".
- Red teaming
- Red teaming is deliberately attacking an AI system, by hand or with automated attackers, to find harmful, insecure or failing behaviour before real users do.
Agents
- Context engineering
- Context engineering is designing everything that goes into a model's context window, instructions, tools, retrieved documents, memory and history, so an agent has the information it needs and little else.
- MCP
- The Model Context Protocol (MCP) is an open protocol for connecting AI applications to external tools, data and prompts through a standard client–server interface, built on JSON-RPC.
- OpenAI-compatible API
- An OpenAI-compatible API accepts the same request and response format as OpenAI's Chat Completions endpoint, so existing SDKs and tools work with it by changing only the base URL and key.
Kojen
- Kata (Kojen)
- In Kojen, a kata is a graded training exercise, at one of four levels from beginner to master, that Sensei scores; 70 or more passes.
- Trust score (Kojen)
- Kojen's trust score is a 0–100 roll-up of an evaluation; for comprehensive evaluations it weights accuracy 50%, hallucination 30% and bias 20%.
See these in action.
Kojen's evaluations report Wilson intervals, pass^k and judge-graded trust scores on your own test cases. Free to start.
Start training free