Agents in Production
Lesson 2 of 6Advanced 9 min

Context engineering

In 2026, the craft has moved from writing one good prompt to curating everything an agent sees over hundreds of turns.

In this lesson you'll

  • Explain why context is a finite budget
  • Use just-in-time retrieval
  • Apply compaction, note-taking and subagents to long tasks

Long-context models still don't use every token equally. Information in the middle of a long prompt is used less reliably than information at the ends [1], and Anthropic describes context rot: as the context grows, recall degrades. They frame context as a finite attention budget and the job as finding the smallest set of high-signal tokens that gets the outcome you want [2].

Techniques

  • Just-in-time retrieval: keep lightweight references (file paths, queries, links) and load the data with tools only when needed, rather than stuffing everything in up front.
  • Compaction: when a conversation nears the limit, summarise it and continue in a fresh window. Clearing old tool results is the safest first step.
  • Structured note-taking: the agent writes notes to memory outside the context window and reads them back later.
  • Subagents: delegate focused work to subagents with clean contexts, and have them return condensed results [2].

Tools are context too

Every tool description and result spends the budget. Keep tool sets small and non-overlapping, and return concise, relevant results rather than raw dumps.

Key takeaways

  • More context isn't always better; relevance is.
  • Load data just in time through tools.
  • Compact, take notes and delegate for long-running work.

Practise in the dojo

Ask the Kojen Agent to train and evaluate a model, and notice how it fetches project details with tools instead of being handed everything up front.

Talk to the Kojen Agent

Check your understanding

Score 70 or more to complete the lesson. Progress is saved in this browser.

1. What is "just-in-time" context?

Further reading

Sources

Checked against the original sources in November 2026.

  1. [1]Liu, N. F. et al. (2023). Lost in the Middle: How Language Models Use Long Contexts. TACL 2024 · arXiv:2307.03172.
  2. [2]Anthropic (2025). Effective context engineering for AI agents. Anthropic Engineering, September 2025.