InterviewKata
AI engineering · Module 02 of 21 · Level 1: How AI works

The context window: everything the model can see

A model knows two things: what it learned in training, and what's in front of it right now. "What's in front of it" is the context window. Managing it well is the single most underrated skill in using AI tools: it decides quality, speed and cost.

1. The idea

A brilliant consultant works at a desk. They can only use the papers on the desk right now; the books they read years ago are a hazy memory. The desk has a fixed size. Pile on too much and they start missing things; clear it at night and tomorrow they remember nothing of today, unless someone leaves them notes.

The context window is the maximum amount of text, measured in tokens, that a model can take in for one request. Everything has to fit in it together:

The surprise for most people: the model has no memory between requests. It is stateless. A chat feels continuous only because the app re-sends the whole conversation every time you press Enter.

Current frontier models (Claude, Gemini and GPT, October 2026) offer windows of about 1 million tokens, roughly 500,000–750,000 English words. That's big, but it isn't free: every token you send costs money and time, and quality drops as the window fills.

2. How it works

Each turn, the app builds the whole context again and sends it. The model reads it, replies, and forgets. The app, not the model, keeps the history.

Because everything is re-sent, each turn is bigger than the last. Step through four turns of a chat:

Now watch a real coding-agent session fill a 200,000-token window. Tool output (file contents and test logs) fills it far faster than your messages.

Context rot: a full desk is a worse deskModels don't use a long context evenly. Chroma's 2025 study of 18 models found every one got less accurate as the input grew, long before the window was full. Instructions buried in the middle of a long session get followed less reliably. That's why tools compact, and why a fresh session often beats a long one.

3. Try it yourself

Build a session and see how much of the window it takes. The sizes are typical estimates; your tool's context command shows the real numbers.

4. Why it matters at work

Everything you do with an AI tool either adds to the desk or tidies it. The habits of people who get great results:

  1. One task, one session. Start fresh for each unrelated task. Old attempts and dead ends confuse the next task.
  2. Point, don't paste. "Read src/billing/refund.ts" lets an agent read just what it needs; pasting 800 lines fills the desk with things it may never need.
  3. Ask for filtered output. "Run the tests and show only failures" instead of a 3,000-line log.
  4. Connect only the tools this repo needs. Every MCP server's tool descriptions ride along on every request.
  5. Keep project instructions short. AGENTS.md is loaded every session; aim for a few hundred lines at most, and move detail into skills that load on demand (Module 11).
  6. Write decisions into files, not chat. The chat will be compacted or cleared; a file in the repo survives.
  7. Compact deliberately when a long task is going well but the window is filling: the summary keeps the decisions and drops the noise.

This discipline has a name: context engineering, deciding exactly what the model sees. Anthropic describes it as the natural next step after prompt engineering, and it's the theme of Modules 05 and 09.

5. In your five tools

Every tool lets you see how full the window is, shrink it, and start over. Checked against each tool's documentation, October 2026.

ToolSee what's in contextShrink itStart fresh
Claude Code/context (a breakdown by category)/compact, optionally with what to keep; also compacts automatically when nearly full/clear
Antigravity CLI/context (a visual breakdown)No compact command documented yet; carry a short summary into a fresh session/clear (or /new)
GitHub Copilot (VS Code)The context-size indicator by the chat input; click it for a breakdownRemove attached files or tools from the chatStart a new chat session
CursorThe context percentage in the chat; hover it for token countsRemove @-mentioned context you no longer needStart a new chat
Codex CLI/status (includes token usage)/compact; also compacts automatically under pressure/new

6. Remember it

Memory hook: "The desk, not the library"The model works only from what's on its desk right now. The desk is a fixed size, clutter makes it worse, and it's cleared between sessions.
How this hook works: what “The desk, not the library” means

What the slogan means, phrase by phrase

“The desk”
The context window: the papers in front of the model for this one request. If it isn't on the desk, the model can't use it, however important it is.
“not the library”
Training knowledge is like a library the consultant read long ago: broad but hazy, and with nothing published after the knowledge cutoff. It can't walk to the library mid-task; someone (you, or a tool) has to bring the page to the desk.
Implied: a fixed-size desk
200K or 1M tokens, whatever the model offers. Pile on more and something falls off, or the request fails.
Implied: cleared every night
Each new session starts with an empty desk, except the notes the app puts there automatically (project instructions and memory files).

Why this picture fits: it explains all four practical rules at once: give it what it needs (it can't fetch from the library), don't pile on (fixed size, clutter hurts), tidy up (compaction), and leave notes for tomorrow (files, not chat).

The story → the concept

In the storyIn the model
Papers on the deskEverything in the context window: instructions, chat, files, tool results
Size of the deskThe context window limit, in tokens
The library read long agoTraining data, stored in the weights; frozen at the knowledge cutoff
An assistant fetching a bookA tool call or retrieval (RAG) that puts fresh text on the desk
Tidying the desk into one summary pageCompaction (/compact)
A fresh desk tomorrowA new session (/clear, /new)
Notes left for tomorrowAGENTS.md, memory files, decisions written into the repo

Watch it on an example

An afternoon fixing a flaky test with a coding agent:

  1. The session starts: the desk already holds the system prompt, tool descriptions and AGENTS.md.
  2. The agent reads six files and runs the test suite three times. Each full log lands on the desk.
  3. Two hours in, it ignores a rule from AGENTS.md it followed earlier. The rule is still on the desk, buried under logs: clutter.
  4. You run /compact. The desk becomes one summary page plus the instructions, and the agent follows the rule again.
  5. Tomorrow's session remembers nothing, except the note you added to AGENTS.md: "the date tests need TZ=UTC".

The trap it saves you from

Assuming the model "knows" your repo, yesterday's chat or your company's docs. Ask: is it on the desk? If not, point to it, or let the agent read it.

It still works when…

You meet memory features (Module 09) and RAG (Module 15). Memory is just notes the app puts on the desk at the start of every session. RAG is a librarian who fetches the few most relevant pages and puts them on the desk before the model answers.

7. Common mistakes

One endless sessionWeeks of history, failed attempts and stale decisions crowd the desk. Start fresh per task and carry over only a short summary.
Pasting whole files and full logsPoint the agent at files and ask for filtered output. It reads only what it needs.
Connecting every MCP server "just in case"Each one adds its tool descriptions to every request. Connect what this repo uses; disable the rest.
Treating a 1M window as permission to dump everythingBigger windows cost more per request and still suffer from context rot. Use them for genuinely large inputs, not as a substitute for choosing well.
Expecting it to rememberA new session starts empty. Anything that must survive goes into a file the tool loads: AGENTS.md, a design doc, or memory.

8. Hands-on lab (25 minutes)

  1. Baseline. Open a coding agent in a real repo and, before typing anything, check the context (/context in Claude Code or Antigravity CLI, /status in Codex, or the indicator in Copilot or Cursor). Note how full it is already.
  2. Fill it. Ask it to read three files and run the test suite. Check the context again. Which category grew most?
  3. Filter it. In a fresh session, ask it to run the tests and report only failures. Compare the growth with step 2.
  4. Compact it. Run /compact (Claude Code or Codex) and check once more. Then ask "what were we doing?" to see what the summary kept.
  5. Leave a note. Add one line to your AGENTS.md that would have saved time today. Start a new session and confirm it's loaded.

9. Self-check quiz

Why does a chat feel continuous if the model has no memory?

Models are stateless. The app keeps the history and sends all of it with every request.

In a typical coding-agent session, what fills the window fastest?

One test log or a few large files can be bigger than an hour of chat. Ask for filtered output.

What is "context rot"?

Studies show every model degrades with longer inputs. A tidy context beats a huge one.

You found a rule the agent should always follow in this repo. Where should it go?

Chat is cleared and compacted. Files the tool loads put the rule on the desk every session.

10. Go deeper

ResourceTimeWhy
Effective context engineering for AI agents (Anthropic)20 minThe best practical guide to what belongs in context and how agents manage it.
Context Rot (Chroma research)15 minThe evidence that longer inputs hurt, across 18 models.
Context windows (Claude docs)10 minHow input, thinking and output share the window.
Long context (Gemini docs)10 minWhen a 1M window is the right tool, and how to use it well.