InterviewKata
AI engineering · Module 12a · Level 3: Configure your tools

Harness engineering: guides and sensors

A coding agent is a model plus everything around it. You can't change the model, but you can change everything around it: what steers the agent before it writes, and what catches its mistakes after. Thoughtworks calls this the agent's harness. This module shows how to build one that makes the agent correct itself, and how to improve it every time a mistake repeats. It works the same way in any coding agent.

A 5-minute conversation: Sam asks, Priya explains
0:00 / 4:34

1. The idea

A new cook in a restaurant kitchen. Recipe cards and a well-laid-out station steer them before they cook. A thermometer in the oven and the head chef tasting each plate catch problems after. When the same mistake happens twice, a good head chef doesn't shout louder: they fix the recipe card or add a check, so the next cook gets it right too.

Birgitta Böckeler (Thoughtworks) defines the harness as "everything in an AI agent except the model itself". The coding agent comes with a built-in harness (its tools, its loop, its prompts). Harness engineering is the part you add for your codebase, and it has two kinds of control:

Each can be computational (deterministic code: fast, cheap, always gives the same answer) or inferential (another model's judgement: slower, costs tokens, can vary, but understands meaning).

Guides: before it writes
Sensors: after it writes
Computational
Templates and codemodsProject scaffolds, generators, OpenRewrite recipes, type definitions the agent builds on.
Checks that can't be argued withCompiler, type checker, linter, tests, structural tests (ArchUnit, import-linter), mutation testing.
Inferential
InstructionsAGENTS.md, skills, specs, examples of good code.
A second opinionAI review, LLM as judge, a review skill (Module 13a).

You need both kinds. Böckeler's point: with only sensors, the agent keeps repeating the same mistakes; with only guides, you write rules and never find out whether they were followed.

2. How it works

The harness loop. Guides steer the agent; fast sensors make it fix its own mistakes before a commit; slower sensors run after integration. A person on the loop turns repeated problems into new guides or sensors.

The last step is what makes a harness get better over time. Thoughtworks calls it the feedback flywheel: treat recurring agent mistakes like a retrospective, and fix the harness instead of correcting the agent by hand again. Follow one recurring mistake:

Agent-written tests have their own trap: they can be "perpetually green", passing whatever the code does because they assert too little. Coverage can't see this; mutation testing can. It makes small deliberate bugs ("mutants") in the code and checks that some test fails for each one.

Write sensor messages for the agentA linter message such as "import not allowed" tells a person something is wrong. The agent needs to know how to fix it: "Domain code can't import infra. Depend on the port in domain/ports/ and inject it (see docs/architecture.md)." Böckeler calls these messages "optimised for LLM consumption". The better the message, the fewer loops the agent needs.

3. Try it yourself

Which control fixes this?

Each line is a problem that keeps happening. Pick the control that fixes it best, then check.

    Trim this AGENTS.md

    This file grew for a year and now the agent ignores half of it. For each line: keep it, turn it into a sensor (a check that enforces it), move it into a skill (loaded only when relevant), or delete it.

      4. Why it matters at work

      Thoughtworks' April 2026 Technology Radar made this its theme "Putting coding agents on a leash", warning that as agents improve, people are tempted to step out of the loop. Where the related practices stand:

      PracticeRadar ringWhat it means
      Curated shared instructionsAdoptA team-maintained AGENTS.md and shared skills, reviewed like code.
      Feedback sensors for coding agentsTrialWire compilers, linters, structural tests and test suites into the agent's workflow so failures trigger corrections, ideally during the session, before a commit.
      Mutation testingTrialA check on weak, "perpetually green" tests; tools such as Stryker, Pitest and cargo-mutants.
      Progressive context disclosureTrialLoad only the instructions the current task needs (skills do this).
      Feedback flywheelAssessCapture what went well and badly in agent sessions and improve the harness; re-check it when you change models.
      Agent instruction bloatCautionInstruction files that grow until rules conflict and important ones get missed.

      Where each sensor runs

      Fast sensors go as early as possible; slow ones later. Böckeler's lifecycle:

      WhenSensorsWhy there
      After every edit, in the sessionTypes, lint, formatter, the fast unit testsSeconds; the agent fixes it while it still has the context.
      Before a commitAll unit tests, structural testsNothing broken leaves the agent's session.
      In CI, after integrationMutation testing, AI review, security scans, slower testsMinutes, costs more; catches what fast checks can't.
      ContinuouslyArchitecture drift checks, production SLOsCatches slow decay and problems only real traffic shows.

      The portable setup

      One check script, called from everywhere. Any agent can run it, a git hook runs it whoever commits, and CI runs it again.

      #!/usr/bin/env bash
      # scripts/check: the fast sensors, in one place. Exit non-zero on any failure.
      set -euo pipefail
      npx tsc --noEmit            # types
      npx eslint . --max-warnings 0   # lint, including architecture rules
      npx vitest run --changed    # tests for what changed
      # AGENTS.md: the guide that points at the sensor (short, so it isn't lost)
      Before you say a task is done, run ./scripts/check and fix every failure.
      Never edit eslint.config.js, the tests' assertions or CI files to make a check pass.
      // eslint.config.js: a sensor whose message tells the agent how to fix it
      rules: {
        "no-restricted-imports": ["error", { patterns: [{
          group: ["**/infra/**"],
          message: "Domain code can't import infra. Depend on the port in domain/ports/ and inject it (see docs/architecture.md#ports).",
        }] }],
      }
      1. Git pre-commit hook runs scripts/check: works with all five tools, because it runs whoever (or whatever) commits.
      2. The agent's own hook (Module 12) runs the fast part after every edit, so mistakes are fixed in the session. Where each tool keeps it is in section 5.
      3. CI runs scripts/check again, plus mutation testing on core logic and the AI review from Module 13a.
      The honest caveatsSensors check what can be checked: Böckeler notes that behavioural correctness (does it do the right thing?) still needs human attention and good tests. Inferential sensors cost tokens and don't always give the same answer, so put deterministic ones first. A harness tuned for one model may be unnecessary, or wrong, for the next: the Radar suggests re-checking it when you switch models. And watch for agents gaming sensors by loosening the rule instead of fixing the code; protect the sensor configuration like CI (Module 13a).

      5. In your five tools

      Guides are portable (AGENTS.md and the SKILL.md format work everywhere); sensors in the session use each tool's hooks. Checked October 2026; see Modules 09, 11 and 12 for details.

      ToolGuides: instructions and skillsSensors in the session: run the check after edits
      Claude CodeCLAUDE.md (reads AGENTS.md if there's no CLAUDE.md, or @AGENTS.md); skills in .claude/skills/A PostToolUse hook in .claude/settings.json on Edit and Write; exit code 2 sends the failure back to the agent
      Antigravity CLIAGENTS.md or GEMINI.md; .agents/rules/; skills in .agents/skills/Hooks via /hooks
      GitHub Copilot.github/copilot-instructions.md, AGENTS.md, .github/instructions/; skills in .github/skills/ or .agents/skills/Agent-scoped hooks in a custom agent's frontmatter; hooks in the cloud agent
      CursorAGENTS.md or .cursor/rules/*.mdc; skills in .agents/skills/.cursor/hooks.json
      Codex CLIAGENTS.md (merged down the folder tree); skills in .agents/skills/.codex/hooks.json or [hooks] in config.toml: PostToolUse

      The git pre-commit hook and CI catch everything the in-session hooks miss, for every tool.

      6. Remember it

      Memory hook: "Recipe card before, taste test after"Guides steer the cook before cooking; sensors catch problems after. And when a mistake repeats, fix the kitchen, not the cook.
      How this hook works: what “Recipe card before, taste test after” means

      What the slogan means, phrase by phrase

      “Recipe card before”
      Guides: AGENTS.md, skills, specs, templates. They make the first attempt more likely to be right.
      “taste test after”
      Sensors: types, lint, tests, structural tests, mutation testing, AI review. They let the agent correct itself.

      Why this picture fits: no kitchen relies only on recipes or only on tasting. And a head chef who fixes the recipe card once saves correcting every cook every night.

      The story → the concept

      KitchenHarness
      Recipe cardsAGENTS.md, skills, specs (inferential guides)
      Station layout, pre-portioned ingredientsTemplates and codemods (computational guides)
      Oven thermometerCompiler, linter, tests (computational sensors)
      Head chef tastingAI review (inferential sensor)
      Fixing the recipe cardThe feedback flywheel

      The trap it saves you from

      Correcting the same agent mistake by hand, session after session. If you've typed the same review comment twice, it belongs in the harness.

      7. Common mistakes

      Rules with no checks"Always use the logger" in AGENTS.md, and nothing that verifies it. If a rule matters, back it with a sensor.
      Sensor messages written for people"Error E1023" makes the agent guess. Say what's wrong and how to fix it.
      All sensors in CIA failure found twenty minutes after the session ended costs a new session. Run the fast checks in the session.
      Instruction bloatHundreds of lines in AGENTS.md: rules conflict and the important ones get lost in the middle. Keep a short core; move the rest into skills or sensors.
      Trusting coverage90% coverage with assertion-free tests catches nothing. Run mutation testing on core logic.
      Letting the agent edit its own sensorsAn agent that can relax a lint rule or delete an assertion will, eventually. Protect those files and treat changes to them as high risk.
      Never pruningA rule added for last year's model may now be noise. Re-check the harness when you change models.

      8. Hands-on lab (roadmap week 12)

      1. Collect the repeats: list three review comments you've made on agent code more than once.
      2. Write the check script: types, lint and fast tests in scripts/check; point AGENTS.md at it in one line.
      3. Turn one repeat into a sensor: a lint rule or structural test whose message says how to fix it. Ask an agent to make the mistake on purpose and watch it correct itself.
      4. Wire it in: a git pre-commit hook for every tool, plus the after-edit hook in tool A.
      5. Run mutation testing (Stryker, Pitest, mutmut or cargo-mutants) on one core module the agent wrote tests for. Add tests until the important mutants are killed.
      6. Trim AGENTS.md: remove or move every line that a sensor now enforces or only one task needs. Compare its length before and after.

      9. Self-check quiz

      A linter that fails when domain code imports infrastructure code is…

      It observes after the agent acts and gives feedback: a computational sensor.

      You've made the same review comment on agent code three times. Best fix?

      The agent doesn't learn between sessions; the harness does. A checked rule beats a louder one.

      Agent-written tests show 95% coverage but you suspect they assert nothing useful. What reveals it?

      If small deliberate bugs don't make any test fail, the tests aren't checking the behaviour.

      Which sensors belong inside the agent's session?

      Fast and deterministic, so the agent fixes problems while it still has the context.

      What's the risk of an AGENTS.md that keeps growing?

      The Radar puts agent instruction bloat in Caution: models attend less to the middle of long contexts.

      Your team switches to a newer model. What should happen to the harness?

      A practice that helped one model may be unnecessary with the next; sensors stay, but guides may shrink.

      10. Go deeper

      ResourceTimeWhy
      Harness engineering for coding agent users (Birgitta Böckeler, martinfowler.com, April 2026)25 minThe source of this module's model: guides and sensors, computational and inferential, and where each runs.
      Thoughtworks Radar, April 2026: Feedback sensors, Feedback flywheel, Agent instruction bloat, Mutation testing10 minOne paragraph each on what teams found in practice.
      Stryker, Pitest, mutmut, cargo-mutants15 minMutation testing for JavaScript/TypeScript/C#, Java, Python and Rust.
      ArchUnit and import-linter15 minStructural tests: architecture rules as sensors, for Java and Python.
      Modules 09, 11, 12 and 13a–Instructions, skills, hooks and AI review: the parts this module joins up.
      Case study C05: Replit's agent deletes productionCaseWhy instructions failed, and the guards and environments that work.
      Case study C06: a GitHub issue hijacks an agentCasePrompt injection through a tool, and the policies that stop the leak.