The harness: everything the agent runs inside

The harness is everything a coding agent runs inside apart from the model: the context it reads, the tools it calls, the permissions and sandbox bounding it, the hooks that check it, the skills and plugins it loads, and its execution environment. Each layer prevents one class of failure, so the harness is versioned, reviewed and evaluated like code.

Your agent ran git push --force last week, although CLAUDE.md says in capitals never to do that. A colleague’s agent keeps formatting files differently from yours, and nobody can say which of the 400 lines of rules is still true. Every fix so far has been another sentence in the rules file. This page gives each of those problems a home in the layer that can actually stop it.

What you’ll walk away with from the harness overview

• A map of the seven layers, the failure each one prevents, and where each lives in Claude Code, Codex and Cursor.
• A decision table that tells you which layer a fix belongs in, so a repeated mistake stops becoming another rule.
• A five-step procedure to put the harness in the repository, review it, validate it in CI and evaluate changes against a baseline.
• Three copy-paste prompts: inventory your harness, route a repeated mistake to the right layer, and build a small eval set.
• The failure modes of the harness itself, from silently ignored settings to rules files that contradict each other.
• A reading order for the more than 20 harness pages spread across five sections of this site.

What is the harness, layer by layer?

The same model behaves differently in different harnesses. Public benchmarks already score the pair, not the model alone: every entry on the official Terminal-Bench 4.0 leaderboard names a model and an agent harness, for example Claude Fable 5.1 (max effort) with Claude Code at 57.9% ± 3.8 (leaderboard read 2026-09-26). Your repository adds its own layers on top of the vendor’s, and those are the ones you control.

#	Layer	What it holds	The failure it prevents	Enforcement
1	Context	Project instructions (CLAUDE.md, AGENTS.md, Cursor Rules), memory	The agent re-derives your conventions, invents the test command, or uses a pattern you retired	Advice: the model may ignore it
2	Tools	MCP servers, CLIs, subagents	The agent guesses from training data about your schema, your tickets or a library’s current API	Capability: what the agent can reach
3	Permissions and sandbox	Permission modes, allow/ask/deny rules, approval policies, OS sandbox, network egress	Destructive or irreversible actions, secret reads, writes outside the workspace	Hard: the client or the OS refuses
4	Hooks	Scripts that run at lifecycle events (before a tool call, after an edit, at stop)	A rule forgotten mid-session: unformatted code, a protected path edited, “done” without tests	Deterministic: code runs every time
5	Skills	Procedures loaded on demand (SKILL.md plus scripts)	The same procedure done differently every time, or always-on rules bloating every session	Advice, loaded only when relevant
6	Plugins	Installable bundles of skills, hooks, subagents and MCP servers	Eight developers running eight drifting copies of the harness	Distribution: one versioned source
7	Environments	Worktrees, containers, cloud environments, seeded data, port blocks	Parallel agents clobbering each other, verification that only works on one laptop, a large blast radius	Isolation: separate state per run

Two things sit next to the harness rather than inside it. The codebase decides whether a check can replace reading: one-command tests, strict types and clear module boundaries (see making a codebase agent-ready). The oracle is the set of tests and gates that decides “done”, and the agent must not be able to edit it (see protecting the oracle). The one-map page places the harness as one of the six factory stations.

Why the layers form a hierarchy of trust

Read the last column of the table from top to bottom. Context is a request, a hook is code, and a sandbox is a wall. A sentence in AGENTS.md that says “never read .env” works most of the time; a deny rule or a sandbox that cannot reach the file works every time. So the rule of thumb for the whole section is: put each control in the most deterministic layer that can express it, and keep context for what only a sentence can say, such as naming conventions, architecture intent and where things live.

The same logic sets the limits of each layer. A command deny rule matches the command text the agent writes, so the same action spelled another way (sh -c '…', a package script, a Makefile target) can slip past it. Treat pattern rules as typo-catchers. The real boundary is what the process can reach: the OS sandbox, network egress, and which credentials exist in the environment at all. A production deploy key that is not on the machine cannot be misused by any prompt.

Which layer should a fix go in?

When the agent makes the same mistake twice, the reflex is to add a sentence to the rules file. Use this table instead. The right-hand column is the evidence that the fix works.

Symptom	Put the fix in	Proof it works
Runs npm test when the repository uses pnpm test:unit	Context: one line naming the command	The next session runs the right command without being told
Edits generated or vendored files	Permissions (deny writes to the path) plus a pre-tool hook	A fixture edit to the path is blocked with a readable reason
Says “done” without running the tests	Hook at stop, or a goal condition that runs the check	A session with a failing test cannot finish
Reads .env or other secrets	Sandbox and permissions; keep real secrets out of the workspace	A read attempt is refused by the client or the OS
Could deploy or write to production	Environment: no production credentials on the machine; deploys are CI’s job	The credential does not exist where the agent runs
Does the release checklist differently each time	Skill with the checklist and a script	Three runs produce the same artifacts in the same order
Guesses the database schema	Tool: a read-only database MCP server or CLI	Queries in the transcript match the real schema
Two parallel agents break each other’s dev server	Environment: one worktree and one port block per agent	Both runs pass while running at the same time
Your harness differs from your teammate’s	Plugin or committed project settings	A fresh clone produces the same /context inventory (Claude Code) or /debug-config layer stack (Codex)
A rule is ignored late in long sessions	Move it from context to a hook	The hook fires in the transcript every time

Where does each layer live in Claude Code, Codex and Cursor?

All three tools implement every layer, with different file names and different trust rules. The workflow on this page is the same for all three; the locations differ.

Claude Code

Checked against Claude Code 2.1.283 (latest channel).

• Context: CLAUDE.md plus auto memory. Claude Code reads AGENTS.md when a project has no CLAUDE.md (from v2.1.277; latest channel only on 2026-09-26).
• Tools: project MCP servers in .mcp.json (unapproved servers stay pending until you approve them), claude mcp add, and subagents in .claude/agents/.
• Permissions and sandbox: permissions allow, ask and deny rules in .claude/settings.json; permission modes (Manual, acceptEdits, plan, auto, dontAsk, bypassPermissions); sandboxed Bash on macOS, Linux and WSL2. auto and bypassPermissions set as defaultMode in project settings are ignored: they belong in user or managed settings.
• Hooks: the hooks block of any settings file; 33 events, handler types command, http, mcp_tool, prompt and agent (experimental).
• Skills: .claude/skills/<name>/SKILL.md.
• Plugins: claude plugin install, marketplaces, and enabledPlugins in settings.
• Environments: --worktree, cloud environments, self-hosted environments.

Precedence, highest first: managed settings, command-line arguments, .claude/settings.local.json, .claude/settings.json, ~/.claude/settings.json. Commit .claude/settings.json; keep personal overrides in the local file, which stays out of git.

{
  "permissions": {
    "deny": ["Read(./.env)", "Read(./.env.*)", "Bash(git push *)"],
    "ask": ["Bash(pnpm publish *)"]
  },
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [{ "type": "command", "command": ".claude/hooks/protect-generated.sh" }]
      }
    ]
  }
}

Codex

Checked against Codex CLI 0.157.1.

• Context: AGENTS.md (/init creates one). Since 0.150.0, untrusted projects no longer supply project-level AGENTS.md.
• Tools: codex mcp add, list and remove manage the servers Codex connects to; subagents are on by default (/subagents).
• Permissions and sandbox: permission profiles (beta, CLI 0.138.0 and later) through default_permissions, with built-ins such as :read-only and :workspace; approval policies untrusted, on-request (default), granular and never; command rules in .rules files (execpolicy); /permissions in the TUI.
• Hooks: 12 events, handler types command, mcp_tool, prompt and agent. Hooks need persisted trust; review them with /hooks.
• Skills and plugins: /skills, codex plugin add, /plugins.
• Environments: --worktree or /worktree, and Codex cloud environments.

Project settings go in .codex/config.toml, which Codex loads for a trusted repository. Admins constrain everything in requirements.toml, for example allow_managed_hooks_only. /debug-config shows which layer set each value.

# .codex/config.toml (loaded only when the project is trusted)
default_permissions = ":workspace"   # permission profile, beta
approval_policy = "on-request"

OpenAI states that permission profiles and the legacy --sandbox flag “do not compose”, so pick one system per configuration.

Cursor

Cursor feature names checked on cursor.com on 2026-08-28; exact file paths and setting names are left out until they are re-verified.

• Context: Rules, the project instructions Cursor’s agent loads.
• Tools: MCP servers and Subagents (“specialized AI assistants that Cursor’s agent can delegate tasks to”).
• Permissions and sandbox: the Run modes page under Agent security, plus the /sandbox command in the CLI.
• Hooks: “spawned processes that communicate over stdio using JSON in both directions” that “can observe, block, or modify behavior”.
• Skills: Agent Skills, “a portable, version-controlled package that teaches agents how to perform domain-specific tasks”.
• Plugins: “Plugins package rules, skills, agents, commands, MCP servers, and hooks into distributable bundles.”
• Environments: Worktrees for local isolation; Cloud Agents “run in isolated VMs in the cloud with full development environments”, prepared by Builds.

Cursor’s SDK (@cursor/sdk 1.0.32) names the settings layers it can load: project, user, team, mdm and plugins. That is the same layering as the other two tools: a team or MDM layer above the project, and the project above the individual.

How do you version, review and evaluate the harness?

A harness file changes what every future session does, so a change to it deserves the same treatment as a change to a shared library. The procedure below works for all three tools.

1. Put the harness in the repository. Commit project instructions, project settings, hooks and their scripts, skills and the MCP server list. Keep personal overrides (.claude/settings.local.json, your user config) out of git. A fresh clone should give a new teammate the same harness you have.

2. Give harness files an owner. Add the paths to CODEOWNERS: AGENTS.md, CLAUDE.md, .claude/, .codex/, .mcp.json, the hook scripts, and .github/ (the workflows that run the checks in step 3). CODEOWNERS only requests a review; it blocks the merge once the branch protection rule or ruleset for the default branch has “Require review from Code Owners” turned on. For one team the owner is usually the tech lead; the operating model covers who owns it when several teams share one harness.

3. Validate the configuration in CI. Headless runs are where a broken config hides. A CI job can fail fast before any agent runs, and every check below fails closed: a tool that crashes or prints nothing fails the job instead of passing it.

   # CI, from the repository root. Needs no API key and no codex login.
   set -euo pipefail
   claude doctor > doctor.txt 2>&1 || true; cat doctor.txt
   grep -q '^Claude Code doctor' doctor.txt || { echo 'claude doctor did not run'; exit 1; }
   if grep -q 'Invalid settings' doctor.txt; then echo 'Invalid Claude Code settings'; exit 1; fi
   jq -e '.permissions.deny | length > 0' .claude/settings.json   # the deny rules you rely on are really there
   claude plugin validate --strict .claude/skills                # skills only; repeat for .claude/agents if present
   export CODEX_HOME="$(mktemp -d)"                               # throwaway Codex home: no login, no user config
   printf '[projects."%s"]\ntrust_level = "trusted"\n' "$(pwd -P)" > "$CODEX_HOME/config.toml"   # trust this checkout so .codex/config.toml loads
   codex doctor --json > codex-doctor.json || true
   jq -e '(.checks[] | select(.id == "config.load") | .status == "ok")
     and ([.checks[] | select(.id | test("^(config|sandbox|mcp)\\.")) | .status] | all(. != "fail"))' codex-doctor.json

   claude doctor reads the project settings without a trust prompt and lists invalid values and wrong types under “Invalid settings”; an unknown or misspelled key is not reported (checked on 2.1.283 and 2.1.287). It exits 0 even then, so the job gates on its output, and the first grep stops a missing or crashed claude from passing as “no invalid settings”. The jq line catches a misspelled deny key that claude doctor accepts in silence. claude plugin validate checks skills, agents and commands, not settings; pointed at a .claude/ that holds only settings.json, it fails with “No manifest found in directory”. codex doctor exits 1 whenever its auth or network checks fail, which they do on a runner without a login, so the job reads its JSON report and asserts only on the config, sandbox and MCP checks (Codex 0.157.1); an empty or malformed report fails the jq -e.

   The two lines before codex doctor are not optional. Codex loads .codex/config.toml only for a trusted project, and a fresh runner has no trust entry, so without them codex doctor never reads the repository’s config and reports config.load as ok even when the file does not parse. Passing the trust setting with -c does not change that; a trust entry in the Codex config does (tested on 0.157.1). The entry uses pwd -P because a symlinked path does not match it. Trusting the checkout makes Codex load the pull request’s own project config, so run this job only on the pull_request trigger, never on pull_request_target, and give it no secrets.

   On a pull_request trigger, GitHub runs the workflow file from the pull request itself, so a pull request that edits .github/workflows/ can edit this job so it passes without checking anything, for example by replacing its steps with exit 0. Make the job a required status check as well, so that deleting it leaves the check pending and blocks the merge. The check above catches mistakes, not tampering; the control against tampering is the required Code Owner review of .github/ from step 2.

   In the agent run itself, add --strict-config to codex exec so an unrecognized config.toml field is an error, not a silent no-op.

4. Evaluate changes against a baseline. Keep 5 to 10 real tasks from recent pull requests, each with a check command that decides pass or fail. Run them with the current harness and with the proposed change, then compare pass rate, turns and tokens. For a harness packaged as a Claude Code plugin, claude plugin eval does this for you: it runs the plugin’s eval cases and, by default, adds a no-plugin baseline arm (--ablation with-without) and reports the score delta. It runs the plugin on your machine with your credentials, so evaluate only plugins you trust; the first run in an untrusted plugin directory asks for confirmation, and --trust-plugin answers it in CI. The method is on continuous evals for agent harnesses.

5. Sign off on the result, not the diff. The owner approves a harness change when the eval set is at least as green as before and the context cost did not grow without a reason. Record the result in the pull request. That record is what lets you delete a rule later with confidence.

Measure what the harness costs in context

Every always-on layer spends tokens in every session: instructions, MCP tool definitions, skill descriptions and plugin components. In Claude Code, /context shows the current session’s context use and claude plugin details <name> shows a plugin’s component inventory and projected token cost. Tool search is on by default in both Claude Code and Codex, which keeps large MCP tool lists from loading in full. See reducing MCP token cost.

Copy-paste prompts for auditing your harness

Copy-paste prompt: inventory the harness

Inventory every harness artifact in this repository and in my user config that
affects how you work here. Cover seven layers: context (CLAUDE.md, AGENTS.md,
rules), tools (MCP servers, subagents), permissions and sandbox, hooks, skills,
plugins, and environments (worktree, container, cloud, ports, seeded data).

For each artifact output one table row: layer | path | what it tells or forces
you to do | the failure it prevents | whether anything tests it | estimated
tokens it adds to every session.

Then list: (1) instructions that contradict each other, (2) rules in context
that a hook or permission rule could enforce instead, (3) procedures longer than
ten lines that belong in a skill, (4) anything that loads in every session but
was not needed for the last five tasks in git log. Do not edit any file.

Copy-paste prompt: route a repeated mistake to the right layer

You made this mistake twice in this repository: you edited files under
src/generated/ although they are produced by `pnpm codegen`.

Do not add another sentence to AGENTS.md or CLAUDE.md. Instead:
1. Add a deny rule for writes to src/generated/ in the project settings.
2. Add a pre-tool hook script that blocks Edit and Write calls to that path and
   prints "src/generated/ is codegen output: change the schema and run
   pnpm codegen".
3. Add a test fixture that calls the hook with a sample edit to
   src/generated/api.ts and asserts it is blocked, and one for src/api.ts that
   asserts it passes.
4. Remove any existing sentence about src/generated/ from the context files.
Show me the diff and the fixture output. Do not touch anything else.

Copy-paste prompt: build a harness eval set from real work

Read the last 30 merged pull requests (git log --merges -30) and pick 8 that
represent our normal work: at least one bug fix, one feature, one refactor, one
test-only change and one dependency update.

For each, create evals/harness/<short-name>/ with:
- task.md: the request as a teammate would have written it, with no hints from
  the final diff;
- base.txt: the commit to start from (the PR's parent);
- check.sh: a command that exits 0 only when the behavior is right (reuse the
  tests the PR added; if it added none, write the narrowest test that proves the
  change).

Do not include the solution anywhere in the eval directory. Print a table of the
8 tasks with the check command for each.

When the harness itself breaks

The harness is software, and it fails like software. These are the failures that show up most often once a team relies on it.

Settings silently ignored in headless runs. In Claude Code, -p skips the workspace trust dialog, and “settings files that fail validation are silently ignored in this mode” (claude --help, 2.1.283). A typo in .claude/settings.json can remove your deny rules in CI, where no error dialog appears to warn you. In Codex, an untrusted project supplies no project AGENTS.md (0.150.0) and project hooks need persisted trust. Recovery: gate CI on claude doctor reporting no invalid settings and on a jq check that the deny rules are present (step 3), and make the first step of every headless job print the configuration it will run with: claude doctor output and jq . .claude/settings.json for Claude Code, codex doctor --json and cat .codex/config.toml for Codex.

The rules file grows until it contradicts itself. Every incident adds a line, nobody deletes one, and the agent follows whichever instruction it read last. Recovery: move enforceable rules to hooks and permissions, move procedures to skills, then prune the rest with the ablation protocol and your eval set.

A hook fails open, or blocks everything. A hook that crashes on unexpected input can let the action through; a hook with a broad matcher can block every edit. Recovery: keep fixture tests for every hook, fail closed only for security gates, and keep formatters separate from gates. See hooks as deterministic guardrails.

A deny rule is bypassed by a different spelling. The rule stops git push --force, and the agent runs a script that pushes. Recovery: move the boundary to the sandbox, network egress and credentials; see permissions, sandboxes and approval modes.

The default model changed under the same harness. On 2026-09-26, Claude Code’s stable channel (2.1.274) still gave Pro and Team Standard seats Sonnet 5 as the default, while the latest channel defaulted to Opus 5.5 from v2.1.280. Two developers with identical repository settings can run different models. Recovery: pin the channel with autoUpdatesChannel: "stable" and the allowed models with availableModels plus enforceAvailableModels in managed settings, and re-run the eval set before moving either. Current defaults are on the models hub.

A third-party skill, plugin or MCP server runs as you. Installing one hands it your shell, your files and your tokens. Recovery: install from a reviewed internal marketplace, pin versions, and read the code first. See skill security and MCP security.

You cannot tell whether the harness or the model is at fault. Recovery: rerun the task with customizations off. claude --safe-mode starts Claude Code with CLAUDE.md, skills, plugins, hooks and MCP servers disabled while managed policy still applies; in Codex, codex exec --ignore-user-config skips your personal config.toml. If the bare run succeeds, bisect the layers.

A good harness is necessary, not sufficient. Dex Horthy’s July 2026 talk at the AI Engineer World’s Fair is titled “Harness Engineering is not Enough: Why Software Factories Fail”. The harness makes runs predictable; the oracle, the evidence and the review still decide whether the result is right.

Where to go next with the harness

The harness pages are spread across five sections of the site. Read them in this order: this section first, then context, then the ecosystem layers, then the team and organization pages when more than one person shares the harness.

Layer	Start here	Then
Context	Context management	Concise AGENTS.md and CLAUDE.md · Shared agent rules
Tools	MCP servers	Scoped subagents · Internal MCP servers
Permissions and sandbox	Permissions, sandboxes and approval modes	Managed policy
Hooks	Hooks as deterministic guardrails	Shared hooks governance
Skills	Agent skills	The skills ecosystem · Shared skills
Plugins	Plugins	A team marketplace
Environments	Parallel agents in worktrees	Ephemeral environments · Agent sandboxes compared
Evaluation	Continuous evals	Evals for coding agents

Permissions, sandboxes and approval modes The next step: the recommended posture per risk class and per run type, and the prerequisite for any unattended run.

Making a codebase agent-ready The repository properties that let checks replace reading, with an audit prompt and a scoring rubric.

The operating model Who owns the harness, the gates and the evals once several teams share them.

Team collaboration: shared context and rules How a team keeps one harness instead of eight drifting copies.