Human in the loop: judgment at the gates, not in every diff

Human in the loop, for agent-written code, means a person makes four decisions: intent (is it worth doing), spec (what “done” means), evidence (did the checks prove it), and risk class (must someone read the code). The agent plans, builds, and checks its own work between those gates. You read diffs only in the escalation classes: auth, money, schema, migrations, and changes to the checks themselves.

This page is for developers who run agents every day and tech leads who have to make that repeatable. Here is the situation it answers. The agent rewrote your authentication module: 15 files, 800 lines, all green. You skimmed it and approved it, because it looked plausible. Two weeks later you find that the rewrite dropped a CSRF check. More careful reading would not have fixed that, because nobody reads 800 lines carefully at four in the afternoon. What was missing were two gates: no acceptance criterion said “cross-site POSTs are rejected”, and nothing routed an auth change to a named reader.

What judgment at the gates gives you

Four gates with the decision each one asks for, the artifact you look at, and a time budget, so you know where your attention goes.
A list of signals that justify interrupting the agent mid-run, and a longer list of things you leave alone.
Four copy-paste prompts: a spec with its assumptions exposed, a scope contract, an evidence packet with a risk class, and a mid-run redirect.
Two artifacts to commit: a CODEOWNERS block that enforces the risk-class gate and a stop-and-ask clause for AGENTS.md or CLAUDE.md.
The controls in Claude Code, Codex, and Cursor that hold each gate.
The ways gate-based review fails, and how to recover from each.

Why reading every diff is the wrong loop

Line-by-line review made sense when a person wrote the diff and review time scaled with writing time. An agent breaks that ratio. It writes faster than you read, so review becomes the bottleneck, and then it becomes a skim. A skim is an approval with a worse audit trail. The review-load data and the case for switching are on reading evidence instead of code. This page takes that switch as given and sets out where your judgment goes instead.

The opposite failure is approving every file write in real time. That keeps you busy without making you effective: you see each step, but never the whole change against what you asked for. The fix for both is the same. Put human decisions where only a human can make them, and let deterministic checks do the rest.

Which decisions stay human? The four gates

Each gate is one decision. The agent prepares the artifact; you decide. The time budgets are starting points for a medium-sized feature, not research findings.

Gate	The decision	What you look at	Budget	What goes wrong without it
1. Intent	Is this worth building, and what is out of scope?	One paragraph: the outcome, who it is for, the non-goals	2–5 min	The agent builds the wrong thing well
2. Spec	What does “done” mean, and which assumptions are guesses?	Acceptance criteria, an assumptions list marked `CONFIRMED` or `GUESS`, and the scope contract (files, dependencies)	10–20 min	Business rules get invented; scope creeps
3. Evidence	Did the checks prove each criterion, and could they have failed?	Spec delta, acceptance results, oracle changes, runtime signals	5–15 min	“Green” means the agent’s own tests agree with the agent
4. Risk class	Merge on evidence, or does a named person read the code?	The changed paths against the escalation classes	1 min to route; reading time where required	Auth, billing, or a migration merges unread

Spend the most time at gate 2. A wrong assumption caught there costs a sentence. Caught at gate 3 it costs a rerun, and caught in production it costs an incident. Gate 2 is also where the acceptance criteria become failing tests before the agent writes code, which is what makes gate 3 fast.

Gate 3 has its own page. Reading evidence instead of code covers what to read, in what order, and how to test whether the oracle could have failed. The agent pull request triage turns it into a 15-minute protocol.

Gate 4 is a routing decision, not a review. The escalation classes are authentication and authorization, money, schema, data migrations, changes to the oracle itself (tests, CI, lint, and type config), and anything no check covers. In those classes a named person reads the code in addition to the evidence, every time. Everywhere else, you approve on evidence. Keep the list short: a list that covers half the repository puts you back at Level 3.

Production is the last gate in the AI-native lifecycle, and the risk class decides how it is held. Low-risk classes ship behind progressive delivery with an automatic rollback. Escalation classes keep a person’s explicit approval.

Between the gates, let the agent run

Between gates your job is to watch for signals, not to supervise keystrokes. Interruption is cheap early and expensive late, so spend it only when one of these shows up:

The agent edits a file outside the scope contract.
It deletes code, or a test, without saying why.
It adds a dependency nobody discussed.
It skips, loosens, or suppresses a check (.skip, // @ts-ignore, a regenerated snapshot).
Its approach diverges from the approved plan.

Leave it alone when it works through the plan in order, touches the files you expected, and iterates on its own test failures. That loop is the agent checking its own work, which is the point.

The first four signals are mechanical, so do not rely on noticing them. A PreToolUse hook can block edits to protected paths, and a CI step can fail a pull request that modifies tests alongside the code they judge. See hooks as deterministic guardrails and protecting the oracle. What stays with you is the fifth signal, because only you know whether a divergence is a better idea or a misunderstanding.

Run one task through the four gates

The example adds per-user rate limiting to app/api/checkout/route.ts in a Next.js app. Each step names the gate and gives the prompt that prepares it.

Gate 1, intent: write it yourself. Two sentences are enough: “Stop one account from hammering checkout; legitimate buyers must never see a limit. Out of scope: IP-based limits and the admin API.” The agent cannot supply this, so do not ask it to.

Gate 2, spec: make the agent expose its guesses. Run the prompt below in plan mode (the controls for each tool are in the next section). Read the assumptions list first; correct every GUESS that matters, then read the plan.

Plan only; do not edit any file. Goal: add per-user rate limiting to
app/api/checkout/route.ts. Out of scope: IP-based limits and the admin API.

1. Assumptions: list every business rule you are guessing at (limit value,
   window, response when limited, identity source, behaviour when the
   limit store is unavailable). Mark each CONFIRMED (cite the file or doc)
   or GUESS.
2. Acceptance criteria: 4-8 observable behaviours, each one a sentence a
   test can check, including the error cases.
3. Plan: numbered steps, each small enough to verify on its own, with the
   command that proves the step is done.
Stop after the plan and wait for my corrections.

Gate 2, scope: approve a contract before any code. The contract turns “the agent went out of scope” from something you notice afterwards into something a check can catch.

Before you write or modify any code, give me a scope contract:

1. Every file you plan to create or modify, with one line on what changes.
2. Every new dependency, with the reason it is needed.
3. Every file on this list that touches auth, billing, schema, migrations,
   tests, CI, or lint config.

Wait for my approval. If you later need a file outside this list, stop
and ask before touching it. Never modify an existing test to make it pass;
report the failure instead.

Between gates: let it build. Approve the plan and switch to execution. The agent writes the tests from the acceptance criteria first, then the code, and runs the suite until it passes. You watch for the five signals above and otherwise leave it alone.

Gates 3 and 4, evidence and risk class: ask for the packet, not the diff. The prompt below asks for the four evidence artifacts and the routing verdict in one pass. The fuller version, with an oracle audit, is on reading evidence instead of code.

Do not edit any file. Report on this branch against main, in four parts:

1. Spec delta: each behaviour that changed, tied to the acceptance
   criterion that asked for it. List separately any change nobody asked for.
2. Acceptance results: criterion -> check (file:line or command) -> result.
   Run the checks; do not infer results. Mark a criterion with no check
   as UNVERIFIED.
3. Oracle changes: every test, snapshot, fixture, CI, lint, or type-config
   file this branch modifies, and whether the change is stricter, looser,
   or neutral.
4. Risk class: every changed file that matches auth, money, schema,
   migrations, or oracle paths, plus every UNVERIFIED criterion. End with
   one verdict: EVIDENCE (approve on evidence) or READ (list the files a
   named person must read, and the question to answer in each).

Decide. An EVIDENCE verdict with every criterion checked and no oracle loosened: approve. A READ verdict: the named reader reads those files, and only those, with the question the packet gave. Here the rate limiter touches the checkout path, which your team may classify as money. If it does, the reader checks one thing: can a limited request still create an order?

When the run drifts, correct it early. A nudge after the first file is cheaper than a rewrite after 15, and a clean restart beats a long argument in a polluted context.

Stop. Your approach no longer matches the approved plan: the plan puts the
limit state in our existing Redis client (lib/redis.ts), and you added an
in-memory store. Revert the changes since step 2, reread lib/redis.ts, then
tell me which plan step you are on and what you will change before you
continue.

Commit the gates, not just the habit

A gate that lives only in your head disappears on the day you are out. Two small artifacts make the gates enforceable for the whole team.

The risk-class gate belongs in CODEOWNERS, with branch protection set to require review from code owners. The forge then enforces the named reader instead of relying on memory. The paths below are starting points; adjust them to your repository.

# .github/CODEOWNERS: escalation classes need a named reader.
# Later lines win, so keep the escalation block at the end.
/src/auth/                 @acme/security-reviewers
/src/middleware/           @acme/security-reviewers
/src/billing/              @acme/payments-owners
/app/api/checkout/         @acme/payments-owners
/db/migrations/            @acme/data-owners
*.sql                      @acme/data-owners
/.github/workflows/        @acme/platform-owners
/.eslintrc*                @acme/platform-owners
/eslint.config.*           @acme/platform-owners
/tsconfig*.json            @acme/platform-owners
*.test.ts                  @acme/tech-leads
**/__snapshots__/          @acme/tech-leads
*.snap                     @acme/tech-leads

The between-gates signals belong in the instructions every session reads: AGENTS.md for Codex, and CLAUDE.md for Claude Code (which also reads AGENTS.md when a project has no CLAUDE.md, from v2.1.277 on the latest channel). Cursor reads its own Rules.

## Stop and ask

Stop and ask before you: edit a file outside the approved scope contract;
modify, skip, or delete an existing test, snapshot, CI step, or lint rule;
add a dependency; touch src/auth, src/middleware, src/billing,
app/api/checkout, or db/migrations. When a check fails, report the failure; do not weaken the check.

Where an organization writes these classes down once, for every repository, is the governance and autonomy policy.

How Claude Code, Codex, and Cursor hold each gate

The gates are identical across tools. What differs is the control that holds each one.

Gate 2 (spec): plan mode blocks edits while the agent plans. Enter it with /plan, cycle to it with Shift+Tab, or start with claude --permission-mode plan.
Between gates: from v2.1.283 (latest channel), interactive terminal and VS Code sessions start in auto mode, where a classifier reviews each action instead of asking you. claude -p runs still start in Manual. Pair either with a PreToolUse hook that blocks the protected paths in your stop-and-ask list.
Redirect: Esc stops the current action. Press Esc twice on an empty prompt, or run /rewind, to restore an earlier checkpoint.
Gate 3 (evidence): run the evidence-packet prompt in the session or headless with claude -p. /code-review gives a second opinion on the local diff.
Gate 4 (risk class): the managed Code Review (research preview, Team and Enterprise) reads review-only rules from a repository-root REVIEW.md, so put the escalation classes there. Its check run never blocks a merge; CODEOWNERS is the gate.

Gate 2 (spec): switch to Plan mode with /plan in the TUI, or start a planning session read-only with codex -c default_permissions=":read-only" (permission profiles are beta; --sandbox read-only is the legacy equivalent, so use one or the other).
Between gates: run execution with codex -c default_permissions=":workspace" -a on-request (the model decides when to ask you), or on its own with codex --approve-for-me (automatic review of approval requests on the legacy workspace-write sandbox). Pick one, as with gate 2: OpenAI says the profile and legacy sandbox systems do not compose (checked on codex-cli 0.157.1). The stop-and-ask clause in AGENTS.md covers what the sandbox cannot see.
Redirect: send the redirect prompt as a follow-up in the same session, or start a fresh session from a corrected plan when the context is cluttered.
Gate 3 (evidence): run the evidence-packet prompt with codex exec. codex review --base main gives a separate code review of the branch (checked on codex-cli 0.157.1).
Gate 4 (risk class): on GitHub, @codex review on a pull request runs the same kind of review; list the escalation classes in AGENTS.md so it names them. CODEOWNERS is the gate.

When gate-based review breaks, and how to recover

The spec gate becomes a rubber stamp. You approve plans without reading the assumptions, because the last ten were fine. Recovery: read the GUESS lines before anything else, and treat a plan with no GUESS lines as a red flag: ask the agent which CONFIRMED items it actually cited, and reject the plan if the citations are missing.

The escalation list quietly sprawls. Every incident adds a directory until half the repository needs a named reader and review is the bottleneck again. Recovery: change the list only in a reviewed pull request to CODEOWNERS, with the reason in the commit message, and remove a path when a stronger check covers it.

The evidence is a paraphrase of the diff. “Updated the handler to validate input” describes code; it proves nothing. Recovery: reject any acceptance line without a file:line or a command and its result, and make the agent run what it cites.

Tests change in the same pull request as the code. An agent that edits its own oracle can make every check agree with it. Recovery: route test, snapshot, CI, lint, and type-config files to a named reader in CODEOWNERS, and add the “never modify an existing test to make it pass” line to your instructions file.

A defect escapes in a class you approve on evidence. Recovery: write the check that would have caught it, add it to the oracle, and move that class back to code reading until it earns trust again. The per-class trust log on reading evidence instead of code is the mechanism; helping the team stop reading every diff rolls it out across a team.

You no longer understand the code you own. Gates move your attention to intent and evidence, and understanding can erode. Recovery: read one sampled pull request a week in full, in a class you own, after asking the agent to explain the design. The human’s job covers which responsibilities never transfer.

Where to go next with human judgment at the gates

Reading evidence instead of code Gate 3 in depth: what to read, in what order, and which changes still need a code reader.

Reviewing an agent's pull request The six-step triage that applies gates 3 and 4 to each pull request.

Executable acceptance criteria Gate 2 made executable: criteria become failing tests before the agent codes.

When to use agent vs ask mode The read-only-then-execute discipline behind the spec gate.

The human's job The six responsibilities that do not transfer to an agent.

Governance and autonomy Writing the risk classes down once, for every repository.