# Test: give the session a feedback loop

URL: https://developertoolkit.ai/en/method/lifecycle/test/
Description: Give every agent session one-command checks, a measurable definition of done, a Stop hook, a fresh-context verifier and evals for its own configuration.

The test stage gives every agent session a fast feedback loop that fails closed: one-command checks, a measurable definition of done, and a rule that the session cannot report success while a check is red. A fresh-context verifier and a CI re-run confirm the result, and evals regression-test the configuration that steers the agent.

This page is for the developer who runs agent sessions and the tech lead who owns the team's verification contract. The agent says "done, all tests pass". You pull the branch, run `make test`, and two tests fail. The session had never run them; it read the code and decided it worked. Now you are the test runner, pasting stack traces back into the chat.

**Traditional:** the signal that code works arrives late. CI reports minutes later, a tester days later, production weeks later. With an agent producing the code, a late signal means a person checks all of its output.

**AI-native:** the session checks its own work before a person sees it. It runs the tests, the build, and the screenshot comparison, reads the failures, and iterates until the checks pass. The person reads evidence, not every line.

The feedback loop and the verifier are two different things. The loop runs throughout the task. The verifier is one final check in a fresh context window, after the session believes it is finished.

## What a session feedback loop gives you

- A verification block for `CLAUDE.md` or `AGENTS.md` that the agent runs before it reports a task complete.
- Four copy-paste prompts: a definition of done, a bug reproduction, a visual check, and a fresh-context verification.
- A Claude Code `Stop` hook that stops a session from finishing while the gate is red, and the equivalents in Codex and Cursor.
- A headless verifier run for CI with `claude -p` or `codex exec`.
- An eval runner that tells you whether a change to instructions, skills, or hooks made the agent worse.

## Before you give the session a feedback loop

- A test suite and a build that each run locally with one command.
- For UI work, a browser tool the agent can drive, such as Playwright MCP.
- An accepted `plan.md` with proof commands, from [Build: plan.md, then implement](/en/method/lifecycle/build/). You can start the loop without one, but the verifier needs something to check the diff against.

## Which checks belong in the session loop?

Put a check in the loop when the agent can run it in seconds to a few minutes and read the result. Leave slow or shared checks to CI, where the agent cannot edit them.

| Check | Runs | Catches | Owner |
| --- | --- | --- | --- |
| Lint and type check on changed files | After every edit (hook) | Syntax, types, style drift | The session |
| Unit tests (fast project) | Before the session reports done | Broken behaviour in the changed module | The session |
| Build | Before the session reports done | Missing imports, bundling errors | The session |
| Screenshot or browser check | UI tasks, each iteration | Layout and interaction bugs | The session |
| Verifier subagent | Once, at the end | Unmet plan items, regressions in neighbouring code | A fresh context |
| Full suite, E2E, security scans | CI on every push | Everything the session skipped or weakened | CI, outside the diff |

## Put the verification commands in the instruction file

1. Wrap each check in a single target that exits non-zero on failure: `make test`, `npm test`, `pytest`. An agent cannot reliably tell a pass from a failure in a command that prints errors and exits 0.

2. Add a verification block to the project instruction file, with the healthy output of each command. Claude Code reads `CLAUDE.md`, Codex reads `AGENTS.md`, and Cursor reads project **Rules** and `AGENTS.md`.

   ```markdown
   ## Verifying your work
   - Build: make build (must finish with "Build succeeded")
   - Test: make test (all green; never skip or delete a failing test)
   - Lint: make lint (zero warnings)

   Run all three before reporting any task complete, and paste the
   output with each exit code. If a test fails, fix the code, not the test.
   ```

3. Break the gate on purpose once. Change one assertion so it must fail, run `make test`, and confirm it goes red. A gate you have never seen fail proves nothing. [How strong is your oracle?](/en/workflows/test/oracle-strength/) covers mutation testing for the same question at scale.

## Give the agent a definition of done it can check

A subjective target ("make sure it works") lets the agent stop when it feels finished. A measurable target lets it stop only when a command says so. Put the target in the prompt, next to the task.

| Subjective target | Measurable target |
| --- | --- |
| "Make sure the code looks good" | "`npm run lint` exits 0 with zero warnings" |
| "Handle the edge cases" | "Three new tests for signature failure, duplicate events and malformed payloads are green" |
| "Match the design" | "The screenshot at 1280 and 375 pixels matches `mocks/checkout.png`; list every difference" |
| "Don't break anything" | "All existing tests in `tests/billing/` still pass" |

**Copy-paste prompt: implement with a definition of done**

```text
Implement the Stripe webhook handler described in plan.md.

Definition of done:
1. All existing tests in tests/billing/ pass.
2. Three new unit tests cover signature verification failure,
   duplicate event IDs, and malformed payloads, and all three are green.
3. npm run typecheck reports zero errors.
4. npm run lint exits 0.

Run the checks yourself after each meaningful change and fix what fails.
Do not edit existing tests to make them pass. Before you report done,
paste the final output and exit code of each command. If a check cannot
pass for a reason outside the code (a service is down, a secret is
missing), stop and tell me instead of retrying.
```

## Make "done" impossible while a check is red

An instruction in `CLAUDE.md` is advisory; the agent can forget it in a long session. A hook runs every time. Wire the fast gate into the moment the agent tries to finish.

**Claude Code**

    A `Stop` hook fires when Claude finishes responding. Exit code 2 blocks the stop and feeds stderr back to Claude, so it keeps working on the failure. Add this to `.claude/settings.json`:

    ```json
    {
      "hooks": {
        "Stop": [
          {
            "hooks": [
              {
                "type": "command",
                "command": "cd \"$CLAUDE_PROJECT_DIR\" && make verify-fast 1>&2 || exit 2"
              }
            ]
          }
        ]
      }
    }
    ```

    `verify-fast` is a make target you add that runs only lint, the type check and the unit tests, for example `verify-fast: lint typecheck test-unit`; the full `make test` stays in CI.

    `Stop` takes no matcher. In Claude Code 2.1.283, a `Stop` hook that blocks eight times in a row is overridden and the turn ends; `CLAUDE_CODE_STOP_HOOK_BLOCK_CAP` changes that cap. A custom script can read `stop_hook_active` from the hook input and return success while it is true. [Test generation and TDD with Claude Code](/en/claude-code/test/testing/) adds a `PostToolUse` hook that runs only the tests related to the edited file.

**Codex**

    Codex CLI 0.157.1 has hooks with 12 events, including `Stop`, `PostToolUse` and `SubagentStop`. Codex reads `.codex/hooks.json` in a trusted project, in the same JSON shape as Claude Code, and runs the command in the session's working directory, so resolve the repository root yourself:

    ```json
    {
      "hooks": {
        "Stop": [
          {
            "hooks": [
              {
                "type": "command",
                "command": "cd \"$(git rev-parse --show-toplevel)\" && make verify-fast 1>&2 || exit 2",
                "timeout": 600
              }
            ]
          }
        ]
      }
    }
    ```

    Exit code 2 continues the turn with stderr as the next prompt. Project hooks run only after you trust them in `/hooks`, and an administrator can restrict a fleet to managed hooks with `allow_managed_hooks_only = true` in `requirements.toml`. For a longer task, `/goal` keeps Codex working toward a stated completion condition. [Use hooks as deterministic guardrails](/en/developer-scorecard-guide/agent-hooks/) has a complete `stop-gate.sh` for both tools.

**Cursor**

    Cursor **Hooks** are processes that exchange JSON over stdio and run before or after stages of the agent loop. Register a `stop` hook that runs the fast gate when the agent finishes; `loop_limit` caps how many times it can send the agent back.

    ```json title=".cursor/hooks.json"
    {
      "version": 1,
      "hooks": {
        "stop": [
          { "command": "node .cursor/hooks/verify-on-stop.mjs", "timeout": 600, "loop_limit": 3 }
        ]
      }
    }
    ```

    The script runs your gates and, on failure, returns the tail of the output as a `followup_message`; [Advanced Cursor techniques](/en/cursor-ide/automate/advanced-techniques/) has the complete `verify-on-stop.mjs`. Confirm the field names in Cursor's Hooks docs for your version before you copy them. Put the verification block where Cursor reads it: in a project rule (an `.mdc` file in `.cursor/rules`) or in `AGENTS.md`, which Cursor reads in the project root and in subdirectories, as it does `CLAUDE.md` (Cursor docs, checked 2026-10-12).

**Keep the stop gate fast**

The `Stop` hook runs after every response. Point it at lint, type check, and the unit project, and leave integration and end-to-end suites to CI. A slow gate makes every conversational turn wait for the whole suite.

## Start a bug fix from a failing test

A bug fix starts with a test that reproduces the bug and fails for the reason you expect. You commit that test before any application code changes. The fix run may not edit it. That order stops the most common way an agent "fixes" a bug: by weakening the check.

**Copy-paste prompt: reproduce the bug as a failing test**

```text
Bug: POST /api/refunds returns 500 when the order has a partial refund
already applied (see the stack trace below).

Do not change application code yet.
1. Write one test in tests/refunds/ that reproduces this bug.
2. Run it and confirm it fails with the same error as the stack trace,
   not with a setup or import error. Paste the failure output.
3. Explain in two sentences why it fails.
Stop there. I will commit the test before we fix anything.

<paste stack trace>
```

After you commit the test, start the fix with: "Make the test in `tests/refunds/` pass without editing any file under `tests/`." Then block test edits mechanically for the fix run. [Protecting the oracle](/en/workflows/test/protect-the-oracle/) has the deny rules, hooks, and `CODEOWNERS` setup, and [Test-driven development with AI assistance](/en/method/principles/test-driven-development/) covers the red-green loop for new features.

## Close the loop on UI with a browser

Unit tests do not see a misaligned button. For UI work, give the agent a browser, the mock, and a measurable comparison, and let it iterate: implement, screenshot, compare, adjust. Playwright MCP (`@playwright/mcp` 0.0.82 on npm, 2026-09-26) is Microsoft's official browser server.

**Claude Code**

    ```bash
    claude mcp add playwright -- npx @playwright/mcp@latest --isolated
    ```

**Codex**

    ```bash
    codex mcp add playwright -- npx @playwright/mcp@latest --isolated
    ```

**Cursor**

    Add the server to `.cursor/mcp.json`:

    ```json
    { "mcpServers": { "playwright": { "command": "npx", "args": ["@playwright/mcp@latest", "--isolated"] } } }
    ```

`--isolated` keeps the browser profile in memory. A persistent profile can be used by only one browser at a time, so parallel sessions in separate worktrees need it. Playwright's README also recommends `@playwright/cli` with skills for coding agents, because MCP tool schemas and page snapshots cost context. Install it with `npm install -g @playwright/cli@latest`, then run `playwright-cli install --skills` to copy its skill into the workspace. [Playwright MCP](/en/ecosystem/mcp/playwright-mcp/) compares the two, and [agent-browser](/en/ecosystem/skills/agent-browser/) is a separate browser CLI for agents if you want a second option.

**Copy-paste prompt: iterate against a mock**

```text
Implement the checkout summary card from mocks/checkout-summary.png.

Start the dev server with npm run dev. After each change, open
http://localhost:3000/checkout at 1280x800 and at 375x812, take a
screenshot, and compare it with the mock. List every visible difference
(spacing, font size, colour, alignment, missing element) before you fix
anything. Also report any console errors and failed network requests.

Stop when the list of differences is empty or only contains items you
cannot fix without new assets. Paste the final screenshots' paths and
the remaining differences.
```

## Finish with a verifier in a fresh context

The session that wrote the code is bound by its own assumptions; its "all tests pass" can mean "all the tests I thought of pass". A verifier subagent starts with a clean context, reads the diff and `plan.md`, runs the checks itself, and reports without fixing anything. Define it once and commit it, so every session uses the same verifier:

**Claude Code**

    A subagent is a Markdown file in `.claude/agents/`. The [Build](/en/method/lifecycle/build/) page has a complete `verifier.md`.

**Codex**

    Ask the session to spawn a subagent; multi-agent is on by default in 0.157.1, and `/subagents` switches between them. Keep the verifier's brief in `AGENTS.md` or a skill so every session sends the same one. [Codex multi-agent workflows](/en/codex/automate/multi-agent-workflows/) covers the settings.

**Cursor**

    **Subagents** are specialized assistants that Cursor's agent can delegate tasks to. Create one with the verifier's brief and keep the hand-off instruction in a project rule, so every session delegates the final check the same way.

Give the verifier read and run tools only (`Bash`, `Read`, `Grep` in Claude Code), and make its instruction adversarial: look for regressions in the neighbouring modules, not only the changed lines.

**Copy-paste prompt: hand off to the verifier**

```text
Before you report this task complete, launch the verifier subagent.
Give it only: the output of git diff origin/main...HEAD, plan.md, and the
verification commands from CLAUDE.md. Do not give it your reasoning or
your summary of what you did.

The verifier must run the commands itself, check every item in plan.md's
Proof section, and exercise the two modules that import the changed
files. It reports a table: plan item, command run, result, pass/fail.

If it reports any failure, fix it, then run the verifier again.
Paste its final table in your last message.
```

## Re-run the checks where the agent cannot edit them

The session's pasted output is a claim. CI re-running the same commands is the proof, but only if the agent cannot edit what CI runs. On the `pull_request` trigger GitHub runs the workflow file, the `Makefile` and every script they call from the pull request's own merge ref, so the same diff can weaken them. Put `/.github/`, the `Makefile` and the test scripts under a `CODEOWNERS` entry and require code-owner review in branch protection, so a change to the gate needs a human who owns it. Run the same `make` targets in CI, so the local gate and the merge gate cannot drift.

For a second opinion in CI, run the verifier headless with read-and-run tools only and a spend cap:

**Claude Code**

    ```bash
    claude -p "Run make test and make build. Compare git diff origin/main...HEAD \
    with plan.md and list any plan item with no passing proof. Do not edit files." \
      --permission-mode dontAsk \
      --allowedTools "Read" "Grep" "Bash(make test)" "Bash(make build)" "Bash(git diff *)" \
      --output-format json --max-budget-usd 2 > verifier.json
    ```

    Name each `make` target: `Bash(make *)` would also allow `make deploy` or `make clean`. `claude -p` starts in Manual permission mode (auto mode on third-party providers or with telemetry off, from v2.1.285, where a classifier decides). With `--permission-mode dontAsk`, a tool call that needs approval and is not in `--allowedTools` is denied instead of prompted, on every provider. `--max-budget-usd` works only with `--print`.

**Codex**

    ```bash
    # The gate: a red check fails the job here, before the agent starts.
    make test > test.log 2>&1 && make build > build.log 2>&1
    codex exec -c default_permissions=":read-only" --ephemeral -o verifier.md \
      "Read test.log and build.log. Compare git diff origin/main...HEAD with plan.md \
    and list any plan item with no passing proof. Do not edit files."
    ```

    The `:read-only` permission profile (beta in Codex 0.157.1) blocks writes, and a build writes output by definition, so CI runs `make test` and `make build` as ordinary steps and the read-only verifier only reads their logs, the diff and `plan.md`. In GitHub Actions, `openai/codex-action@v1` runs `codex exec` for you.

**Cursor**

    ```bash
    # The gate: a red check fails the job here, before the agent starts.
    make test > test.log 2>&1 && make build > build.log 2>&1
    agent -p "Read test.log and build.log. Compare git diff origin/main...HEAD with plan.md \
    and list any plan item with no passing proof. Do not edit files." \
      --mode ask --trust --output-format text > verifier.md
    ```

    `--mode ask` is the read-only mode that answers without editing files, so CI runs the checks as ordinary steps, as in the Codex tab. `--trust` answers the headless workspace-trust prompt; use it only on a checkout you trust. These flags come from Cursor's cookbook and public workflows, so confirm them in `agent --help` on your runner. [Scripting the Cursor CLI](/en/cursor-ide/automate/automation-workflows/) has the full GitHub Actions job, and Cloud Agents can run the same checks in an isolated VM.

**Do not hand secrets to an untrusted ref**

Run agent jobs on `pull_request` for branches in your own repository, with `permissions: contents: read` and `persist-credentials: false`. Never run an agent on a fork's code under `pull_request_target`: hooks, MCP servers, and instruction files from that checkout would run with your API key. [Deploy: review in both directions, gate production](/en/method/lifecycle/deploy/) covers the full CI job.

## Regression-test the harness with continuous evals

The code has tests; the configuration that steers the agent needs them too. A change to `CLAUDE.md`, `AGENTS.md`, a rule, a skill, a hook, or the model can make the agent worse at work it did well last week, and nothing in the product test suite notices. An eval suite is that test.

1. Collect 20 to 50 real tasks from recent work, each with the base commit and the accepted outcome. Include the ones the agent got wrong.

2. Write each task as a case: `prompt.md`, `base-commit`, and a `check.sh` that decides pass or fail with tests kept outside the worktree the agent runs in, so the agent under test cannot see or edit them. Keep the cases out of the base commits, or store them outside the repository.

3. Run the suite on every change to the harness files, and on a schedule to catch model and tool updates.

4. Gate the change on the pass rate. A skill edit that drops the rate gets reviewed before it merges.

5. Turn each production incident into a case, written by the team that owned the incident, and keep it as a regression test.

A minimal runner, `evals/run.sh`, run from the repository root:

```bash
#!/usr/bin/env bash
set -u
root=$(pwd); pass=0; total=0; mkdir -p evals/out
for case in evals/cases/*/; do
  id=$(basename "$case"); total=$((total + 1)); dir=$(mktemp -d)
  git worktree add --detach "$dir" "$(cat "$case/base-commit")" >/dev/null \
    || { echo "SKIP $id (bad base commit)"; total=$((total - 1)); continue; }
  # The base commit has the OLD harness: copy the version under test into it.
  cp -R CLAUDE.md .claude "$dir"/ 2>/dev/null
  (cd "$dir" && claude -p "$(cat "$root/$case/prompt.md")" \
     --permission-mode acceptEdits \
     --allowedTools "Bash(make test)" "Bash(make build)" "Bash(make verify-fast)" \
     --max-budget-usd 2 --output-format json > "$root/evals/out/$id.json")
  if bash "$case/check.sh" "$dir"; then pass=$((pass + 1)); echo "PASS $id"; else echo "FAIL $id"; fi
  git worktree remove --force "$dir"
done
echo "pass rate: $pass/$total"
[ "$total" -gt 0 ] || { echo "no cases ran"; exit 1; }
[ $((pass * 100 / total)) -ge "${EVAL_THRESHOLD:-90}" ]
```

For Codex, replace the agent line with `codex exec -C "$dir" -c default_permissions=":workspace" --ephemeral "$(cat "$root/$case/prompt.md")"` (permission profiles are beta in Codex 0.157.1) and copy `AGENTS.md` and `.codex/` instead. Run the suite several times per change: agents are not deterministic, and one run of one case is a sample, not a verdict. Cases that every configuration passes no longer discriminate; replace them with new ones from monitoring. [Continuous evals](/en/developer-scorecard-guide/continuous-evals/) covers scheduling and reporting, and [Maintain](/en/method/lifecycle/maintain/) covers turning incidents into cases.

## How do you know the feedback loop works without reading every line?

DORA's 2025 report (Google Cloud, 2025-09-23) found that "without robust control systems, like strong automated testing, mature version control practices, and fast feedback loops, an increase in change volume leads to instability". The loop is that control system for each session. Check it by its evidence:

- **Evidence arrives with the claim.** Every "done" message contains the pasted output and exit code of each verification command. A message without them is not done.
- **CI agrees with the session.** The first CI run on an agent's pull request is green. When the session said green and CI says red, the loop is broken; find out why before the next task.
- **Bug fixes keep their test.** The failing test was committed before the fix, and the fix diff touches nothing under the protected test paths.
- **The verifier's table is attached.** The pull request carries the verifier's plan-item table. [The evidence bundle](/en/workflows/ship/evidence-bundle/) defines the full set.
- **Harness changes carry an eval result.** A pull request that edits instructions, skills, or hooks shows the pass rate before and after.

**Who signs off.** The developer accepts the evidence for routine changes and reads the diff for intent and risk, not for compile errors. The tech lead owns the verification block, the stop gate, and the eval threshold, and reviews every change to them, because an agent that can edit the gate can weaken it.

**Leading indicators:** first-pass CI success rate for agent-written changes, and eval pass rate over time. **Lagging indicators:** review time per pull request, change failure rate, and regressions caught in CI versus in production. [Metrics for an AI-native SDLC](/en/method/lifecycle/metrics/) defines each one.

**Developer Scorecard, question 14**

Question 14 asks how the agent verifies its own work before a human reviews it. The top answer is enforced verification: measurable targets, automated loop iterations, and a verifier subagent running in a fresh context. This page is the setup for that answer.

## What breaks in a session feedback loop, and how to recover

**The agent reports done without running anything.** The verification block is in the instruction file, but the session skipped it late in a long context. Add the `Stop` hook, so finishing is impossible while the gate is red, and treat any "done" without pasted exit codes as not done.

**The agent edits the test instead of the code.** It deletes an assertion, adds a skip, or special-cases the test input. Commit the reproducing test first, block edits to test paths during fix runs, and have CI flag any fix diff that touches tests. [Protecting the oracle](/en/workflows/test/protect-the-oracle/) has the mechanics.

**The loop spins on something the code cannot fix.** A database is down, a secret is missing, or an external API rate-limits the tests. The agent retries or starts mocking production code. Put mock boundaries for external services in the instruction file, tell the agent to stop and report environment failures, and rely on the `Stop` hook cap to end the turn.

**The gate is green but proves nothing.** Tests assert nothing, or a health route returns 200 without touching the database. Break an assertion on purpose whenever the gate changes, and measure test strength with mutation testing.

**Flaky tests teach the agent to rerun until green.** The session reruns a failing test three times, it passes once, and the session reports success. Quarantine known flaky tests out of the session gate, and tell the agent to report a test that fails and then passes as flaky instead of calling it fixed.

**The verifier agrees with everything.** It received the implementing session's summary and inherited its assumptions. Give it only the diff, the plan, and the commands, and require it to run each command itself.

**Parallel sessions fight over the browser.** Two worktrees share one Playwright profile and one dev-server port. Use `--isolated` and assign each worktree its own port.

**The eval suite passes forever.** Every case is solved by every configuration, so the suite no longer tells configurations apart. Retire saturated cases and add new ones from recent failures and incidents.

## Where to go next after the test stage

- [Deploy: review in both directions](/en/method/lifecycle/deploy/) — Send the checked diff and its evidence through review and the production gate.
- [Protecting the oracle](/en/workflows/test/protect-the-oracle/) — Lock the tests and checks the agent must not edit.
- [The evidence bundle](/en/workflows/ship/evidence-bundle/) — What an agent's pull request must prove before review.
- [Reading evidence instead of code](/en/ladder/evidence-not-diffs/) — Why the reviewer's job moves from the diff to the evidence.
