Test-Driven Bug Fixing — Red Test First, Lock the Test

Test-driven bug fixing requires an AI agent to reproduce an identified defect with a failing automated test, preserve the accepted reproduction, and fix application code without weakening the test. Protecting the test reduces false-green repairs, but it does not prove the reproduction is complete; a human or independent review must still validate behavior and scope.

Scorecard question: How do you handle bug fixes with AI agents (test-first fixing)? Max‑score answer (3 pts): Strict test-first protocol: committed failing test, hook blocking test file edits during fix, verified green run.

Why this matters in 2026

When engineers prompt an agent with a stack trace and say “fix this bug,” the model frequently hallucinates success. Without a reproducible test, the agent modifies application logic in ways that hide the symptom or break adjacent workflows while declaring the problem solved. Worse, when asked to write tests alongside fixes, agents often adjust assertions to match their buggy implementation.

The test-first protocol enforces an empirical barrier: the agent must prove the defect exists by authoring a failing unit or integration test. Once the failing test is committed, the agent is restricted from modifying test files during the fix phase. The only acceptable outcome is application code changes that turn the failing test green.

What “max score” actually looks like

A max-score Q15 workflow follows three strict stages:

1. Reproduction first: The agent authors an isolated test reproducing the bug and runs it to demonstrate a non-zero exit code.
2. Committed red state: The failing test is committed to version control as evidence before any production code changes are introduced.
3. Locked test files: Build hooks or permissions prevent the agent from editing test files while fixing application code.
4. Verified green state: The test suite runs and passes cleanly with zero changes to the reproduction test.

Step-by-step implementation

1. Instruct the agent to write a reproduction test.

   Provide the bug report or error log and forbid modifications to source code:

   Claude Code

   Bug report: When a user changes their email, the session token is invalidated
   unexpectedly.
   Write a reproduction test in tests/auth/email_change.test.ts reproducing this.
   Run the test and confirm it fails. Do not modify any files in src/.

   Cursor

   In Agent mode, instruct:

   Reproduce issue #402 by adding a failing test in tests/unit/payment.test.ts.
   Run `npm test tests/unit/payment.test.ts` and verify it fails with exit code 1.
   Do not touch src/ yet.

   Codex

   Add an integration test reproducing the race condition described in INCIDENT_REPORT.
   Confirm the test fails under `pytest tests/test_concurrency.py`.
   Do not alter application code.

2. Commit the failing test.

   Commit the reproduction test as a standalone commit:

   git add tests/auth/email_change.test.ts
   git commit -m "test: reproduce unexpected session invalidation on email change"

3. Enforce test file locking during the fix.

   Prevent the agent from altering test files to force a green result. Use a pre-tool hook or permission setting:

   # Example .claude/hooks/protect-tests.sh or git pre-commit hook
   #!/bin/bash
   # Blocks edits to test files when fixing bugs
   changed_files=$(git diff --name-only)
   if [[ "$FIX_MODE" == "true" ]] && echo "$changed_files" | grep -q "^tests/"; then
     echo "Error: Modifying test files during a bug fix is forbidden." >&2
     exit 2
   fi

4. Task the agent with the fix.

   Prompt the agent to make the failing test pass:

   The reproduction test is committed in tests/auth/email_change.test.ts.
   Fix the issue in src/auth/ to make the test pass.
   Do not edit tests/auth/email_change.test.ts. Run the test suite to prove the fix.

5. Verify the clean green run.

   Confirm that the test passed, no other tests regressed, and the diff only modifies application files.

Common pitfalls

• Simultaneous test and fix: Allowing the agent to write the fix and the test in the same prompt. The agent will often write a test that asserts its own buggy behavior.
• Assertion dilution: When an agent encounters an edge case during fixing, it softens the test assertion (e.g., changing toBe(expected) to toBeTruthy()) instead of solving the logic bug.
• Ignoring adjacent tests: Fixing the reproduction test while breaking 3 existing tests. Always require a full regression suite run before finalizing the task.

How to verify you’re there

• Bug fix pull requests contain at least two commits: one adding the failing test, and subsequent commits providing the fix.
• Git diffs on bug-fix tasks never show relaxed assertions in existing test files.
• Hooks automatically block edits to tests/** when fix mode is active.
• Escaped bug regressions drop as reproduction tests permanently join the regression test suite.

Further reading

AI-Native SDLC Test Stage Setting up test loops, regression suites, and continuous evals.

Q14 · Session Feedback Loop How to build self-correcting agent test loops.

Q17 · Continuous Evals Turn production incidents into permanent regression evals.