Agent-driven browser verification — prove the user flow

Agent-driven browser verification means the coding agent executes a changed user flow in a real, isolated browser, checks it against written acceptance criteria, and records the actions, assertions, console errors, and network failures it saw. Critical flows then become deterministic Playwright tests in CI, because an agent’s browser run is evidence for one change, not a regression gate.

This page is for the developer whose agent now builds most of the frontend. Unit tests pass, the pull request says “verified in the browser”, and two days later someone finds that the Save button shows a success toast while the name never reaches the database. Nobody can say what the agent clicked, on which account, or whether it reloaded the page.

Scorecard Q16: How do you verify visual and end-to-end (E2E) browser functionality?

Max-score answer: the agent runs an acceptance-criteria browser flow, captures reproducible evidence, and promotes critical paths into deterministic CI tests.

What you’ll walk away with from agent browser verification

• A choice between Playwright MCP, Playwright CLI with its skill, and agent-browser, and when context cost should push you to a CLI
• Setup for Claude Code, Codex, and Cursor that gives every agent an isolated, headless browser writing evidence to disk
• A browser scenario format that the agent executes and cannot quietly reinterpret
• Three copy-paste prompts: verify the flow, promote it into a Playwright test, and diagnose a failure
• A way to prove the new test catches the bug without reading it line by line
• The failure modes that turn browser checks into green theater, and how to recover from each

The organization-wide rule (which changes need which evidence, retention, protected tests) lives on the E2E policy page. Test-writing technique, such as page objects, authentication fixtures, and visual regression, lives on end-to-end test automation. This page is the developer’s loop between them.

How is Q16 scored, and what moves you up?

The scorecard awards zero to three points. Each level adds a control, not a tool.

Points	Your answer today	The next move
0	No browser testing; manual clicking after deployment	Write one browser scenario for the flow you changed last week and have the agent run it locally
1	Manual click-through locally after the agent finishes	Give the agent its own browser so the click-through happens, with evidence, before you look
2	Playwright or Cypress suite run through terminal commands	Tie each agent run to acceptance criteria and keep the evidence in the pull request
3	Acceptance-criteria browser flow, reproducible evidence, critical paths promoted into deterministic CI tests	Keep it honest: prove each new test fails without the fix, and protect it from silent edits

Which browser tool should the agent drive?

All three options below are deterministic drivers: the coding agent decides every click, and the tool reports what happened. Pick by how much context you can spend and by what your client supports.

Tool	Shape	Best for	Watch out for
Playwright MCP (@playwright/mcp, Microsoft)	MCP server over stdio; accessibility snapshots, screenshots optional	The default for all three tools; console and network tools built in	Tool schemas and page snapshots consume context on every step
Playwright CLI (@playwright/cli, Microsoft)	Shell commands plus an agent skill	Long sessions where context matters; the Playwright MCP README itself suggests it for coding agents	Needs a shell tool; the skill must be installed per workspace or globally
agent-browser (Vercel Labs)	Rust CLI plus a thin skill	Fast QA passes with short element references (@e2)	Short @eN refs and a thin skill keep context small, but it is a third tool to learn if your team already standardizes on Playwright

Claude Code and Codex also ship built-in browsers: Claude Code’s --chrome flag enables the Claude in Chrome integration (claude --help, v2.1.283), and Codex 0.157.1 ships browser_use and in_app_browser switched on (codex features list). Both drive a browser that can carry your own logged-in sessions. Use them to explore, and use an isolated browser for evidence you attach to a pull request.

For performance traces and Lighthouse audits, use Chrome DevTools MCP instead; the comparison is on browser MCP servers, and the MCP-versus-CLI trade-off is on Playwright MCP.

Connect an isolated browser to Claude Code, Codex, and Cursor

Run these in the repository root. The flags come from npx @playwright/mcp@0.0.83 --help: --isolated keeps the profile in memory, --headless needs no display, and --output-dir puts screenshots where the pull request can link them. A persistent profile serves one browser at a time, so parallel agents in separate worktrees need --isolated.

Claude Code

--scope project writes .mcp.json, so the team reviews and shares one pinned server.

claude mcp add --scope project playwright -- npx @playwright/mcp@0.0.83 \
  --isolated --headless --output-dir .evidence/browser
claude mcp list   # confirm the server connects

Codex

codex mcp add writes the server to ~/.codex/config.toml. This is a user-level entry, so every developer runs it once; the relative --output-dir resolves against the directory Codex starts in.

codex mcp add playwright -- npx @playwright/mcp@0.0.83 \
  --isolated --headless --output-dir .evidence/browser
codex mcp list    # confirm the entry

Cursor

Add a project .cursor/mcp.json in the shape the Playwright MCP README documents for Cursor, then enable the server in Cursor’s MCP settings.

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@0.0.83", "--isolated", "--headless", "--output-dir", ".evidence/browser"]
    }
  }
}

To use the CLI route instead, install it once and add its skill to the workspace. install --skills takes claude (the default) or agents, which writes to .agents/skills/.

npm install -g @playwright/cli@0.1.22     # checked with 0.1.22; not the deprecated unscoped "playwright-cli" package
playwright-cli install --skills claude    # writes .claude/skills/playwright-cli/; or: --skills agents

Add .evidence/ to .gitignore; screenshots belong in CI artifacts or the pull request, not in history.

Run the changed flow from its acceptance criteria

The agent verifies only what you wrote down. A vague criterion such as “settings work” lets it declare success after opening the page. Write the scenario before the agent starts building, next to the rest of the spec.

1. Add a browser scenario to spec.md. Name the starting state, the actions, the observable outcome, and the evidence you expect:

   Scenario: an authenticated user renames the workspace
   Given: a disposable local account "qa+rename@example.test" with workspace "Acme"
   When: the user opens /settings, enters "Northstar" in "Workspace name", and clicks Save
   Then: a "Saved" status appears, and "Northstar" is still shown after a full reload
   Evidence: numbered actions, accessibility snapshot after reload, screenshot,
             console errors, status of the save request

2. Seed a clean local or test environment. Never point an unattended browser agent at production data, real payments, or third-party accounts.

3. Ask the agent to execute the scenario exactly, with the first prompt below. It returns PASS, FAIL, or BLOCKED and an evidence block.

4. If the result is FAIL, switch to diagnosis with the third prompt. Do not let the same run edit code.

5. If the flow is business-critical (sign-in, checkout, permissions, destructive actions), promote it into a Playwright test with the second prompt.

Copy-paste prompt: verify the changed flow in a browser

Read the browser scenario "an authenticated user renames the workspace" in spec.md.
Start the dev server with the command documented in AGENTS.md and wait until it responds.
Using the Playwright browser tools, execute exactly the scenario's actions, in order.
After the save, reload the page and take an accessibility snapshot and a screenshot.
Collect console errors and the HTTP status of every non-GET request.

Return:
- RESULT: PASS, FAIL or BLOCKED
- a numbered list of the actions you performed
- each "Then" clause with observed evidence next to it
- console errors and failed requests (or "none")
- paths of the files you saved

Do not edit code, do not change the scenario, and do not report PASS for a clause
you did not observe in the browser. If the environment will not start, return BLOCKED.

A good result reads like this, and it goes into the pull request unchanged:

Browser verification: PASS
Environment: local dev server, fresh seed
Actions: 1 open /settings · 2 fill "Workspace name" = "Northstar" · 3 click Save · 4 reload
Then: "Saved" status seen after step 3 · "Northstar" in the field after step 4
Console/network: no errors · PATCH /api/workspace 200
Files: .evidence/browser/settings-after-reload.png

Promote a stable flow into a Playwright test

An agent’s browser run proves the change once, on one machine. Critical flows need a test that runs on every later change, including changes made by agents that never read this scenario. Keep the test small and assert the outcome the user cares about, not the implementation:

tests/e2e/settings-rename.spec.ts

import { test, expect } from '@playwright/test';

test('workspace rename persists after reload', async ({ page }) => {
  await page.goto('/settings');
  await page.getByLabel('Workspace name').fill('Northstar');
  await page.getByRole('button', { name: 'Save' }).click();
  await expect(page.getByRole('status')).toHaveText(/saved/i);
  await page.reload();
  await expect(page.getByLabel('Workspace name')).toHaveValue('Northstar');
});

Copy-paste prompt: promote the scenario into a Playwright test

Convert the passing scenario "an authenticated user renames the workspace" from spec.md
into one Playwright test in tests/e2e/settings-rename.spec.ts.

Rules:
- use getByRole, getByLabel or getByTestId locators; no CSS classes, no XPath
- assert the persisted outcome after a reload, not only the toast
- no waitForTimeout; rely on web-first assertions
- reuse the existing auth fixture instead of logging in through the UI

Then run: npx playwright test tests/e2e/settings-rename.spec.ts --repeat-each=5 --retries=0 --trace=on
Report the command and the pass count. If any run fails, fix the test or report the
product bug; do not add retries, sleeps or test.skip to make it pass.

Playwright also ships Test Agents: a planner, a generator, and a healer. npx playwright init-agents --loop=claude (with @playwright/test 1.63.0) writes three subagents to .claude/agents/, a playwright-test server entry to .mcp.json, a seed.spec.ts, and a specs/ folder for plans. --loop=codex writes the agents to .codex/agents/ instead. Version 1.63.0 has no Cursor loop (the choices are claude, codex, copilot, opencode, vscode, and vscode-legacy), so in Cursor use the MCP server above with the same prompts.

The healer can make a red test green by changing what it checks

The generated healer instructions allow “fixing assertions and expected values” and, when it is confident the test is right, marking a test test.fixme() so it is skipped. Both turn a real regression into a passing run. Review every healer diff that touches an expect line or adds fixme, and see how to protect the oracle from agent edits.

How do you know the browser check proved anything without watching it?

You do not need to read the agent’s transcript or every line of the test. You need four machine-checkable facts.

1. The test fails without the change. Commit the change, check out the base branch in a separate worktree, run the new test there, and confirm it goes red; your own checkout stays untouched. A test that passes on the old code does not protect the flow. Oracle strength covers this check in depth.
2. The test is stable. --repeat-each=5 --retries=0 passes five out of five. In CI, add --fail-on-flaky-tests so a retry can never turn a flaky test green.
3. The evidence traces to a criterion. Every “Then” clause in the evidence block has an observation next to it, and the file paths exist in the CI artifacts.
4. CI runs the test where the agent cannot edit it. The Playwright job runs on every pull request, and changes under tests/e2e/ need a named reviewer through CODEOWNERS.

Copy-paste prompt: prove the new test catches the bug

Prove that tests/e2e/settings-rename.spec.ts detects the bug it was written for.
The change is committed on this branch; the base branch is origin/main.
1. Run: git worktree add ../base origin/main   (a clean checkout of the old code)
2. Copy tests/e2e/settings-rename.spec.ts into ../base/tests/e2e/, install
   dependencies in ../base, and run the test there once. It must FAIL.
   Record the failing assertion.
3. Run: git worktree remove --force ../base   (this checkout was never touched)
4. In this checkout, run the test with --repeat-each=5 --retries=0. It must pass 5/5.
Report both outputs. If step 2 passes, the test does not cover the change:
say which assertion is missing and stop.

Sign-off stays with people and CI: the CI job is the merge gate, and the reviewer checks the evidence block and the red-then-green proof instead of replaying the flow by hand. The full pull request contract is the evidence bundle.

Copy-paste prompt: diagnose a failed browser scenario

The scenario "an authenticated user renames the workspace" returned FAIL.
Reset to a fresh seed and reproduce it once more with the Playwright browser tools.
Compare expected and observed behavior clause by clause. Inspect console messages and
the request and response of the save call. Identify the owning code path (file and
function). Stop after diagnosis: do not edit code until I confirm the cause.

What breaks in agent-driven browser verification?

Failure	What you see	Recovery
Green theater	PASS after opening the page, or a screenshot with no actions listed	Reject evidence without numbered actions and one observation per “Then” clause; the first prompt enforces this
Toast-only assertion	The success message appears, but the data never persisted	Add a reload or a second page to the scenario and assert the value there
Shared browser profile	Parallel agents fail to launch the browser, or one agent sees another’s login	Start every server with --isolated, or give each worktree its own --user-data-dir
Wrong dev server	The run passes against another worktree’s server on the default port	Pin each worktree’s port and have the prompt state the exact URL it must open
Healer weakens the test	A red test turns green after an expect change or a test.fixme()	Require review for any diff that touches assertions under tests/e2e/; rerun the red-then-green prompt
Flaky locator or timing	Passes three times, fails in CI	Replace CSS selectors and sleeps with role locators and web-first assertions; run --repeat-each before merge
Context exhaustion	Long sessions slow down or lose earlier instructions	Switch long flows to Playwright CLI, or split them into one scenario per session
Untrusted page content	Text on a page tells the agent to do something outside the scenario	Test only local or disposable environments; --allowed-origins narrows requests, but its own help says it “does not serve as a security boundary”

Where to go next with browser verification

Test: give the session a feedback loop The lifecycle stage this page belongs to: checks, definition of done, and a fresh-context verifier.

End-to-end test automation Page objects, authentication fixtures, visual regression, and cross-browser setup.

E2E policy for agent-built UI The team rule: evidence by risk class, retention, and protected tests.

agent-browser The Vercel Labs browser CLI and skill for quick QA passes.

Q17 · Continuous evals Turn observed agent failures into repeatable evaluation cases.

Q18 · Layered PR review Review behavior, security, and evidence before merge.