Accessibility testing with AI agents: WCAG 2.2 AA gates in CI

Accessibility testing with AI agents combines a deterministic scanner (axe-core running inside Playwright), keyboard and focus tests written against roles and accessible names, and a short human screen-reader pass. The agent writes the tests and the fixes; a committed baseline, changed only with code-owner approval, blocks new critical and serious WCAG 2.2 A or AA violations in CI.

This page is for developers and tech leads who ship UI that agents now write. Your company has just received a demand letter about the signup form, and the audit finds inputs with no labels, a menu that traps the keyboard, and buttons whose contrast fails AA. The agent fixed the obvious ones in an afternoon. What you need now is proof that the next hundred agent-written pull requests do not put them back.

What you’ll walk away with from accessibility testing

• A Playwright + @axe-core/playwright gate that fails CI on new critical or serious WCAG 2.2 A/AA violations, with a baseline for the ones you already have.
• Keyboard and focus tests for dialogs, menus, and forms that break when an accessible name disappears.
• A browser MCP setup for Claude Code, Codex, and Cursor so the agent can audit states the scanner never reaches.
• Four copy-paste prompts: set up the gate, burn down the baseline, write keyboard tests, and prepare the human screen-reader pass.
• A sign-off split that says which checks are automatic, which a model grades, and which a person signs.

What can an automated accessibility check prove?

Split WCAG into what a machine decides and what a person decides before you ask an agent for anything. The split drives every gate on this page.

Check	Decided by	Gate
Missing labels, names, alt, invalid ARIA, contrast	axe-core rules (deterministic)	CI, blocking
Tab order, focus trap, focus return, Escape	Playwright keyboard tests	CI, blocking
JSX anti-patterns before the page renders	eslint-plugin-jsx-a11y or oxlint’s jsx-a11y rules	lint, blocking
Is the alt text meaningful? Is the error message clear?	a rubric judge	CI, advisory (model-graded checks)
Does the flow make sense with a screen reader?	a person with VoiceOver or NVDA	release sign-off, sampled

axe-core reports what it can decide. A page with zero violations can still announce “image” for every product photo, or read a checkout in an order nobody would follow. Treat a clean scan as “no known mechanical defects”, never as “accessible”.

Build the axe-core baseline gate

The gate runs axe-core on every critical route, compares the result with a committed baseline, and fails only on violations that are not in the baseline. That lets you switch the gate on today, on a codebase with hundreds of existing violations, and burn them down over time.

1. Install the scanner next to Playwright (@axe-core/playwright 4.13.0 and @playwright/test 1.63.0 on 2026-09-26):

   npm install -D @playwright/test @axe-core/playwright

2. Add the scan spec. It filters on the WCAG A/AA tags up to 2.2 and fingerprints each failing node by rule id and selector. The second assertion is the ratchet: it fails on baseline entries that no longer occur, so a fixed violation cannot stay in the file and mask the same regression later:

   tests/a11y/scan.spec.ts

   import { readFileSync } from 'node:fs';
   import { test, expect } from '@playwright/test';
   import AxeBuilder from '@axe-core/playwright';
   
   const ROUTES = ['/', '/signup', '/login', '/dashboard', '/settings'];
   const TAGS = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];
   const BLOCKING = new Set(['critical', 'serious']);
   const baseline: Record<string, string[]> = JSON.parse(
     readFileSync('tests/a11y/baseline.json', 'utf8'),
   );
   
   for (const route of ROUTES) {
     test(`no new WCAG A/AA violations on ${route}`, async ({ page }) => {
       await page.goto(route);
       const { violations } = await new AxeBuilder({ page }).withTags(TAGS).analyze();
       const found = violations
         .filter((v) => BLOCKING.has(v.impact ?? ''))
         .flatMap((v) => v.nodes.map((n) => `${v.id} ${n.target.join(' ')}`));
       const known = new Set(baseline[route] ?? []);
       expect(found.filter((f) => !known.has(f))).toEqual([]);
       const stale = [...known].filter((k) => !found.includes(k));
       expect(stale, 'remove fixed entries from baseline.json').toEqual([]);
     });
   }

3. Generate tests/a11y/baseline.json once from the current violations (the first prompt below does it), commit it, and review it as a list of debts, not as an approval.

4. Protect the baseline, the scan config, and the CI workflows. Add tests/a11y/, .github/, playwright.config.ts, and package.json to CODEOWNERS with your accessibility owner, and turn on Require review from Code Owners in branch protection, so an agent’s pull request cannot add baseline entries, change tags, or edit the gate without a human approval. The last two matter because the gate runs through them: a testIgnore: '**/a11y/**' in playwright.config.ts, or a package.json test script that skips tests/a11y, keeps the e2e job green while no accessibility spec runs. If your CI calls the test command from another file, protect that file too. The agent can still edit those files; without that setting, CODEOWNERS only requests a review and does not block the merge. The general pattern is in protecting the oracle.

5. Run the gate in the same CI job as your end-to-end suite. It reuses the dev server and the browsers that end-to-end testing already set up.

Two decisions in that spec matter more than they look.

Impact is not the WCAG level. axe-core’s impact rates harm to the user; every rule in the tagged set is an A or AA failure. Blocking only critical and serious is a starting ratchet. Once the baseline is empty, add moderate to BLOCKING.

Scan states, not just routes. axe-core checks the DOM that exists when you call analyze(). A closed menu, an unopened dialog, and a form before validation are all invisible to it. After you open a dialog, scan it on its own with new AxeBuilder({ page }).include('[role="dialog"]').withTags(TAGS).analyze().

Copy-paste prompt: set up the axe-core baseline gate

Set up an accessibility gate for this app with Playwright and @axe-core/playwright.

1. Read playwright.config.ts and reuse its baseURL and webServer. Do not start a second server.
2. Create tests/a11y/scan.spec.ts. For each route in ROUTES, run AxeBuilder with the tags
   wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa. Fingerprint each node of every critical or
   serious violation as "<rule id> <target selector>". Fail on fingerprints missing from
   tests/a11y/baseline.json (keyed by route) and on baseline entries that no longer occur.
3. Fill ROUTES from the router (src/app or src/pages); list the 5 routes with the most traffic
   first and ask me before adding more than 15.
4. Run the spec once with an empty baseline, then write the current fingerprints into
   tests/a11y/baseline.json, sorted, one array per route.
5. Do not call disableRules, exclude, or remove tags to make anything pass.
6. Finish by running npx playwright test tests/a11y and pasting the summary line, plus a
   table of baseline entries per rule id.

Add the lint half of the gate

The jsx-a11y rules catch some defects before a page renders: an <img> without alt, a click handler on a <div> with no keyboard handler, an invalid ARIA attribute. Two linters ship them.

With oxlint, the rules are built in. Append jsx-a11y to the plugins you already use in .oxlintrc.json:

{ "plugins": ["typescript", "unicorn", "oxc", "jsx-a11y"] }

Setting plugins replaces oxlint’s default set instead of extending it, so "plugins": ["jsx-a11y"] on its own silently switches off the typescript, unicorn and oxc rules (checked with oxlint --print-config on 1.81.0 and in the 1.86.0 configuration schema).

With ESLint, install the plugin and add its recommended flat config. eslint-plugin-jsx-a11y 6.10.2 (2026-09-26) declares a peer range that ends at ESLint 9, while ESLint itself is at 10.11.0, so on an ESLint 10 project a plain install stops with ERESOLVE. On ESLint 10, either use oxlint’s built-in rules above, or install with --legacy-peer-deps and run the lint job once to confirm the plugin loads:

npm install -D eslint-plugin-jsx-a11y                     # ESLint 9 or earlier
npm install -D eslint-plugin-jsx-a11y --legacy-peer-deps  # ESLint 10

eslint.config.js

import jsxA11y from 'eslint-plugin-jsx-a11y';

export default [
  // ...your existing config
  jsxA11y.flatConfigs.recommended,
];

Either way, the lint job must be a required check, as described in code quality gates.

Burn down the baseline with the agent

The baseline is a work queue. Give the agent one route or one rule at a time, and judge the result by the scan and the diff, not by reading every changed line.

Copy-paste prompt: fix one rule across the app

Fix every "label" and "button-name" entry in tests/a11y/baseline.json.

Rules:
- Fix the markup at the source component, not at each call site. If one component
  causes many entries, fix it once.
- Prefer a visible <label for> or visible text. Use aria-label only when there is no
  visible text to reference, and aria-labelledby when there is.
- Never add aria-hidden, role="presentation", or tabindex="-1" to make a rule pass.
- Never touch disableRules, exclude, the tag list, or add entries to the baseline.
- Remove the fixed fingerprints from tests/a11y/baseline.json.

Done means: npx playwright test tests/a11y passes, the baseline only shrank, and you
list each component you changed with the WCAG success criterion it now meets.

Contrast is a rule an agent tends to “fix” in the wrong place. When color-contrast fails across many pages, the cause is usually one design token. Tell the agent to change the token in your theme file and to stay within your palette, then check the change in a screenshot rather than approving a hex value by eye.

A cheap mechanical detector catches the other common cheat, silencing the scanner instead of fixing the page. CODEOWNERS cannot catch it, because it triggers on file paths and an aria-hidden can land in any component. So the job below fails on a hit, requests review from the accessibility owners, and passes only once one of them has approved the pull request:

.github/workflows/a11y-silencing.yml

name: a11y-silencing
on:
  pull_request:
  pull_request_review:
    types: [submitted]
permissions:
  contents: read
  pull-requests: write
jobs:
  silencing-detector:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0 # the default shallow clone has no base branch to diff against
          persist-credentials: false # gh reads the token from GH_TOKEN; keep it out of .git/config
      - name: Block hidden content and silenced axe without owner approval
        shell: bash # explicit bash adds pipefail: a failing pipeline fails the job
        env:
          GH_TOKEN: ${{ github.token }}
          PR: ${{ github.event.pull_request.number }}
          BASE: ${{ github.event.pull_request.base.ref }}
          A11Y_OWNERS: "alice bob" # GitHub logins of the accessibility owners
        run: |
          # a failed diff stops the job here (fail closed), instead of reading as "no hits"
          diff=$(git diff "origin/$BASE...HEAD" -- . ':!tests/a11y/baseline.json')
          hits=$(printf '%s\n' "$diff" \
            | grep -iE '^\+.*(disableRules|\.exclude\(|aria-hidden|role="presentation"|tabindex)' || true)
          [ -z "$hits" ] && exit 0
          echo "$hits"
          approvers=$(gh pr view "$PR" --json reviews \
            --jq '.reviews[] | select(.state == "APPROVED") | .author.login')
          for owner in $A11Y_OWNERS; do
            echo "$approvers" | grep -qx "$owner" && exit 0
          done
          gh pr edit "$PR" --add-reviewer "${A11Y_OWNERS// /,}"
          echo "::error::This change hides content or silences axe; an accessibility owner must approve it"
          exit 1

Make silencing-detector a required status check, and turn on Dismiss stale pull request approvals when new commits are pushed in branch protection, so an approval given before the agent’s next push does not carry over. The job needs fetch-depth: 0 on actions/checkout: the default checkout fetches one commit, so the base branch is missing and git diff fails. The script fails closed: a failed git diff or gh call stops the job red instead of reading as “no hits”. persist-credentials: false keeps the token out of .git/config: the gh calls read it from GH_TOKEN, so the checkout has no reason to store a token that can write to pull requests. A pull request from a fork gets a read-only token, so there gh pr edit fails and the owner has to be requested by hand.

The check is only as trustworthy as its workflow file. On pull_request, GitHub runs the workflow file from the pull request’s own merge commit, so an agent’s pull request can edit a11y-silencing.yml to exit 0 and the required check turns green. The control is the .github/ entry in CODEOWNERS with required code-owner review from step 4, which puts any change to the gate itself in front of the owner.

A hit is not always wrong: a decorative icon inside a labelled button should carry aria-hidden="true". The point is that a person decides, not the agent that wanted the build green.

Test keyboard access and focus with roles

Write keyboard tests with getByRole and accessible names. Such a test fails when a button loses its name, so it doubles as an accessibility assertion at no extra cost. getByTestId passes on a button nobody can identify.

tests/a11y/keyboard.spec.ts

import { test, expect } from '@playwright/test';

test('delete-account dialog traps focus and returns it', async ({ page }) => {
  await page.goto('/settings');
  const trigger = page.getByRole('button', { name: 'Delete account' });
  await trigger.focus(); // shortcut; see the stricter Tab loop below
  await page.keyboard.press('Enter');

  const dialog = page.getByRole('dialog', { name: 'Delete account?' });
  await expect(dialog).toBeVisible();
  await expect(dialog.getByRole('button', { name: 'Cancel' })).toBeFocused();

  for (let i = 0; i < 6; i++) {
    await page.keyboard.press('Tab');
    await expect(dialog.locator(':focus')).toHaveCount(1); // focus never leaves the dialog
  }

  await page.keyboard.press('Escape');
  await expect(dialog).toBeHidden();
  await expect(trigger).toBeFocused();
});

trigger.focus() jumps straight to the button, so this test still passes when the button is unreachable by Tab. For a trigger that must be reachable from the keyboard, replace that line with a bounded Tab loop from the top of the page:

for (let i = 0; i < 40 && !(await trigger.evaluate((el) => el === document.activeElement)); i++) {
  await page.keyboard.press('Tab');
}
await expect(trigger).toBeFocused(); // fails if 40 Tabs never reach it

Copy-paste prompt: keyboard tests for interactive components

Write Playwright keyboard tests in tests/a11y/keyboard.spec.ts for every dialog, menu,
combobox, and tab list in src/components/ui/.

For each component, follow the matching WAI-ARIA Authoring Practices pattern and assert:
- it opens with Enter or Space from the trigger, reached by Tab only (no mouse);
- focus moves into it, stays inside while open, and returns to the trigger on Escape;
- arrow keys, Home, and End move between items where the pattern requires it;
- focus is visible: the focused element's outline or box-shadow differs from its
  unfocused state.

Use getByRole with accessible names for every locator. No getByTestId, no CSS classes.
After each dialog or menu opens, run AxeBuilder scoped with include() to that element
using tags wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa.
If a test fails because the component is broken, leave the test failing, fix the
component, and tell me which one.

Component-level scans in Jest or Vitest (jest-axe, vitest-axe) are fast, but they run in jsdom, and the jest-axe README states that colour contrast checks do not work there. Keep contrast in the browser gate above.

Let the agent audit in a real browser

A scanner sees one DOM state; an agent with a browser can walk a flow, open every menu, fill the form wrong on purpose, and read the accessibility tree as a screen reader would. Playwright MCP (@playwright/mcp, 0.0.82 on npm on 2026-09-26) drives the browser through that tree. The setup differs per tool; the prompt after it is the same in all three.

Claude Code

# --isolated: an in-memory profile, so parallel agents do not fight over one browser
claude mcp add playwright -- npx @playwright/mcp@latest --isolated

Codex

codex mcp add playwright -- npx @playwright/mcp@latest --isolated

Cursor

Add the server to .cursor/mcp.json in the project:

{ "mcpServers": { "playwright": { "command": "npx", "args": ["@playwright/mcp@latest", "--isolated"] } } }

Caution

Accessibility snapshots are large, and every MCP tool schema sits in context. The Playwright MCP README itself says coding agents “might benefit from using the CLI+SKILLS instead” (@playwright/cli). Use the MCP server for exploratory audits and the committed spec for CI. The trade-off is covered in Playwright MCP versus Playwright CLI.

Copy-paste prompt: exploratory audit of the signup flow

Use the Playwright browser tools. Start at http://localhost:3000/signup.

Walk the flow using only the keyboard: Tab, Shift+Tab, Enter, Space, Escape, arrows.
1. Submit the form empty, then with an invalid email, then with a password that is too short.
2. After each step, take an accessibility snapshot and report:
   - every focusable element in tab order with its role and accessible name;
   - whether each error is tied to its field (aria-describedby or aria-errormessage)
     and whether it is announced (role="alert" or an aria-live region);
   - any element that is focusable but unnamed, or named but not focusable.
3. Open every menu, popover, and dialog you can reach and repeat the snapshot.

Do not change code. Output a table: step, element, problem, WCAG success criterion,
and a proposed Playwright assertion that would catch it. I will turn the table into tests.

The last line is the point: an exploratory finding is only worth something once it becomes a committed test that fails without the fix.

For a static review of UI code before it runs, the web-design-guidelines skill from Vercel Labs checks components against its Web Interface Guidelines, including accessibility and focus rules, and reports file:line findings. Install it with npx skills add vercel-labs/agent-skills --skill web-design-guidelines (skills CLI 1.7.0); see installing and managing skills and the other frontend skills.

Who signs off on an accessible release?

No one reads every agent diff for ARIA attributes. The evidence comes from the gates, and one person owns the parts machines cannot judge.

• CI, blocking: the axe baseline gate, the keyboard specs, and the jsx-a11y lint rules. A pull request that is red on any of them does not merge.
• Accessibility owner, on change: approves any edit to baseline.json, the scan spec, playwright.config.ts, package.json, or .github/ through CODEOWNERS with required code-owner review, and approves any pull request the silencing detector flags, which stays red until one of the owners has approved it.
• Rubric judge, advisory: grades new alt text and error messages against a rubric and flags weak ones for the owner. The authoring model does not grade its own output; see model-graded checks.
• A person with a screen reader, per release: walks the changed flows with VoiceOver or NVDA from a checklist the agent prepares, and records pass or fail.

Attach the axe JSON diff and the checklist result to the pull request or release as part of its evidence bundle, so an auditor or a legal team can see what was checked and when.

Copy-paste prompt: screen-reader checklist for the release

Compare this branch with origin/main and list every user-facing flow whose markup changed.
For each flow, write a screen-reader test script for a QA tester using NVDA with Firefox
and VoiceOver with Safari:
- the exact keys to press, step by step;
- what the screen reader should announce at each step (role, name, state), in quotes;
- the WCAG 2.2 success criterion each step checks.
Limit it to 20 minutes of manual work in total; put the flows with forms, dialogs, and
dynamic updates first. Save it as docs/a11y/release-checklist.md.

When accessibility gates break

The first scan reports hundreds of violations. Do not ask the agent to fix everything in one pull request; the diff becomes unreviewable and the gate is still off. Commit the baseline, switch the gate on, then burn down by rule. A handful of shared components usually accounts for most entries, so start with the rule that has the most entries.

The agent turns the build green by hiding the problem. aria-hidden, role="presentation", disableRules, and a quietly grown baseline all pass the scan; a fixed violation left in the baseline would hide its own return, which is why the scan also fails on stale entries. Recover by reverting those lines, keeping the baseline, the scan config, playwright.config.ts, package.json, and .github/ under CODEOWNERS with required code-owner review, and making the silencing detector a required check on every pull request.

The fingerprints churn after a harmless refactor. axe-core selectors follow the DOM, so a renamed wrapper changes the fingerprint, and the “new” violation is an old one. Regenerate the baseline in a dedicated pull request that the owner approves, and check that the count per rule did not grow.

color-contrast lands in incomplete, not violations. axe-core cannot decide contrast over background images, gradients, or overlapping elements, so it reports those nodes as needing review. The gate above ignores them; add the affected components to the screen-reader and visual checklist instead of assuming they pass.

The keyboard test passes but the component is still unusable. A test that tabs six times proves focus stays inside the dialog, not that the order makes sense. Take tab order from an exploratory audit snapshot and assert it explicitly for your highest-risk forms.

Accessible code does not match the design system. State the constraints in the prompt or in your agent instructions file: which tokens to use for focus rings, the minimum focus indicator, and which components are the approved accessible primitives. Fix contrast at the token, never with a one-off colour.

Where to go next with accessibility testing

End-to-end testing The Playwright suite this gate runs inside.

Protect the oracle Keep baselines and checks out of the agent's reach.

Model-graded checks Grade alt text and error copy with a calibrated rubric judge.

Code quality gates Run lint, types, and static analysis in the agent loop and as required CI jobs.

Mobile testing Accessibility checks on native and mobile surfaces.

Testing with agents overview The whole verification shelf in order.