# AI Code Review Bots Compared: Claude Code Review, Codex, Bugbot, CodeRabbit, Greptile and More

URL: https://developertoolkit.ai/en/ecosystem/observability/ai-code-review-bots/
Description: Which AI code review bot runs where: Claude Code Review, codex review, Copilot, Bugbot, CodeRabbit, Greptile and ocr on trigger, config and cost per PR.

AI code review bots are second models that read a diff and post findings before a human approves it. The main options in September 2026 are Claude Code Review, `claude-code-action`, `codex review`, GitHub Copilot code review, Cursor's Bugbot, CodeRabbit, Greptile and Alibaba's open-source `ocr`. No single bot is enough: run them in layers and measure each on precision and cost per pull request.

Your team merges more pull requests every month, most of them written by agents, and review has become the queue everyone waits in. Someone installs a review bot, it posts 30 comments on the first pull request, and within a week people resolve its threads without reading them. This page is for developers who want a second opinion they can trust before pushing, and for tech leads who have to choose, configure and pay for the bots the whole team lives with.

## What you'll get from this review bot comparison

- **A comparison table** of eleven review options: where each runs, what triggers it, which file tunes it, and what a review costs.
- **A four-layer review workflow** (local self-review, second model, pull request bot, human) with the order and the cost of each layer.
- **The `claude-code-action@v1` review workflow** adapted from Anthropic's docs, a `REVIEW.md` that cuts noise, and a workflow that turns the neutral check run into a required status check.
- **The equivalent wiring for Codex and Cursor**, and install commands for Greptile, CodeRabbit and `ocr`.
- **Six copy-paste prompts**, four metrics that prove a bot earns its cost, and the traps that make each tool misfire.
- **An eval set built from your incident and revert pull requests**, a noise measure, a verification pattern and a one-repository rollout.

This page compares the bots. The procedure a human follows on an agent's pull request is [reviewing an agent's pull request without reading every line](/en/workflows/ship/agent-pr-review/); read that first if you have not set up a review checklist yet.

## Which AI code review bot fits which job?

Every tool below was checked on 2026-09-26 against the vendor's docs, its GitHub repository or npm, or the installed CLIs (Claude Code 2.1.283, Codex CLI 0.157.1). Rows marked **secondary** rest on search extracts or on a check dated 2026-08-28, not on a vendor page read that day; confirm them on the vendor's site before you roll them out. The Claude Code Review row and the Claude Code tab were re-read against the Claude Code docs fetched on 2026-10-16.

| Tool | Runs where | Trigger | Tuned by | Cost per review | Evidence |
| ---- | ---------- | ------- | -------- | --------------- | -------- |
| **Claude Code Review** (Team and Enterprise) | Anthropic's infrastructure, GitHub | PR open, every push, or `@claude review` | `REVIEW.md`, `CLAUDE.md` | Billed as usage credits; the launch announcement (9 March 2026) said reviews "generally average $15–25", and your own average is in Admin settings | Vendor docs and launch announcement |
| **`/code-review`** (any plan) and **`claude ultrareview`** | Your terminal; ultrareview in Anthropic's cloud | You run it | `CLAUDE.md` (not `REVIEW.md`) | Normal usage; ultrareview: 3 free runs on Pro and Max, then "typically $5 to $25" | Vendor docs, CLI |
| **`claude-code-action@v1`** | Your GitHub Actions runner | Workflow events you choose | The prompt, the plugin, `CLAUDE.md` | Tokens on your API key or subscription | Vendor docs |
| **`codex review`** and **`openai/codex-action@v1`** | Your terminal or runner | You run it; workflow events | Custom instructions, `AGENTS.md` | Codex usage limits or API tokens | CLI help, action README |
| **`@codex review`** on GitHub | OpenAI's cloud | PR comment | `AGENTS.md` | Codex plan | **Secondary** (checked 2026-08-28) |
| **GitHub Copilot code review** (paid Copilot plans; not Copilot Free) | GitHub; its agentic context gathering runs on Actions runners | Request Copilot under **Reviewers**, or an automatic-review setting or ruleset | `.github/copilot-instructions.md`, `.github/instructions/*.instructions.md`, `AGENTS.md` | AI credits: an estimated $0.05–1 per review at Lite effort, $0.25–5 at Balanced, plus Actions minutes | github/docs |
| **Cursor Bugbot** | Cursor's cloud, GitHub and GitLab | PR open or update, `bugbot run` | `.cursor/BUGBOT.md` | Usage-based; Cursor says the average run costs $1.00 to $1.50 (checked 2026-10-12) | Cost: Cursor docs and blog; trigger and `BUGBOT.md`: **secondary** |
| **CodeRabbit** (app and CLI) | Vendor cloud; CLI locally | PR; `coderabbit review` | Vendor config | Not verified | CLI flags verified from CodeRabbit's own skill |
| **Greptile** (app and CLI) | Vendor cloud; CLI locally | PR; `greptile review` | `--instructions` | Not verified | npm `greptile` 3.6.0 README |
| **Alibaba Open Code Review (`ocr`)** | Your terminal or runner, your model | You run it | Rule templates | Your model's tokens | GitHub README, npm 1.12.9 |
| **Graphite Agent** (formerly Diamond, Cursor-owned) | Graphite's cloud | PR | Vendor config | Not verified | **Secondary** |

Three differences matter more than the feature lists:

- **Who pays and how.** The managed bots bill per review or per seat; the CLI and Actions routes bill tokens against a key you already own. Token prices live on the [models hub](/en/ecosystem/models/), not here.
- **Whether it can block a merge.** A Claude Code Review review "never approves or blocks the pull request", and its check run never concludes as a failure (Claude Code docs, read 2026-10-16). Copilot leaves a "Comment" review by default, which does not count toward required approvals; only with Copilot approvals (public preview) switched on can it approve. A gate is something you build from the bot's output: require conversation resolution on the branch, or read the severity count in CI as shown below.
- **What the bot reads for rules.** Each bot has its own file. A rule you put in `.cursor/rules` never reaches Bugbot, and a rule in `REVIEW.md` never reaches the local `/code-review`.

## How does layered AI code review work?

One bot run by the same model that wrote the code shares that model's blind spots. Layers work because each one catches a different class of problem at a different price, and each one is cheaper than the next. The tool-specific setups are in [code review with Codex](/en/codex/ship/code-review/) and [automated code reviews in Claude Code](/en/claude-code/ship/review-automation/); the team-level review contract, with focused passes for correctness, security, tests and spec compliance, is in [govern layered pull-request review](/en/developer-scorecard-guide/layered-pr-review/).

1. **Local self-review, before you push.** Run the author tool's own review in a fresh context: `/code-review` in Claude Code, `codex review --uncommitted` in Codex, or the Cursor agent with the first prompt below. It takes seconds to minutes within your normal usage and removes the obvious bugs before anyone else sees them.

2. **A second model, still local.** Ask a different vendor's model for an adversarial pass: `/codex:review` or `/codex:adversarial-review` from Claude Code, `greptile review`, `coderabbit review --agent`, or `ocr review`. A different model family disagrees with the author for reasons of its own, which is the point.

3. **The pull request bot, once per pull request.** One bot owns the pull request: Claude Code Review, a `claude-code-action` workflow, `@codex review`, Copilot code review, or Bugbot. It posts inline comments with severities, and CI reads its output to decide whether the merge is blocked. Review on every push is an opt-in for high-risk pull requests, because it multiplies the cost.

4. **A human, on escalation classes only.** A named person reads the code for authentication, payments, migrations and anything else on your escalation list, and signs off on everything else from the evidence. The escalation list and the verdicts are in [agent PR review](/en/workflows/ship/agent-pr-review/).

**What a layered review costs per pull request.** Layers 1 and 2 run on usage you already pay for. Layer 3 is the line item: with Claude Code Review at the $15–25 average its launch announcement (9 March 2026) gave, a team merging 200 pull requests a month with one review each spends roughly $3,000–5,000 a month, and "after every push" mode multiplies that by the pushes per pull request. That is arithmetic on Anthropic's average, not a measurement, so read the per-repository average cost in the admin settings after the first month. A `claude-code-action` or `codex-action` workflow bills only tokens plus runner minutes, and you set the model and effort, so it is the cheaper layer 3 when you can live without the managed verification step.

## Set up a pull request review bot in each tool

The workflow above is the same in every tool. The wiring differs, so pick your tab.

**Claude Code**

    **Managed Code Review (Team and Enterprise).** An organization Owner enables it at `claude.ai/admin-settings/claude-code`, installs the Claude GitHub App, selects repositories, and chooses a trigger: once after PR creation, after every push, or Manual. To verify setup, open a test pull request; a check run named **Claude Code Review** appears within a few minutes. On a Manual repository, comment `@claude review` for one review, or `@claude review always` to subscribe the pull request to later pushes. It is not available to organizations with Zero Data Retention or with the HIPAA configuration.

    **Your own workflow, on any plan with an API key or OAuth token.** This workflow is adapted from Anthropic's GitHub Actions docs (we add `persist-credentials: false`). Save it as `.github/workflows/claude-code-review.yml`:

    ```yaml
    name: Code Review
    on:
      pull_request:
        types: [opened, synchronize, ready_for_review, reopened]
    jobs:
      review:
        runs-on: ubuntu-latest
        permissions:
          contents: read
          pull-requests: read
          issues: read
          id-token: write
        steps:
          - uses: actions/checkout@v7
            with:
              fetch-depth: 1
              persist-credentials: false
          - uses: anthropics/claude-code-action@v1
            with:
              anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
              plugin_marketplaces: "https://github.com/anthropics/claude-code.git"
              plugins: "code-review@claude-code-plugins"
              prompt: "/code-review:code-review --comment ${{ github.repository }}/pull/${{ github.event.pull_request.number }}"
              claude_args: '--allowedTools "mcp__github_inline_comment__create_inline_comment"'
    ```

    Two lines decide where the review goes. `--comment` posts inline comments on the pull request; without it, findings stay in the run log. Keep the `claude_args` line: the action starts the inline-comment MCP server only when `--allowedTools` names it. The workflow skips drafts, closed pull requests and pull requests Claude already commented on. The job triggers on `pull_request`, not `pull_request_target`, so on public repositories GitHub withholds `ANTHROPIC_API_KEY` from fork pull requests and forks get no review. `persist-credentials: false` keeps the `GITHUB_TOKEN` out of `.git/config` while the agent runs on the pull request's code. To authenticate with a subscription instead, run `claude setup-token` locally and pass the token as `claude_code_oauth_token`.

    **Local layers.** `/code-review` reviews your branch and uncommitted changes as a background subagent; `/code-review high` widens coverage, `low` reports only the most confident findings, `--fix` applies findings, and `--comment` posts them to a pull request. For a deep pass before merge, `claude ultrareview 482` reviews pull request 482 in the cloud, where "every reported finding is independently reproduced and verified"; `--json` prints the raw payload.

**Codex**

    **Local review (Codex CLI 0.157.1).** `codex review` takes either one target flag or custom instructions, never both. In 0.157.1, passing `--uncommitted`, `--base` or `--commit` together with a prompt fails at parse time with `the argument '--base <BRANCH>' cannot be used with '[PROMPT]'`:

    ```bash
    # Terminal, from the repository root (Codex CLI 0.157.1)
    codex review --uncommitted
    codex review --base main
    codex review --commit 3f9c2ab
    # No target flag: the instructions alone drive the review, so name the scope in them
    codex review "Review the changes on this branch against main. Focus on auth and SQL injection."
    ```

    A `-` in place of the prompt reads the **instructions** from standard input, not a diff, and the same no-target rule applies. For CI, `codex exec review` takes the same targets with the same restriction, plus `--json`, `--output-schema FILE` and `-o FILE`, which writes the final message to a file a later step can post or parse:

    ```bash
    # CI step, after checking out the pull request with full history (Codex CLI 0.157.1)
    codex exec review --base origin/main -o review.md
    ```

    **Your own workflow.** `openai/codex-action@v1` installs the CLI and runs `codex exec` with your prompt; its README ships a complete pull request review example that posts the final message as a comment. For a review job, set `permission-profile: ":read-only"`; the action refuses to start if you also set `sandbox`, because "the profile and legacy sandbox systems do not compose". Since v1.13 (2026-10-05), Linux `drop-sudo` requires `setfacl` (the `acl` package) and ACL support on `/run` (`openai/codex-action` CHANGELOG, checked 2026-10-12); see [headless agents in CI](/en/ecosystem/agent-tools/headless-agents-ci/).

    **On GitHub (secondary).** OpenAI's docs, checked 2026-08-28, describe enabling Codex code review for a repository and then commenting `@codex review` (or `@codex security review`) on a pull request, with custom rules in `AGENTS.md`. Re-check the setup on OpenAI's site before rolling it out.

**Cursor**

    **Bugbot (secondary).** Cursor describes Bugbot as reviewing "pull requests" and identifying "bugs, security issues, and code quality problems" (cursor.com, checked 2026-10-12). According to search extracts of Cursor's help pages, you connect GitHub or GitLab under the Cursor dashboard's integrations, enable Bugbot per repository, and it reviews on every pull request open and update. Cursor's docs now say Bugbot is configured in Automations (checked 2026-10-12), so those steps may not match the current setup; follow Cursor's Bugbot page. Comment `bugbot run` to re-run it. Project rules for Bugbot go in `.cursor/BUGBOT.md`; `.cursor/rules/*.mdc` files do not apply to it (Cursor's docs say rules do not apply to Bugbot PR reviews, checked 2026-10-12). Cursor moved Bugbot from a $40-per-seat subscription to usage-based billing for Teams and Individual plans (Cursor blog, 2026-05-11); effort levels (Low, Default, High, Smart) exist only on usage-based Bugbot plans, `@cursor remember [fact]` on a PR teaches it a rule, and Autofix spawns a Cloud Agent to fix what a review found (Cursor docs, checked 2026-10-12). On Teams and Enterprise, Cursor's changelog of 2026-09-23 also lists Security Review, which posts one comment per pull request reporting exploitable bugs, and Rollouts, which monitors each pull request from review to production; this page does not compare them, and [code review in Cursor](/en/cursor-ide/ship/code-review/) covers Cursor's own bots.

    **Second model from Cursor.** In Cursor, the quickest second model is a new agent chat on a model from a different vendor than the one that wrote the change, given the adversarial prompt below. For a reviewer outside Cursor, use a CLI: `greptile skills install` writes Greptile's skills to `.agents/skills/`, which Cursor reads, and `greptile review` runs in Cursor's terminal. `npx skills add coderabbitai/skills` installs CodeRabbit's review skill the same way.

    **Graphite Agent (secondary).** Graphite's AI reviewer, formerly called Diamond, is reported as owned by Cursor since December 2025. Its stacked-PR CLI is npm `@withgraphite/graphite-cli` (1.8.6); there is no `@graphite/cli`.

### Tune Claude Code Review with a `REVIEW.md`

`REVIEW.md` sits at the repository root and reaches every agent that finds and verifies findings, so a rule there lands more reliably than the same rule in a long `CLAUDE.md`. Anthropic lists seven patterns; the four that cut the most noise are redefining Important, capping nits, skip rules and repository-specific checks, plus a re-review rule so a pull request converges instead of collecting new nits on every push. This version, adapted from Anthropic's example, is a good first commit:

```markdown
# Review instructions

## What Important means here
Reserve Important for findings that would break behavior, leak data,
or block a rollback: incorrect logic, unscoped database queries, PII in
logs or error messages, and migrations that aren't backward compatible.
Style, naming and refactoring suggestions are Nit at most.

## Cap the nits
Report at most five Nits per review. If you found more, say "plus N
similar items" in the summary. After the first review of a PR, post
Important findings only.

## Do not report
- Anything CI already enforces: lint, formatting, type errors
- Generated files under `src/gen/` and any `*.lock` file

## Always check
- New API routes have an integration test
- Log lines don't include email addresses, user IDs or request bodies
- Database queries are scoped to the caller's tenant
```

Keep it short. Anthropic's own guidance is that "a long `REVIEW.md` dilutes the rules that matter most". The service reads `REVIEW.md` from the base or default branch and never from the pull request's own changes, so a change to the file applies only after it merges (Claude Code docs, read 2026-10-16).

### Turn the neutral check run into a merge gate

The last line of the **Claude Code Review** check run's details is a machine-readable severity count. Anthropic's launch announcement (9 March 2026) put the average Code Review at around 20 minutes, so the gate has to wait for it rather than poll once. This workflow runs when the check run completes and posts a commit status on the reviewed commit: `success` when the Important count is zero, `failure` otherwise. Save it as `.github/workflows/claude-review-gate.yml` on the default branch:

```yaml
name: Claude review gate
on:
  check_run:
    types: [completed]
permissions:
  checks: read
  statuses: write
jobs:
  gate:
    if: github.event.check_run.name == 'Claude Code Review'
    runs-on: ubuntu-latest
    steps:
      - name: Turn the severity count into a commit status
        env:
          GH_TOKEN: ${{ github.token }}
          REPO: ${{ github.repository }}
          RUN_ID: ${{ github.event.check_run.id }}
          SHA: ${{ github.event.check_run.head_sha }}
        run: |
          # The severity line is JSON like {"normal": 2, "nit": 1, "pre_existing": 0}; "normal" = Important
          important=$(gh api "repos/$REPO/check-runs/$RUN_ID" \
            --jq '.output.text | split("bughunter-severity: ")[1] | split(" -->")[0] | fromjson | .normal') || important=""
          if [ "$important" = "0" ]; then state=success; desc="No Important findings"
          else state=failure; desc="Important findings: ${important:-unknown}"; fi
          gh api "repos/$REPO/statuses/$SHA" -f state="$state" \
            -f context="claude-review-gate" -f description="$desc"
```

To gate on threads instead, require conversation resolution on the branch: resolving a thread clears a finding and a reply alone does not (Claude Code docs, read 2026-10-16). To gate on the count, mark `claude-review-gate` as a required status check in branch protection or a ruleset; a check that fails but is not required blocks nothing. The workflow posts a commit status instead of relying on its own job result because, per GitHub's docs, a `check_run` workflow runs against the default branch, so its job never appears on the pull request. Until the status arrives, the required check shows as expected and the merge stays blocked, which also covers a review that is still running. On a Manual repository, nothing is posted until someone comments `@claude review`.

A commit status belongs to one commit and does not carry forward. If the author pushes after the review, the gate on the new tip stays pending and the merge stays blocked until that commit is reviewed. That is the intended behaviour, because an unreviewed commit should not merge, but plan for it: comment `@claude review` on the final commit before merging, or use `@claude review always` (or the repository's review-on-every-push trigger) on pull requests that change after review. Budget for the extra runs, since each review is billed.

For Greptile, `greptile review status` exits `0` when the commit has a completed review, `3` while one is running, `4`/`5` for a failed or cancelled review, and `1` when there is none or you are signed out, so a pre-push hook or CI step can require a finished review before anything merges. Treat every code other than `0` as "not reviewed".

## Add a second model: Greptile, CodeRabbit, ocr and the Codex plugin

Each of these gives you a reviewer from outside your author tool. Install commands are verified against each project's own README or npm entry on 2026-09-26, except CodeRabbit's installer page, which is secondary.

**Codex plugin for Claude Code** (OpenAI, `openai/codex-plugin-cc`). It runs your local Codex install from inside a Claude Code session and counts against your Codex usage limits. `/codex:review` is "a normal read-only Codex review" and takes no focus text; `/codex:adversarial-review` is steerable and challenges the design.

```text
# Inside Claude Code
/plugin marketplace add openai/codex-plugin-cc
/plugin install codex@openai-codex
/reload-plugins
/codex:setup
/codex:adversarial-review --base main challenge whether this retry design is safe under concurrent writes
```

`/codex:setup --enable-review-gate` adds a `Stop` hook that runs a Codex review on every Claude response and blocks the stop when it finds issues. The README warns that the gate "can create a long-running Claude/Codex loop and may drain usage limits quickly", so enable it only in a session you are watching.

**Greptile CLI** (npm `greptile` 3.6.0, Node 22 or later). `greptile review` compares the merge base with `HEAD`; `--plus` and `--apex` raise the effort.

```bash
npm install -g greptile          # or: brew install greptileai/tap/greptile
greptile                         # signs you in on first run
greptile init                    # admin only: enables the repository
greptile review -b main --instructions "focus on retry cancellation"
```

**CodeRabbit CLI.** Install it from CodeRabbit's official installer or Homebrew, not from npm. CodeRabbit's own `coderabbitai/skills` SKILL.md points to `https://www.coderabbit.ai/cli`. The install lines below are **secondary**: they come from search extracts of CodeRabbit's docs, and the installer page was not re-read on 2026-09-26. Do not install the npm name: the npm package `coderabbit` is a "security holding package" (version `0.0.1-security.1`). `--agent` returns findings an agent can act on, with severities from critical to info.

```bash
curl -fsSL https://cli.coderabbit.ai/install.sh | sh   # or: brew install --cask coderabbit (secondary)
coderabbit auth login
coderabbit review --agent --uncommitted
coderabbit review --agent --base main
```

**Alibaba Open Code Review (`ocr`)**, Apache-2.0, npm `@alibaba-group/open-code-review` 1.12.9. It pairs deterministic file selection and rule matching with an LLM agent, and works with any OpenAI- or Anthropic-compatible endpoint, so the code never leaves the provider you already use.

```bash
npm install -g @alibaba-group/open-code-review
ocr config provider && ocr config model
ocr review --from main --to feature-branch --format json --output result.json
```

Alibaba's README reports higher precision and F1 than Claude Code with the same model, at about one ninth of the tokens, and says recall is lower, "a deliberate trade-off favoring precision over noise". That is a vendor benchmark (200 pull requests from 50 repositories), not reproduced here. Choose `ocr` for a CI layer where noise costs you more than a missed nit.

As of 2026-09-26, `anthropics/claude-code-action` had 8,951 GitHub stars, `openai/codex-action` 1,248, `openai/codex-plugin-cc` 33,594 and `alibaba/open-code-review` 41,400; in Anthropic's plugin directory, `greptile` showed 56,611 installs and `coderabbit` 32,361 (GitHub and claude.com/plugins, read for this site's ecosystem catalogue). Stars measure attention, not review quality. Both plugins are in Anthropic's official marketplace (`claude-plugins-official`, checked 2026-09-26):

```text
# Inside Claude Code
/plugin install greptile@claude-plugins-official
/plugin install coderabbit@claude-plugins-official
```

The plugins carry context cost in every session. Before you keep one, run `claude plugin details greptile` or `claude plugin details coderabbit` in a terminal, which shows the plugin's components and projected token cost, and compare `/context` before and after.

## Copy-paste prompts for layered AI code review

**Copy-paste prompt: local self-review before you push**

```text
Review the changes on this branch against main as if a different engineer
wrote them. Start a fresh read: do not rely on anything you remember from
writing this code.

Report only findings that could break behavior in production: wrong logic,
unhandled errors on external calls, missing authorization or tenant scoping,
data loss, race conditions, and tests that assert nothing. No style, naming
or refactoring comments.

For each finding give: file:line, severity (Important or Nit), the input or
state that triggers it, and the smallest fix. If you are not confident, say
so and give the command that would confirm it. End with "No Important
findings" if that is the case. Do not edit any file.
```

**Copy-paste prompt: adversarial second-model review**

```text
You are reviewing a change another AI model wrote and already reviewed.
Assume its review missed something. Read the diff against main and:

1. List the three assumptions the change depends on that are not checked
   by any test, with file:line.
2. For each, describe the concrete input, timing or failure that breaks it.
3. Name the test that would catch it, in one sentence.

Ignore anything a linter, formatter or type checker already enforces.
Do not repeat findings that are about style. If you find nothing that could
break production, say "No production-relevant findings" and stop.
```

**Copy-paste prompt: triage the bot's comments before a human reads them**

```text
Collect every unresolved review comment on this pull request from review
bots (Claude, Codex, Bugbot, CodeRabbit, Greptile). For each comment:

- Classify it: REAL BUG, ALREADY FIXED, DUPLICATE, FALSE POSITIVE or STYLE.
- For REAL BUG, write a failing test that proves it first, then fix it.
- For FALSE POSITIVE, quote the code that shows why, in one sentence.

Do not resolve or delete any comment. Post one summary comment with the
counts per class and the commit SHA the tests ran on.
```

The third prompt produces the numbers you need for the precision metric below. In Claude Code with the GitHub CLI installed, the agent reads comments with `gh api`; in Codex and Cursor, use the GitHub MCP server or `gh` in the terminal.

## How do you prove a review bot is worth its cost?

A review bot is itself an unverified agent until you measure it. Track four numbers per bot per month, and review them with the team:

| Metric | Definition | Act when |
| ------ | ---------- | -------- |
| **Precision** | Findings that led to a code change, divided by findings raised | It falls below the level where people still read the comments; tighten `REVIEW.md` or `BUGBOT.md`, or lower the effort |
| **Escapes** | Production defects in code the bot reviewed, where a comment from the bot would have been enough | It rises; add the missed class to the bot's "Always check" list |
| **Cost per merged PR** | Bot spend for the month divided by merged pull requests | It rises faster than merged PRs; move the bot from "every push" to "once per PR" or Manual |
| **Time to first review** | Minutes from the PR opening to the bot's first comment | It exceeds your review SLA; Anthropic's launch announcement (9 March 2026) put Claude Code Review at about 20 minutes on average, so do not make humans wait for it |

The bots never own the decision. CI owns the gate (tests, types, lint, the severity-count check above), the bot owns the comments, and the approver named on the pull request owns the merge. The code owners in `CODEOWNERS` sign off on escalation classes, whatever the bots said. Canonical definitions for team metrics live in [metrics frameworks for agentic engineering](/en/org/metrics-frameworks/).

**Traps that make review bots misfire**

- **`/code-review` is not Code Review.** The local command runs on any plan and does not read `REVIEW.md`; the managed PR service is Team and Enterprise only and reads both `REVIEW.md` and `CLAUDE.md`.
- **`/review` in Claude Code** is an alias of `/code-review` since v2.1.223, not the old single-pass PR review.
- **`@claude review` is ignored** when your organization membership is private and you are not a direct collaborator on the repository. Make your membership public or ask an admin to add you.
- **`npm i coderabbit`** installs a security holding package. **`npm i @graphite/cli`** does not exist; the CLI is `@withgraphite/graphite-cli`.
- **Bugbot does not read `.cursor/rules`** (Cursor docs, checked 2026-10-12); it reads `.cursor/BUGBOT.md` (secondary).
- **`/codex:review` ignores focus text.** Use `/codex:adversarial-review` when you want to steer the review.
- **Ultrareview is not available** on Bedrock, Google Cloud's Agent Platform or Microsoft Foundry, or with Zero Data Retention; `/code-review ultra` then runs a local review in your session instead.

## How do you know a reviewer got better, not just different?

The four metrics above are a monthly dashboard, and next month's pull requests differ from this month's. To know whether a change to the reviewer's prompts, tools, model or `REVIEW.md` helped, rerun a fixed set of cases. The Claude Code team's set ("How the Claude Code team does code review", Addy Osmani, Morgan Westlee Lunt and Qing Wang, 15 October 2026) is past pull requests that caused incidents, with their fix-forwards and reverts, each bug confirmed by engineers who know the code. The reviewer runs against it daily, every change to the reviewer is scored against all of it, and it grows with every incident. The article does not say how a case is replayed, so the steps below are one way.

1. **Collect the cases** from your incident tracker and your reverts. The revert titles carry the caveat from [the review queue page](/en/teams/review-queue/#how-do-you-measure-human-review-time-with-reverts-as-the-guardrail):

    ```bash
    # Terminal, inside the repository, gh authenticated
    gh pr list --state merged --limit 200 \
      --json number,title \
      --jq '.[] | select(.title | startswith("Revert")) | [.number, .title] | @tsv'
    ```

2. **Have someone who knows the code confirm each case:** the commit or range that introduced the bug, the file and lines, what goes wrong, and who confirmed it. An unconfirmed case tests the reviewer against your guess.

3. **Save the cases next to the reviewer's configuration** (tab-separated, the row is an illustration):

    ```text title="evals/reviewer/cases.tsv"
    case	review_range	bug_location	bug	confirmed_by
    inc-0412	9c1e7a2^..9c1e7a2	src/refund.ts:88	Retry lacks idempotency key	@anna
    ```

4. **Replay each case from a checkout of the code as it was.** A reviewer that reads files around the diff would otherwise see today's code, with the bug already fixed, and recall would be inflated. Use a throwaway worktree at the range's end commit, run the reviewer from it, and save each output in the main checkout as `evals/reviewer/runs/RUN_ID/CASE.md` (`RUN_ID` names the run, `CASE` the case), so that the worktree stays clean and `git worktree remove` can delete it:

    ```bash
    # Terminal, main checkout: detached worktree at the end commit
    git worktree add --detach ../eval-inc-0412 9c1e7a2
    # run the reviewer from there; write its output to the main checkout
    git worktree remove ../eval-inc-0412
    ```

    Run the reviewer from that worktree:

    - **`/code-review` in Claude Code:** start the session in the worktree and run `/code-review 9c1e7a2^..9c1e7a2`. It reads `CLAUDE.md` and not `REVIEW.md`, so it does not stand in for the managed service.
    - **`codex review`:** run `codex review --commit 9c1e7a2` (Codex CLI 0.157.1).
    - **A hosted bot such as Claude Code Review or Bugbot:** push the case's parent and end commits as two scratch branches and open a pull request from the second into the first, so the bot sees the code as of that commit. Then request a review with `@claude review` or `bugbot run` (secondary).

    A hosted replay is billed like any review. Anthropic runs its set daily; run yours weekly until you know the cost.

5. **Score recall and count the rest.** A case is found when the reviewer posts a finding at the bug's location that describes it. Recall is cases found divided by all cases; every other finding goes into the noise count below. The first prompt below scores a run.

6. **Rescore after every change** to a prompt, tool, model, level or rules file. The service never reads `REVIEW.md` from the pull request's own changes (Claude Code docs, read 2026-10-16), so a change to it applies only after it merges to the base or default branch; rescore then. A change that raises recall and noise together is not yet an improvement.

**Copy-paste prompt: score a reviewer run against the eval set**

```text
Read evals/reviewer/cases.tsv and the reviewer outputs in
evals/reviewer/runs/RUN_ID/ (one file per case, named after the case).

For each case:
1. Mark FOUND if the output has a finding at the case's bug_location (same
   file, within 10 lines) that describes the same bug. Quote that finding.
   Otherwise mark MISSED.
2. List every other finding in that file as an extra, with file:line and one
   line on what it claims.

Do not judge whether an extra is a real bug. Print a table: case, FOUND or
MISSED, number of extras. Then print recall (FOUND divided by all cases) and
the total number of extras. If evals/reviewer/runs/PREVIOUS_RUN_ID/ exists,
list every case that was FOUND there and is MISSED now. Do not edit any file.
```

Replace `RUN_ID` with this run's folder name and `PREVIOUS_RUN_ID` with the last run's.

## How do you measure a reviewer's noise?

A reviewer can raise recall by flagging more of everything. The article: "It's easy to catch more bugs if you don't mind flagging a lot of non-bugs, so we need a check on that." Anthropic sorts every comment by how the person on the pull request responded (fixed it, said it was wrong, said the behavior was intentional, or decided it didn't matter), and runs the reviewer on pull requests known to be clean, counting reports that would have led someone to make a harmful change.

The article does not say how responses are recorded. One cheap convention: start each reply with `fixed:`, `wrong:`, `intentional:` or `minor:`, which the prompt below tallies. Each response points at a lever:

| What the author did | What it tells you | What to change |
| ------------------- | ----------------- | -------------- |
| Fixed it | A useful finding; their share is the precision metric above | Keep the rule that produced it |
| Said it was wrong | A false positive | Require a `file:line` citation for that class of finding, or add a skip rule |
| Said it was intentional | The reviewer lacked context | Put the context in `CLAUDE.md` or the rules file |
| Decided it didn't matter | True, and not worth a round | Cap the nits and raise the bar for Important |

For the clean set, use merged pull requests that nobody later reverted, fixed forward or tied to an incident, and replay them the same way, from a checkout at each pull request's head. The recall score hides the harmful reports you count there.

Anthropic's written priority is real bugs first, fewer review rounds second. Write yours the same way: a change that cuts rounds by missing known bugs then fails the eval, and one that finds more at the cost of more rounds is a trade you make on purpose.

How often a reviewer speaks is separate from how often it is right. At the research-preview launch, the share of Anthropic's internal pull requests with substantive review comments went from 16% to 54% (launch announcement, 9 March 2026, repeated in the 2026-10-15 article); most pull requests over 1,000 lines get findings and most small ones come back clean. That is a coverage figure about its own pull requests, so read it next to the noise measures.

Noise costs more when the author is an agent: a person waves off a nit in seconds, while an agent fixes it, which means another commit and another round. Nit caps and "Important only after the first review" rules in `REVIEW.md` matter most there; [the round cap](/en/workflows/ship/agent-pr-review/#why-does-an-agent-review-loop-need-a-round-cap) covers the rest.

**Copy-paste prompt: tally what authors did with each bot comment**

```text
With gh api or the GitHub MCP server, list every review comment that a bot
(Claude, Codex, Bugbot, CodeRabbit or Greptile) posted on pull requests merged
in the last 30 days. For each comment, read its thread and sort it by the
author's response:
- FIXED: the author pushed a change for it, or replied "fixed:".
- WRONG: the author replied "wrong:", or showed the finding is incorrect.
- INTENTIONAL: the author replied "intentional:".
- MINOR: the author replied "minor:".
- UNANSWERED: no reply, or the thread was resolved with no reply and no change.

Print one table per bot with the count for each response and the share of
FIXED among all comments. Quote one WRONG and one INTENTIONAL example per bot.
Do not edit, reply to or resolve any thread.
```

## How does verification by voting cut a reviewer's noise?

In the article, several agents look for bugs at once, then a separate group votes on each finding, and a finding is posted only if enough of them agree it is real. Style-, diagnostics- and cleanup-only findings are dropped, filters the team A/B tested on its own pull requests. What remains is tagged Important, Nit or Pre-existing, "so a bug the PR didn't introduce doesn't look like the author's fault". The Claude Code docs describe the public service as find, verify, rank and post, with verify filtering out false positives (read 2026-10-16); the voting detail is the article's. For your own reviewer as a subagent or skill, copy the shape:

1. **Separate the finder from the verifier.** The verifier sees one finding and the code it points at, without the finder's reasoning, and tries to refute it.
2. **Post only what survives:** a majority of several verifiers, or one verifier's `file:line` evidence. Drop style-, diagnostics- and cleanup-only findings unless the style hides a bug.
3. **A/B test each filter on the eval set before you keep it.** Recall must not fall, and noise must.

**Copy-paste prompt: verify review findings before anyone reads them**

```text
Read review-findings.md. Each finding has a file:line and a claim. Treat every
finding as if you were a separate verifier that sees only that finding and the
code it points at, not the reviewer's reasoning. Use a fresh subagent or
session per finding if your tool supports it.

For each finding:
1. Try to refute it: find the code path, test or guard that makes the claim
   false, and quote file:line.
2. If you cannot refute it, reproduce it: name the input or state that
   triggers it and the command that would show it.
3. Vote KEEP or DROP. DROP any finding that is only about style, diagnostics
   or cleanup, unless it hides a bug.

Print a table: finding, vote, one-line reason. Do not edit any file and do not
post anything.
```

## How do you roll out and tune a reviewer?

The article's advice is one repository first, a few weeks of tuning, then widening:

1. **Start on one repository** where pull requests are frequent and a missed bug has a cost. For Claude Code Review, the costs page suggests a pilot with a monthly spend limit and **Review runs** on **Once after PR creation** or **Manual**, then reading each repository's average cost per review (Claude Code docs, read 2026-10-16).

2. **Write the rules file short.** The article's good `REVIEW.md` files say what failure looks like in that repository (for docs, "it lies"; for config, "it misroutes"), cap the nits, list paths to skip, ask for a file and line as evidence, and ask for a summary that starts with a count and says when there is nothing to fix. The same traits suit `AGENTS.md` for `@codex review` and `.cursor/BUGBOT.md` for Bugbot. The eval above checks that the reviewer follows the file.

3. **Match the level to the cost of a miss.** An Owner sets Claude Code Review to Quick, Standard or Deep, with a per-repository override; "Deep is worth it where one missed bug costs more than the extra review time". Bugbot has effort levels (Low, Default, High, Smart) on usage-based plans, as the Cursor tab above records.

4. **Each week, fix repeats at the source:** the skill, a lint rule or the instruction files. When the reviewer misses something, ask what it did not have in front of it and put that into `CLAUDE.md`, a skill or a check. Then rescore the eval and read the response table.

5. **Widen after a few weeks,** starting the next repository from the first one's rules file and eval cases.

## What breaks when you run AI code review bots?

**The bot drowns the real bug in nits.** Thirty comments, one of them important, and the team stops reading. Recovery: cap nits in `REVIEW.md` or `BUGBOT.md`, add "after the first review, post Important findings only", and track precision. If precision keeps falling, switch that bot to Manual.

**Two bots argue on the same pull request.** Claude Code Review and Bugbot both comment, disagree, and the author fixes one and reopens the other. Recovery: one bot owns layer 3. Run the second model locally (layer 2), where its output goes to the author, not to the pull request.

**Push-triggered reviews multiply the bill.** A pull request with 12 pushes gets 12 reviews. Recovery: use "once after PR creation" or Manual, and ask for `@claude review always` only on high-risk pull requests. Set a monthly spend cap for Claude Code Review at `claude.ai/admin-settings/usage`.

**Forks and drafts get no review, and nobody notices.** GitHub withholds secrets from fork pull requests, `claude-code-action` skips drafts, and Code Review reviews forks only on a comment command. Recovery: do not rely on the review job as a required check, because GitHub counts a skipped job as passed. Require the `claude-review-gate` status above instead: it stays pending, and blocks the merge, until the tip commit has a review. Have a maintainer comment `@claude review` on fork pull requests before approving.

**The team treats a clean bot run as approval.** "The bot found nothing" becomes the merge reason. Recovery: gate on tests and the severity count, leave Copilot approvals off unless a human still signs off on escalation classes, and keep a named human approver on every pull request.

**A change raises recall and noise together.** The eval shows more known bugs found, and nobody counted the extras. Recovery: score both on every run, and keep the change only if your written priority allows the trade.

**The eval scores the reviewer's opinion.** Cases nobody who knows the code confirmed reward whatever the reviewer already says. Recovery: have a person who knows that code confirm every case, as Anthropic's set does, and drop any case nobody can confirm.

**A review loop drains usage.** The Codex plugin's review gate or an autofix bot keeps finding and fixing new issues. Recovery: enable the gate only in watched sessions, stop at a round cap (three rounds in Anthropic's setup, two returns on the agent PR review page), and escalate to a human with the open findings listed; [the round cap section](/en/workflows/ship/agent-pr-review/#why-does-an-agent-review-loop-need-a-round-cap) explains why later rounds mostly find old code.

## Where to go next with AI code review

- [Reviewing an agent's pull request](/en/workflows/ship/agent-pr-review/) — The six-step human triage that sits on top of the bots, with verdicts, escalation classes and a round cap for review loops.
- [The evidence bundle](/en/workflows/ship/evidence-bundle/) — What an agent's pull request must prove before any reviewer, human or bot, reads it.
- [Security gates for agent code](/en/ecosystem/observability/agent-security-gates/) — Deterministic scanners (Semgrep, Gitleaks, Snyk Agent Scan) that sit beside the review bots.
- [Running the review queue](/en/teams/review-queue/) — Size budgets, risk lanes and a human-time measure with reverts as the guardrail.
- [Code review with Codex](/en/codex/ship/code-review/) — One review standard across the Codex CLI, desktop app and GitHub.
- [Automated code reviews in Claude Code](/en/claude-code/ship/review-automation/) — CLI-driven review in Claude Code: pre-commit checks, PR analysis and quality gates.
- [Govern layered pull-request review](/en/developer-scorecard-guide/layered-pr-review/) — A team review contract with focused passes for correctness, security, tests and spec compliance.
- [Codex GitHub Action](/en/codex/ship/ci-cd/) — Running Codex in your own CI with permission profiles.
- [Model prices and defaults](/en/ecosystem/models/) — Token prices behind the CLI and Actions routes.
