Running the review queue when agents open the pull requests
PLA review queue for agent-opened pull requests stays healthy when pull requests arrive no faster than the team can review them. A tech lead keeps that balance with four controls measured by time in review: a work-in-progress (WIP) cap that pauses agent dispatch, a PR size budget, risk lanes that decide who reviews what, and load-balanced reviewer assignment.
Your six engineers each run two or three agents, and on Monday morning there are 34 open pull requests. Each one looks reasonable. None of them is urgent. By Thursday the oldest has been waiting four days, two people have quietly started approving without reading, and a senior engineer tells you in a one-to-one that they spend the whole day reviewing and never build anything. That is the review ceiling from Level 3 of the autonomy ladder, and this page is the operating manual for it.
This page is for the tech lead who owns one team’s flow of work and assumes agents already open pull requests; if not, start with the agent-ready backlog.
What a managed review queue gives your team
Section titled “What a managed review queue gives your team”- A one-line capacity check that tells you whether the queue will grow or drain this week.
- A WIP cap derived from your own numbers, and a script that stops agent dispatch when the queue is full.
- A PR size budget that the agents follow and CI enforces, with an escape hatch you control.
- A risk-lane table that routes each pull request to a green pipeline, an automated reviewer, one human, or two.
- Eight metric definitions you can compute this afternoon with
ghandjq, and the thresholds that trigger a change. - A human-time measure with revert and incident rates as the guardrail, and approval rules and a landing policy you can commit.
- A sustainable-pace policy: session caps, a review rota, and the signals that tell you to lower concurrency.
Why agent pull requests overwhelm review
Section titled “Why agent pull requests overwhelm review”Generation scaled and reading did not. Faros AI’s AI Engineering Report 2026: The Acceleration Whiplash (April 2026, telemetry from 22,000 developers) measured both sides at once. The pull request merge rate per developer rose 16.2%. In the same window, median time in review rose 441.5%, and median time to first review rose 156.6%. Incidents per pull request rose 242.7%, and 31.3% more pull requests merged with no review at all. The pull requests also grew: Faros reports PR size up 51%. DX’s panel (June 2026) saw the median go “from 44 lines to 72 lines per pull request between July 2025 and June 2026”.
The unreviewed-merge number is the one to worry about: in an unmanaged queue, reviewers do not refuse the load, they stop reading and keep approving.
DORA names the counter-measure in its AI Capabilities Model. On “working in small batches”, Google Cloud’s DORA write-up (December 2025) says: “AI can easily generate massive blocks of code, which are hard to review and test. Enforcing the discipline of small batches counteracts this risk”. The rest of this page turns that sentence into rules you can enforce.
Check whether your review queue can drain
Section titled “Check whether your review queue can drain”A queue is stable only when the arrival rate is below the service rate. Measure both for the last four weeks, not from memory:
- Arrival rate (λ): non-draft pull requests opened per working day, agent and human together.
- Review throughput (μ): pull requests your reviewers can finish per working day. Compute it as reviewer-hours spent on review per day divided by the median hours of review per pull request.
A worked example with round numbers: four reviewers each protect 90 minutes a day for review, which is 6 reviewer-hours. A median pull request takes 30 minutes of review, so μ is 12 pull requests a day. If agents and humans open 18 a day, the queue grows by six every day, forever. No amount of reviewer effort fixes that; only fewer or smaller arrivals do.
Queueing theory adds a warning: waiting time grows sharply, not linearly, as utilisation (λ ÷ μ) approaches 1. A team whose reviewers are fully loaded waits a long time even when the arithmetic works. Plan for review to be busy about three quarters of the time, and treat anything above that as a queue about to jam.
Set a WIP cap that pauses agent dispatch
Section titled “Set a WIP cap that pauses agent dispatch”Little’s law links the three numbers you care about: items in the queue = throughput × time in the queue. Choose the time in review you want, multiply by your measured throughput, and that is your WIP cap. With μ = 12 per day and a target of one working day in review, the cap is 12 open non-draft pull requests awaiting review, team-wide.
The cap only works if it stops dispatch, not pull request creation: an agent that cannot open a pull request leaves a branch nobody sees, which is the same inventory hidden somewhere worse. So put the gate in front of whatever starts agent work: the issue-to-PR workflow, a scheduled routine or automation, or the command your engineers use to launch a background agent.
#!/usr/bin/env bash# Exit 1 when the review queue (open, non-draft, not yet approved) is at or above the WIP cap.set -euo pipefailCAP="${REVIEW_WIP_CAP:-12}"open=$(gh pr list --state open --search "draft:false -review:approved" \ --limit 200 --json number --jq 'length')if [ "$open" -ge "$CAP" ]; then echo "Review queue full: $open PRs awaiting review (cap $CAP). Review before dispatching more." >&2 exit 1fiecho "Review queue $open/$CAP: dispatch allowed."The script counts exactly what the cap defines: every open non-draft pull request without an approval, from agents and humans alike. Approved pull requests waiting to merge no longer need a reviewer, so they do not count. Keep the cap in one place, the REVIEW_WIP_CAP variable in your CI settings, so changing it is a one-line decision rather than a code change. Review the cap every two weeks against the metrics below; it is derived from throughput, so it moves when throughput moves.
Enforce a PR size budget the agents follow
Section titled “Enforce a PR size budget the agents follow”Small pull requests are the cheapest capacity you can buy: they are faster to review, they pinpoint the fault, and they revert cleanly. Set the budget from your own history, for example at the size below which three quarters of your human-reviewed pull requests fall. If you have no history, start at 400 changed lines excluding lockfiles and snapshots, and tune it after a month. That number is a starting point, not a finding.
Tell the agents first, in the instruction file every session reads, so they plan the split before writing code:
## Pull request budget- One concern per pull request. Stay under 400 changed lines, excluding lockfiles and snapshots.- If a task needs more, stop before coding and propose a split into stacked pull requests, each independently green and reviewable on its own.- Never change tests, CI config or lint rules in the same pull request as the code they check. Open those separately with the label `oracle-change`.- Open every pull request with `gh pr create --label agent` and put the evidence bundle (acceptance criteria, commands run, results) in the description.Then make CI the backstop, because an instruction is a request and a failing check is a rule:
name: pr-size-budgeton: pull_request: types: [opened, synchronize, reopened, labeled, unlabeled]permissions: contents: readjobs: size: if: ${{ !contains(github.event.pull_request.labels.*.name, 'size-exception') }} runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 with: fetch-depth: 0 persist-credentials: false - name: Compare changed lines with the budget env: BASE: ${{ github.event.pull_request.base.sha }} BUDGET: 400 run: | changed=$(git diff --numstat "$BASE" HEAD -- . \ ':(exclude,glob)**/package-lock.json' ':(exclude,glob)**/pnpm-lock.yaml' \ ':(exclude,glob)**/*.lock' ':(exclude,glob)**/__snapshots__/**' \ | awk '{ a += $1; d += $2 } END { print a + d + 0 }') echo "Changed lines: $changed (budget $BUDGET)" if [ "$changed" -gt "$BUDGET" ]; then echo "::error::Over the $BUDGET-line review budget. Split the change, or ask the tech lead for size-exception." exit 1 fiThe size-exception label is the escape hatch for mechanical changes such as a rename across the codebase. Make it team policy that only the tech lead applies it. GitHub does not enforce that, because anyone with triage or write access can add a label, so the weekly metrics list every use of size-exception and who applied it (from the pull request’s timeline), which makes misuse visible.
Two more limits make this a backstop rather than a lock. A job that its if: skips reports as skipped, and a skipped job satisfies a required check, so the label is a real bypass and not only a signal. And on the pull_request trigger GitHub runs this workflow from the pull request’s merge ref, so a pull request can raise the budget or delete the job. The /.github/ line in the CODEOWNERS file below, with “Require review from Code Owners” on, is what makes such an edit wait for the tech lead.
Route each pull request by risk
Section titled “Route each pull request by risk”Not every pull request needs the same reader. The evidence-based review model says what a human reads instead of the diff; the lanes below say which pull requests reach a human at all. Write the table down, commit it as docs/review-lanes.md next to CODEOWNERS, and let the paths decide the lane.
| Lane | What falls in it | Gate before merge | Who signs off |
|---|---|---|---|
| A. Green pipeline | Dependency patch bumps, docs, copy changes, generated clients, changes that match a pattern you have already approved many times | Types, tests, lint, and the size budget pass; automated reviewer raises no blocking finding | The pipeline; the tech lead samples a few each week |
| B. Automated plus sampled human | Ordinary feature and bug-fix work inside one module with strong tests | Lane A gates plus an automated reviewer; a human reads the evidence bundle, and reads code only when the evidence is thin | One reviewer, assigned by load |
| C. Code owner | Public APIs, shared libraries, performance-sensitive paths, anything crossing module boundaries | Lanes A and B plus a code owner’s approval | The owning team via CODEOWNERS |
| D. Two humans, code read | Authentication and authorisation, money, schema and data migrations, and any oracle-change (tests, CI, lint, or type config) | All gates plus two approvals, one from a code owner; a deep automated review first | Two named reviewers; the tech lead for oracle changes |
CODEOWNERS enforces the code-owner approval for lanes C and D without a new tool, provided the branch protection rule “Require review from Code Owners” is on:
/db/migrations/ @acme/data-owners/src/auth/ @acme/security-reviewers/src/billing/ @acme/security-reviewers/.github/ @acme/tech-leads/tests/ @acme/tech-leads/eslint.config.js @acme/tech-leadsGitHub branch protection has a single required-approvals count for the whole branch, and that shapes the other lanes. Keep the count at 1, and satisfy lane A in one of two ways. Either bind an auto-approver to the lane A criteria (Cursor PR Routing & Approval, or a GitHub Action that approves only lane A paths; the Action needs the repository setting “Allow GitHub Actions to create and approve pull requests”), or give the automation that merges lane A (a GitHub App or bot account) bypass permission in the branch ruleset, and have it merge only pull requests whose paths all match lane A. Either way, the tech lead’s weekly sample is the check on it.
A count of 1 also means nothing enforces lane D’s second approval: CODEOWNERS requires one code-owner approval, not two. Give that second approval its own control. Either add a required CI check that fails when lane D paths changed and the pull request has fewer than two approvals (read them with gh pr view <number> --json reviews), or keep it as policy and audit it in the weekly metrics below, where the Unreviewed merges row also counts lane D merges with fewer than two approvals.
Lane A starts empty. A class of change earns its way into it after a run of merges with no reverts and no escaped defects, and falls back out after one. The trust-transfer protocol describes that staging in detail.
How does risk-scaled approval shrink the review queue?
Section titled “How does risk-scaled approval shrink the review queue?”The lanes above are the lever: every pull request that needs no human, or one instead of two, leaves the queue. The Claude Code team at Anthropic uses three tiers (“How the Claude Code team does code review”, Addy Osmani, Morgan Westlee Lunt and Qing Wang, claude.dev blog, 15 October 2026):
| Risk | Examples | Human approvals |
|---|---|---|
| Low | Docs, copy, internal-only features, additive changes behind a flag | None required once automated review and security checks are green |
| Medium | Ordinary product changes, and changes to sensitive areas such as caching, the auto-updater, or system prompts | One person |
| High | The highest-risk code, plus a small set of paths that always need two people | Two people |
Roughly, Low is lane A, Medium is lanes B and C, and High is lane D. The article attaches four rules to the table:
- The change sets the tier, not the author. A classifier picks the tier, and a second one asks whether this change needs another person even when its tier says it does not. Anything the owners listed as sensitive, and any CI workflow file, always needs a person; a path list in
CODEOWNERSis the simplest version. - Only trusted authors get the lighter paths. The trusted list is mostly the owning team; everyone else needs a second person. An agent’s approval never counts as a person’s: it can clear a low-risk change but can’t fill a slot that calls for a human.
- Whoever prompted the change is on the hook. On an agent-written pull request, the person directing the agent counts as one of the required reviewers.
- An approval covers one specific commit. After commits the system can’t vouch for, such as an agent’s push or a merge resolved by hand, the approval has to be given again.
The adoption checklist adds: start strict and loosen with evidence, and dismiss approvals on new commits. In GitHub, the closest setting is “Require approval of the most recent reviewable push”. This snippet is stricter than Anthropic’s rule, because it re-approves after any push:
## Approval rules- The change sets the lane, not the author. The lane comes from the paths changed and `CODEOWNERS`.- Lane A is open only to the authors in `docs/trusted-authors.md`. Everyone else's pull request needs one person.- A bot's approval never fills a slot that calls for a person.- On an agent-written pull request, the person who directed the agent is one of the required reviewers.- An approval covers one commit. Any push after approval, including an agent's, or a merge resolved by hand needs a new approval.- Start strict. Move a class of change to a lighter lane only after a run of merges with no reverts and no incidents.Balance reviewer load instead of assigning by habit
Section titled “Balance reviewer load instead of assigning by habit”Left alone, review requests flow to the two people who answer fastest, and they become the bottleneck and then the burnout. GitHub’s team code review settings fix the mechanics. Under Team settings › Code review, select Enable auto assignment and choose the Load balance routing algorithm rather than round robin.
GitHub’s documentation says load balance “considers the number of outstanding reviews for each member”. It aims for each member to review an equal number of pull requests in any 30-day period. Round robin, by contrast, alternates “regardless of the number of outstanding reviews they currently have”.
Two more settings matter. Members whose GitHub status is Busy are not selected, so a focus block becomes a real state rather than a polite fiction. Never assign certain team members keeps someone off the rotation during onboarding or an incident week.
Pre-review and routing in Claude Code, Codex, and Cursor
Section titled “Pre-review and routing in Claude Code, Codex, and Cursor”The queue controls above are forge-level and identical for all three tools. What differs is how each tool reviews a change before a human sees it and how it helps route the result.
- Before the pull request opens: have the agent run the bundled
/code-reviewskill on its own diff and fix what it finds. Add--fixto apply fixes in the same pass. - On the pull request: the managed Code Review service (Team and Enterprise plans) posts inline findings sorted into Important, Nit and Pre-existing, billed on token usage: read each repository’s average cost per review in Admin settings and multiply by its pull requests per month (Claude Code docs, read 2026-10-16). A review never approves or blocks the pull request and its check run never concludes as a failure, so it informs lane B and cannot be lane A’s gate. To make findings block, require conversation resolution on the branch.
- Lane D deep review: run
claude ultrareview 482from the shell, where482is the pull request number (/code-review ultrainside a session). It runs a multi-agent review in the cloud and independently reproduces each finding; Anthropic lists it at typically $5 to $25 after three free runs on Pro and Max. It is not available on Bedrock, Google Cloud, or Foundry, or to Zero Data Retention organisations. - Watching the fleet:
claude agentsopens agent view (research preview), which lets you “dispatch and manage many Claude Code sessions from one screen”. Use it to see in one place how many sessions your engineers are supervising.
- Before the pull request opens: run
codex exec review --base mainin the agent’s worktree or as a CI step, or/reviewinside the interactive session.--base,--commit, and--uncommittedcannot be combined with a custom prompt (checked on Codex CLI 0.157.1), so put custom instructions in the prompt alone, for examplecodex exec review "Review the changes against main and check them against the lane table in docs/review-lanes.md", or put the rules inAGENTS.md. - On the pull request: comment
@codex reviewto request a review, or enable automatic reviews in the GitHub integration, and put your review rules inAGENTS.md(Codex integration docs, checked 2026-08-28). - Watching the fleet:
codex agentsbrowses every agent session on the local app-server daemon (Codex CLI 0.157.1). - Codex’s auto-review (
--approve-for-me) reviews sandbox-boundary approvals during a run. It is not a pull request reviewer and does not belong in the lane table.
- On the pull request: Bugbot “reviews pull requests and identifies bugs, security issues, and code quality problems”. Treat its findings as lane B input.
- Routing: PR Routing & Approval “assigns reviewers based on code ownership and commit history, and can approve low-risk PRs when your criteria are met” (Cursor docs, checked 2026-08-28). Write its approval criteria so that they match your lane A definition exactly, and never let it approve a path listed under lanes C or D.
- Instructions: put the PR budget text above into your Cursor project rules so the agent plans a split before it writes code.
- See Rollouts and PR approval in Cursor for the Cursor-specific setup.
An automated reviewer that cannot block trains people to scroll past it. Decide which findings it may fail the build on, a short list such as secrets, SQL built from strings, and disabled tests, and treat everything else as a comment. The agent PR review workflow covers how to configure that layer, and PR review automation for teams covers rolling it out across a team.
Measure the queue with time in review
Section titled “Measure the queue with time in review”Throughput counts are easy to game and tell you nothing about the queue. Track these eight, weekly, split by lane and by agent versus human author:
| Metric | Definition | Act when |
|---|---|---|
| Arrival rate | Non-draft pull requests opened per working day | It exceeds review throughput for two weeks running |
| Review throughput | Pull requests reaching merge or close after a review, per working day | It falls while arrivals hold steady |
| Queue depth | Open non-draft pull requests with no approval yet, agent and human (what the gate script counts) | It sits at the WIP cap for more than two days |
| Time to first review | Opened (or marked ready) until the first submitted human review, median and 90th percentile | The median exceeds half a working day |
| Time in review | Opened (or marked ready) until merge, median and 90th percentile | The 90th percentile exceeds your target by 50% |
| PR size | Changed lines excluding lockfiles and snapshots, median | The median rises two weeks in a row |
| Unreviewed merges | Share of merges with no submitted human review outside lane A, plus lane D merges with fewer than two approvals | Any merge outside lane A has no review, or any lane D merge has fewer than two approvals |
| Reviewer load | Open review requests per person, and the spread between the busiest and the least busy | One person carries twice the team median |
The thresholds are starting points for a conversation, not industry benchmarks. The first four weeks give you your baseline; after that, compare against it. For how these fit the DORA and SPACE families, see engineering metrics frameworks.
This command exports the timing, size and changed-path columns for the last 200 merged agent pull requests. Run it in a terminal inside the repository, with gh authenticated:
gh pr list --state merged --label agent --limit 200 \ --json number,author,createdAt,mergedAt,additions,deletions,reviews,files \ --jq '.[] | (.createdAt | fromdateiso8601) as $o | .author.login as $me | [ .number, .createdAt, .mergedAt, (.additions + .deletions), (([.reviews[] | (.author.login // "") as $r | select($r != $me and ($r | test("\\[bot\\]$|^(cursor|coderabbitai|sentry|github-actions|chatgpt-codex-connector|gitguardian)$|^(copilot|greptile|linear)") | not)) | .submittedAt] | min) as $f | if $f then ((($f | fromdateiso8601) - $o) / 3600 | floor) else "none" end), (((.mergedAt | fromdateiso8601) - $o) / 3600 | floor), ([.files[].path] | join(",")) ] | @tsv' > review-queue.tsvThe columns are pull request number, created and merged timestamps, changed lines, hours to first review (none means merged unreviewed), hours in review, and the changed paths, comma-separated. gh fetches at most the first 100 files per pull request, so a larger one lists only part of its paths. The filter drops reviews by bots and by the pull request’s own author, so the first-review column measures human review; the automated reviewers above would otherwise almost always review first. Review bots carry no [bot] suffix in this output. The pattern lists the common ones, so list the logins once with the first command in the human-time section and add any bot you see.
createdAt includes draft time, so the numbers overstate the queue if agents open drafts first. When that distortion matters, use the ready_for_review event from the timeline API instead. The size column also counts lockfiles, which the CI budget excludes.
How do you measure human review time, with reverts as the guardrail?
Section titled “How do you measure human review time, with reverts as the guardrail?”The eight metrics above measure the queue. They do not show whether the time moved somewhere worse. The Claude Code team counts human time on both sides of review’s cost: “the time people spend reviewing, and the time they spend fixing what bad code breaks”. Its goal is that most pull requests “merge with at most one human in the loop, who can be the author, with no increase in reverts or incidents”. It tracks human approvals per pull request (median and 95th percentile), human comments, and how long a pull request waits on a person. “Revert and incident rates are the guardrail: if they rise, we haven’t saved human time, we’ve only moved it.”
Anthropic reports that from June to September 2026 its merged pull requests more than doubled while the share reverted fell by more than 40 percent, with Claude writing about 90 percent of Claude Code’s own code. These are self-reported figures, so do not read them as a benchmark.
| Measure | Definition |
|---|---|
| Human approvals per PR | Distinct people, not bots, who approved; median and 95th percentile |
| Human comments per PR | Comments from people other than the author and the bots; median and 95th percentile |
| Time a PR waits on a person | Ready for review until a person acts; the first-review column of the export above approximates it |
| Revert rate (guardrail) | Reverted pull requests divided by merged pull requests |
| Incident rate (guardrail) | Merged pull requests traced to an incident, divided by merged pull requests |
First list who commented or reviewed. Review bots carry no [bot] suffix in gh output; this repository’s list shows chatgpt-codex-connector, coderabbitai, cursor, gitguardian, linear-code and sentry next to one person.
# Terminal, inside the repository, gh authenticatedgh pr list --state merged --limit 200 --json reviews,comments \ --jq '[.[] | (.reviews[], .comments[]) | .author.login] | unique'Add every bot you see to the pattern below, which covers the common ones, then export approvals and comments for the last 200 merged pull requests and take the median and 95th percentile:
gh pr list --state merged --limit 200 --json number,author,reviews,comments \ --jq '.[] | .author.login as $me | def human: select((.author.login // "") as $l | $l != $me and ($l | test("\\[bot\\]$|^(cursor|coderabbitai|sentry|github-actions|chatgpt-codex-connector|gitguardian)$|^(copilot|greptile|linear)") | not)); [ .reviews[] | human ] as $r | [ .comments[] | human ] as $c | [ .number, ([$r[] | select(.state == "APPROVED") | .author.login] | unique | length), ([$r[] | select(.body != "")] | length) + ($c | length) ] | @tsv' > human-time.tsv
# Median and 95th percentile of approvals (column 2) and comments (column 3)for col in 2 3; do cut -f"$col" human-time.tsv | sort -n \ | awk '{ v[NR] = $1 } END { printf "median %s, p95 %s\n", v[int((NR + 1) / 2)], v[int(NR * 0.95 + 0.999)] }'doneThe columns are the pull request number, the distinct human approvers and the human comments. gh pr list returns review summaries and conversation comments but not inline diff comments, so the comment count is a floor. This export covers every merged pull request, while the queue export above covers --label agent only. Waiting time is the hours to first review column of the queue export, which covers the first wait only.
For the guardrail, count reverts as merged pull requests whose title starts with Revert. GitHub’s revert button titles them Revert "<original title>" in our observation, which GitHub’s docs and gh pr revert --help do not state, and a hand-made revert can carry any title (this repository’s #24, “Revert scorecards back to 0-75 scale”, has no quoted original). Treat the count as a floor, and take incidents from your incident tracker:
# Terminal, inside the repository: merged and reverted pull requests since 2026-09-01gh pr list --state merged --search "merged:>=2026-09-01" --limit 500 --json title \ --jq '{merged: length, reverts: ([.[] | select(.title | startswith("Revert"))] | length)}'Write your goal in the article’s shape, with a baseline from your first four weeks: “Most pull requests in REPO merge with at most one human in the loop, who can be the author, with no increase in reverts or incidents against the four weeks before the change”. Read the guardrail weekly, and when it rises, tighten the lane that changed first. Anthropic says it does not yet have circuit breakers that do this automatically, and that its engineers want them before more code moves to merges with no human. Until you build one, a person reads the numbers.
When can the author land a change and own the fallout?
Section titled “When can the author land a change and own the fallout?”The article says: “If you land a small or medium change and something still goes wrong, it’s a team norm that you usually fix it.” It lists five conditions that keep this from going badly. Check them before you move a class of change to a lighter lane:
| Condition (the article’s wording) | Where to build it |
|---|---|
| Changes are small, so they are easy to review, debug and revert | The size budget above |
| Everything is reversible: flags, reverts and staged releases undo most mistakes in minutes | Progressive delivery and a “How to revert” line in the pull request template |
| Blast radius sets the bar: expensive code gets more people and more checks | The risk lanes above |
| Regressions surface fast, through telemetry and internal use | Agent telemetry |
| It isn’t blame: owning a fix means you have the most context, and incident reviews look at the system | When an agent causes an incident |
If one row is missing, keep the lane stricter until it is there. Commit the norm so it covers whoever directs an agent:
## You land it, you own itThis applies to lane A and lane B changes. Lane C and lane D changes keeptheir named reviewers.
- The person who directed the agent lands the change, and fixes it if it breaks.- You land a change only inside the size budget, and only with a "How to revert" line that works.- An incident review asks which check should have caught the problem. It does not ask who merged it.Raise a different direction early, not as a blocking comment
Section titled “Raise a different direction early, not as a blocking comment”Agents make it cheap to start things, so some work should not land. The article says someone in the team’s channel usually asks why, points at an earlier decision, or suggests a simpler route, and the author closes or reshapes the pull request without feeling a loss. Its advice: “raise a different direction in the thread early, not as a blocking comment on a finished PR”. For reviewers:
- Ask where the work starts. Put direction questions in the ticket or the thread where the agent’s plan is posted, before it writes the change.
- Ask in the thread. A reviewer who sees a different direction on a finished pull request raises it as a question, and the author decides whether to close or reshape it.
- Treat a closed pull request as a normal outcome. Leave it out of the waste count in your queue metrics.
Keep a sustainable pace when people supervise parallel agents
Section titled “Keep a sustainable pace when people supervise parallel agents”The queue has a human cost that the metrics above see late. Anthropic’s study of its own engineers (December 2025) found some reporting that their work shifted “70%+ to being a code reviewer/reviser rather than a net-new code writer”, and it names a “paradox of supervision”: overseeing agents needs the very skills that over-delegation erodes. Faros AI’s earlier 2025 AI Productivity Paradox report (secondary source: search extracts, not the report itself) put PR review time up about 91% on teams with high AI adoption. Supervising several agents at once is tiring in a way no dashboard shows until someone stops reading.
Adopt these four rules as a team policy, and revisit them at each retrospective:
-
Cap concurrent sessions per engineer at what they can review the same day. Two or three is a reasonable place to start. An agent that finishes at 5 p.m. with nobody to read its output has produced inventory, not progress.
-
Protect review blocks and focus blocks separately. Schedule review in fixed blocks rather than as interrupts, and have people set their GitHub status to Busy during focus time so load-balanced assignment skips them.
-
Rotate a daily queue owner. One person per day watches the queue depth, chases pull requests past the time-to-first-review target, and applies nothing but routing. The role rotates so nobody becomes the permanent reviewer, and so juniors see how the whole queue behaves.
-
Lower concurrency before you lower the bar. When time to first review stays above target for two weeks, when unreviewed merges appear outside lane A, or when two people in the same week say they are only reviewing, reduce the WIP cap and the per-engineer session cap first. Relaxing the lanes to clear the queue is how teams end up looking like the Faros data: more unreviewed merges and more incidents per pull request.
What breaks when you manage the review queue
Section titled “What breaks when you manage the review queue”The WIP cap moves inventory into branches. Engineers hold agent branches locally until the queue has room. Recover by counting open agent branches as well as pull requests in the weekly review, and by gating dispatch rather than pull request creation, as the script above does.
Agents meet the size budget with splits that cannot be reviewed alone. Four 390-line pull requests that only make sense together are one 1,560-line pull request with extra overhead. Require each stacked pull request to pass the suite on its own and state its single concern; reject the stack when it does not.
Lane A grows by habit. A class of change enters the green-pipeline lane because it was convenient in a busy week. Recover by keeping lane membership in the committed table, reviewed by the tech lead, and by dropping a class back to lane B after any revert or escaped defect.
The metrics punish drafts or reward tiny pull requests. createdAt counts draft time, and a team measured on time in review alone can split changes past the point of usefulness. Read time in review together with PR size, reverts, and escaped defects; never report one of them alone.
The busiest reviewer burns out anyway. Load balancing equalises request counts, not difficulty, and lane D reviews cost far more than lane B. Track who carries lane D and rotate it deliberately.
Human time moves instead of shrinking. Approvals per pull request fall, but reverts or incidents rise, so the time shows up in fixing. Recovery: read the guardrail next to the approval counts every week, and move the class of change that changed back one lane.
The sign-off becomes a rubber stamp. When the person who directed the agent is the one required reviewer, a quick approval is the cheapest answer. Recovery: ask the approver what the checks would miss; the article’s test is “If you can’t say what the checks would miss, ask for a real review rather than a quick approval”.
Automated review becomes background noise. If everyone scrolls past the bot, its useful findings are lost too. Cut its blocking list to the few findings that justify a failed build.
Where to go next with the review queue
Section titled “Where to go next with the review queue”Did this guide help you?
One tap, no sign-in.
What will you do with it?
How did it read?
What got in the way? Pick any that apply
Your role?
Thanks. I read every answer. Michał Jaskólski
Did this guide help?
One tap each. Close whenever you like.
no sign-in