Software Factories: How to Build One and Keep It From Rotting

A software factory is a pipeline that lets coding agents pull work from a queue, build and test it, and merge it with little or no human reading the diff. Factories hold up when each loop’s autonomy is capped by a verification oracle the team trusts more than the model, and they rot when autonomy is declared factory-wide because the harness looks impressive.

You run six agents. By Thursday there are forty pull requests, two people who can meaningfully review them, and a queue that is now the whole company’s critical path. The Faros AI 2026 telemetry study puts numbers on that exact week: median time to first review up 156.6%, and pull requests merged with no review at all up 31.3%.

The obvious move is to automate the review too. That is the move the industry spent 2026 arguing about, and the argument is worth understanding before you copy anyone’s pipeline, because the two loudest voices are both right and they are not arguing about the same thing.

What you’ll walk away with from running a software factory

• A two-axis map of the factory spectrum, with real companies plotted on it, so “are we a software factory” stops being a vibe
• The autonomy ledger: four questions per loop that decide whether it may run unattended, and which answer disqualifies it
• A build order for the factory floor, including the context feed that most published pipelines leave out
• The six metrics that detect rot before your incident count does, and the WIP limit that prevents it
• Four copy-paste prompts: a factory-readiness audit, a spec that carries production context, an adversarial gate reviewer, and a weekly erosion sweep

What a software factory actually is

The loop is not complicated, which is part of why it spreads so fast:

1. Humans decide what to build and put it in a ticketing system.
2. Agents build and test a change that satisfies the requirement, then open it for CI and review.
3. Approved code deploys to production through CD.
4. Rejected code goes back to an agent to iterate until it passes.
5. Users engage with what shipped and emit data, feedback, and signals that inform what gets built next.

None of that is new. The scaffolding is version control, CI/CD, and monitoring, and the standardised, repeatable process is the thing every team already claims to have. What changed is who stands at the stations.

The metaphor is also older than it looks. Hitachi opened its Software Works in 1969, and NEC, Toshiba, and Fujitsu followed through the 1970s, all chasing process standardisation and reuse. Michael Cusumano’s study of that era found the factories genuinely good at routine, production-oriented software and persistently weak at operating systems and applications, which is a useful prior: factories optimise the work whose quality you can specify in advance.

Where does your team sit on the software factory spectrum?

There is no membership test, so treat it as two axes: how much of the code agents write, and how closely humans still read it. Published coordinates, as of August 2026:

Team	Agent-written share	Human reading	Where that lands
Most teams	Assisted, human-authored PRs	Full review	Not a factory, just fast autocomplete
Ramp, Cursor, Uber	Background agents on real tickets. Uber reports ~1,800 fully agent-written changes per week, ~11% of live backend updates, with 95% of engineers using AI monthly	Full review	Lit factory
PostHog	~70% of PRs	Humans skim at least 80%	Lit, with the lights dimmed
Stripe	Minions merge 1,300+ PRs a week containing zero human-written code	Human-reviewed	Lit, at scale
StrongDM	All of it. Three people, 32,000 lines of production security software	None. Charter forbids humans writing or reviewing code	Lights-off

The top-right corner is the contested one. A “lights-off” or “dark” factory needs three things at once, and teams usually have one:

• Real AI adoption. Access to coding agents is not usage. A factory where half the team still hand-writes PRs is a factory with a permanent manual station in the middle of it.
• Validation to the point of mergeability. Peer review by another agent, local execution, browser clickthroughs. An agent writing unit tests for its own code and then reasoning about that code is not validation; it is the same model grading its own exam.
• Guardrails that assume failure. Sandboxes, scoped permissions, deterministic checks. Humans stay in the loop as decision-makers on high-risk changes only.

Why the software factory debate blew up in 2026

Two arguments landed weeks apart, and taking either one alone will build you the wrong pipeline.

Dex Horthy’s case: harness engineering is not enough

HumanLayer’s CEO ran a lights-off factory from July 2025 and shut it down that November. Agents wrote, reviewed, and deployed without human review, and inside about four months the codebase had degraded far enough that a single bug took weeks of manual debugging to find. His conclusion, delivered as the “Harness Engineering is not Enough” keynote at AI Engineer World’s Fair and then as an essay series, is that the failure was not in the pipeline. It was in the models.

The mechanism is the reward signal. Coding agents are trained against one-shot correctness: if the tests pass, the rollout wins. Nothing in that loop penalises eroding the codebase on the way to green. And the reason nothing penalises it is that there is no fast oracle for maintainability, because whether a design was good depends on what someone asks for next, which is unknown at design time and unique to every codebase.

That claim stopped being rhetorical when SlopCodeBench measured it. Instead of single-shot tasks, the benchmark makes agents repeatedly extend their own prior solutions under evolving specifications, then tracks two trajectory-level signals: verbosity (redundant or duplicated code) and structural erosion (complexity mass concentrating in a few high-complexity functions). The results, from Orlanski et al.:

• No agent solves any problem end to end. The best strict per-checkpoint rate is 14.8%.
• Structural erosion rises in 77% of trajectories; verbosity rises in 75.5%.
• Agent code is 2.3x more verbose and 2.0x more eroded than human code on the same problems.
• Human repositories degrade less often and by smaller margins across their real development history.
• Prompting for quality up front cut initial erosion and verbosity by up to a third without changing the rate of decay.

That last bullet is the one to sit with. Better instructions moved the starting point and left the slope alone. No amount of harness engineering patches a missing reward signal, which is why Horthy’s recommendation is to keep reading the code, or to move human effort upstream into planning.

PostHog’s case: the pipeline starves the agent

PostHog’s reply does not dispute the erosion. It disputes the diagram.

Every published software factory enforces a clean separation between deciding what to build and building it. A human does the planning, compresses the result into a ticket, and hands it to an agent with zero prior context on the problem. Horthy’s own remedy keeps that shape: a better quality oracle would be whether output passes all verifiers for an incrementally-divulged spec, and the spec still arrives pre-written.

PostHog’s objection is that this is not how good product engineering works. Designing a system well needs the technical constraints and a real understanding of the problem, which is why product engineers talk to users and why, at PostHog, engineers rather than PMs decide what to build. Agents inherit that requirement. One makes better architectural calls when it can see how the product is actually used, who the customer is, what users are complaining about, and what else is moving in the codebase.

Their own numbers show why this bites hardest on maintainability: 63% of PostHog’s changed lines land in files that already exist, and fix is their largest commit type at 40%. Most software work is a reaction to observed usage. When you last chased an incident, you leaned on logs and traces, not on someone’s secondhand summary of the problem, and an agent handed a ticket is working from the secondhand summary every single time.

Why autonomy is earned per loop, not declared

Both cases survive scrutiny, and they compose. Horthy identifies a ceiling you cannot engineer past. PostHog identifies a self-inflicted wound sitting well below that ceiling. Fixing the wound does not raise the ceiling, and neither one alone survives contact with a real codebase.

The practical consequence is that “are we a dark factory” is the wrong question, because it is asked at the wrong granularity. A factory is not one autonomy setting. It is dozens of loops with wildly different verification economics, and the useful unit of decision is the loop.

Addy Osmani names the constraint: you can hand a loop as much autonomy as you can cheaply and reliably verify, and not one inch more. He calls it back pressure, and it is the reason a funnel with a wide mouth and a narrow gate does not go faster when you widen the mouth.

The autonomy ledger

Before a loop runs unattended, answer four questions about it in writing:

1. What oracle decides “done”? Name the specific thing. “Tests pass” is an oracle. “It looks right” is not.
2. Can the agent fake it? If the agent can edit the oracle, the oracle is a suggestion. Tests inside the repo fail this question by construction.
3. How long until a wrong answer surfaces? Seconds is a green light. “A human notices in three weeks” is a red one, and it is the answer for every architectural decision, which is precisely Horthy’s point.
4. What is the blast radius if it is wrong? Reversible and contained, or a schema migration.

A loop that answers cleanly on all four can run dark today. A loop that fails question 2 or 3 stays lit no matter how good your harness is.

Loop	Oracle	Fakeable	Feedback latency	Verdict
Dependency bumps, CVE patches	Build plus full test suite across consumers	No, the suite predates the change	Minutes	Runs dark
Flaky test stabilisation	Run the test 1,000 times, measure the failure rate	No, it is statistical	Minutes	Runs dark
Mechanical codemods, lint and type fixes	Compiler and linter	No	Seconds	Runs dark
Feature work in a well-covered module	Existing tests plus new ones the agent wrote	Yes, the agent owns both sides	Minutes	Lit: human reads the plan
Schema, auth, pricing, migrations	Production behaviour	No, but it is slow	Days to weeks	Lit: human reads the diff
Architectural refactors	None that runs in CI	N/A	Weeks to months	Lit, and front-load the planning

One more constraint from the same literature: loops stay reliable at roughly 3 to 10 steps, and past about 20 the agent loses the thread as context accumulates. A “loop” that needs forty steps is a pipeline you have not decomposed yet.

Tip

Copy-paste prompt for a factory-readiness audit:

Audit this repository for agentic factory readiness and produce an autonomy
ledger. Do not propose a pipeline yet.

Step 1. Inventory the classes of change this repo actually receives. Use
`git log --since="6 months ago" --name-only` and group commits by what they
touched, not by commit message prefix. Report the share of changed lines that
landed in files that already existed versus brand new files.

Step 2. For each class of change, fill in this table honestly:
  - the specific oracle that decides "done" (name the command)
  - whether an agent editing this repo could weaken or delete that oracle
  - how long after a wrong answer merges before anything surfaces it
  - the blast radius of a wrong answer

Step 3. Rank the classes into three buckets: can run unattended today, needs a
human reading the plan, needs a human reading the diff. Justify every
"unattended" with the answers from step 2, and flag any where the only oracle
is a test file the agent would also be editing.

Step 4. List the missing infrastructure that would let the top-ranked lit
class move down a tier, cheapest first. Be concrete about what does not exist
yet rather than recommending generic best practices.

How to build the factory floor

Build in this order. Steps 1 and 2 are the ones teams skip, and skipping them is what produces a fast pipeline that ships erosion.

1. Make the repo legible to an agent. An AGENTS.md or CLAUDE.md that states the architecture and the non-obvious constraints, typed boundaries between modules, and test seams that pin behaviour. Osmani’s point is worth repeating because it is unglamorous: strong signatures, clear organisation, legible call stacks, and dependency injection are not new practices, they are the safety net that decides how much damage a wrong agent decision does.

2. Wire the context feed. This is PostHog’s argument made operational, and it is the step almost no published pipeline includes. Before an agent plans anything, it should be able to reach the same evidence a human engineer would open: error traces, logs, usage data, session replays, and the actual customer complaint. Connect the analytics and error-tracking MCP servers your team already pays for and make reading them part of the planning step, not an optional tool the agent might call. An agent that reads the Sentry issue writes a different fix than one handed a ticket that says “checkout is broken sometimes.”

3. Build the queue. A label on an issue is enough to start. The queue’s job is to make the work list explicit and rate-limited, so what enters the factory is a decision rather than an accident.

4. Build the gate, in order of cost. Deterministic checks first: compiler, linter, type checker, test suite, and any property or contract tests. Then an LLM reviewer prompted adversarially. Then a human, on the classes your ledger says need one. Running the expensive reviewer before the free ones burns tokens on diffs that a type error would have rejected.

5. Sandbox, scope, and meter. Each agent gets an isolated environment, credentials scoped to what its loop needs, and a spend meter. Ask the question a factory operator would: in the system’s worst hour, which operations are mediated, which identities apply, what limits are enforced, what evidence survives, and where must a human authorise continuation? Any path that bypasses your gateway (direct API calls, shell access, browser use) needs its own answer.

6. Instrument the factory itself. You are now operating a production system whose output is code. If you cannot see queue depth, review latency, and rework rate per loop, you are flying it blind.

Caution

Tests inside the repo are not a gate against the agent that can edit them. This is the failure that makes step 4 look fine while producing nothing. StrongDM’s answer is worth stealing regardless of what you think of the rest of their approach: they keep behavioural scenarios as a holdout set external to the codebase, exactly like a validation split in training. The agent cannot see or modify what it is being scored against. If your only oracle lives in a directory the agent has write access to, downgrade every loop that depends on it by one tier.

Which tool runs the factory floor?

The three tools occupy different surfaces. This split matches the one in continuous delivery, and the same reasoning applies here.

Cursor

Cursor covers the review station best. BugBot reviews PRs automatically once enabled on a repo and posts inline comments, re-triggered on demand with bugbot run. Autofix, GA since February 2026, spawns a background Cloud Agent that opens a follow-up PR with the proposed fix, so the gate produces a diff to approve rather than a comment to action. Background and Cloud Agents handle the build station, and the Agents Window is where you watch the fleet.

Pick Cursor when the factory’s review gate lives in the GitHub UI and you want failures to come back as PRs.

Claude Code

Claude Code is the scriptable station. claude -p runs headless inside a GitHub Action, which is how you wire a reviewer, a gate, or a scheduled sweep. The flags carry the guardrails: --allowedTools restricts what the agent may touch, and --output-format json makes the result parseable so a job can fail on it. PreToolUse hooks stop a risky command before it runs, and subagents let one loop delegate without inheriting the whole context.

Pick Claude Code when the factory is defined in .github/workflows and you want every station invoked from a script with explicit permissions.

Codex

Codex is the async station. Codex Cloud runs tasks in a hosted environment rather than on your machine, the GitHub integration opens and reviews PRs, the Slack integration lets a teammate start a task from a channel, and Automations run recurring prompts and deliver findings to the task inbox. codex exec covers non-interactive workers when you want a local loop that exits.

Pick Codex when work should enter the factory from chat and approvals should happen there too.

Tip

Copy-paste prompt for a spec that carries production context:

You are writing the implementation plan for the issue below. Do not write code
yet, and do not trust the issue description as a complete account of the
problem.

First, gather evidence:
- Pull the related error-tracking issue: full stack trace, frequency, first
  seen, affected releases, and which users hit it.
- Pull usage data for the surface involved: how often the path runs, what the
  drop-off looks like, and whether the behaviour changed recently.
- Find every existing file that already touches this behaviour. Report what
  fraction of the work is extending existing code versus adding new files.
- Search for prior attempts at this in git history and say what happened.

Then write the plan, and open it with a section titled "What the evidence says
the problem actually is" that either confirms or contradicts the issue
description. If they contradict, stop and say so rather than planning around it.

The plan must name the files it will change, the existing abstraction it will
extend rather than duplicate, and the specific oracle that will prove it worked.
If no such oracle exists, say that instead of inventing a test that only checks
the code you are about to write.

Issue:

How to manage a factory once it runs

Building it is a weekend. Running it without accumulating debt you cannot see is the job.

Set the WIP limit by review capacity, not token budget

The factory is a funnel with a wide mouth and a narrow gate. Adding agents widens the mouth. If the gate is unchanged, the only thing that grows is the queue in front of it, and queues in front of gates turn into merges without gates. That is not a prediction: Faros measured pull requests merged with no review at all rising 31.3%, with 7-day-stalled work up 26% and daily PR contexts per developer up 67.4%.

Cap concurrent agents at the number of changes your gate can genuinely clear per day, then raise the cap only by making the gate faster or cheaper. Every other lever is theatre. Team parallelism covers calibrating that ceiling per developer.

Watch the six metrics that detect rot early

Your incident count is a lagging indicator, and by the time it moves the erosion is months deep. Track these instead, split by agent-authored versus human-authored files so the comparison is visible:

Metric	What it tells you	Faros’ 2026 baseline
Code churn (deleted-to-added line ratio on merged code)	Throughput that is not real	Up 861%
Incidents per PR	Whether the gate is working	Up 242.7%
Bugs per developer	Defect rate escaping the gate	Up 54% (from 9% in the 2025 report)
PRs merged without review	Whether the gate is being skipped	Up 31.3%
Time to first review	Where the queue is forming	Median up 156.6%
Duplication and complexity concentration	SlopCodeBench’s two erosion signals, locally	Not in the report; measure it yourself

The Faros study covers 22,000 developers across 4,000+ teams over two years, comparing each organisation’s lowest and highest AI-adoption periods, and it is worth quoting in both directions. The same dataset shows real gains: epics completed per developer up 66.2%, task throughput up 33.7%, PR merge rate up 16.2%. The output is genuine. So is the bill. Anyone selling you one half of that table is selling something.

Tip

Copy-paste prompt for a weekly erosion sweep:

Run an erosion sweep on the code our agents wrote this week and report on the
two signals that CI cannot see.

1. Identify the agent-authored changes. Use `git log --since="7 days ago"` with
   whatever trailer, author, or branch prefix our factory stamps on them, and
   tell me which identifier you used so I can correct it.

2. Verbosity. Run `npx jscpd --min-lines 8 --reporters json` over the changed
   paths. Report the duplication ratio, then find the near-duplicates jscpd
   misses: functions that reimplement an existing helper with different names.
   List each one next to the helper it should have called.

3. Structural erosion. Rank every function in the changed files by cyclomatic
   complexity and report what share of total complexity sits in the top 10% of
   functions. Compare that share against the same files one month ago.

4. Report the trend, not just the level. A stable ratio is fine. A ratio that
   climbed every week for a month is the failure mode we are looking for, and I
   want it flagged as such.

Output a table sorted by severity, and for the worst three, propose the specific
refactor with the existing abstraction it should collapse into. Do not open a PR.

Meter cost per loop, not per seat

Autonomous loops spend without a human deciding each time, which makes budgets behave differently. StrongDM treats $1,000 per engineer per day in tokens as a floor rather than a ceiling, on the reasoning that under-spending means under-automating. Uber went the other way and capped employee AI spending after burning through its budget in four months.

Both are the same lesson from opposite ends: the meter belongs on the loop. A dependency-bump loop that runs dark and costs pennies per change should not share a budget line with an architectural refactor loop that burns a fortune and still needs a human. Cost governance covers the mechanics.

Fix the platform before you scale the factory

DORA’s research on AI-assisted development lands on a finding that is inconvenient and well-evidenced: AI is an amplifier. Its effect on organisational performance depends on internal platform quality. Where platform quality is low the effect is negligible; where it is high the effect is strong and positive. Investing in agents without investing in the platform is local optimisation.

Read that alongside the Faros finding that organisations with mature DevOps practices experienced the same downstream deterioration as everyone else. The two are not contradictory: a good platform raises what the factory can produce, and it does not by itself stop erosion. You need the platform and the ledger.

What breaks in a software factory

Nobody understands the code any more. The gap between how much code exists and how much any human can still explain widens quietly, and it only becomes visible when a bug needs someone to reason about the system. Recovery is expensive: quarantine the affected area, write down the behaviour you actually need from it, and regenerate from that spec rather than debugging code nobody owns. Prevention is cheaper: read plans even on loops where you have stopped reading diffs.

The agent edited the test instead of the code. The suite is green, the bug is in production, and the diff touched the assertion. This is question 2 of the ledger going unanswered. Move the oracle out of reach, keep behavioural scenarios external to the repo, and treat any green run where the test file changed as an unreviewed change.

Erosion that no check catches. Every gate passes and the codebase still gets worse, because duplication and complexity concentration are invisible to compilers and test suites. Schedule the erosion sweep above and treat its trend line as a real signal.

Review theatre. Approvals arrive within seconds of the PR opening, on diffs too large to have been read. A skimmed 800-line diff is worse than an unread one because it produces a record saying somebody looked. Audit a sample properly instead of skimming everything, and shrink PR size, which Faros found ballooned 51.3% under high AI adoption.

Queue starvation and stalls. Work enters, an agent picks it up, and it sits. Alert on age in state rather than on state, and route anything past its threshold to a human instead of another retry.

Blast radius through the paths you did not mediate. Your gateway sees the model calls it proxies. It does not see direct API calls, shell commands, or browser sessions. Enumerate the unmediated paths explicitly, because an autonomous loop with broad credentials is a different security posture than an engineer with the same ones. MCP security covers the tool-access side.

Tip

Copy-paste prompt for the adversarial gate reviewer:

Review this diff as the last automated gate before a human sees it. Assume the
author was an agent optimising to make checks pass, and that it had write
access to the tests.

Report only findings you can tie to a specific failure. For each one, state the
inputs or state that trigger it and what goes wrong. Skip style, naming, and
anything a linter already covers.

Check, in this order:
1. Did this change weaken its own oracle? Flag any modified, deleted, skipped,
   or loosened assertion, and say whether the production code justifies it.
2. Does this duplicate logic that already exists in the repo? Search before
   concluding it does not, and name the function it should have called.
3. What does this do on the unhappy path: empty input, concurrent callers,
   partial failure, and a downstream timeout?
4. What is the blast radius if this is wrong in production, and is it reversible
   without a deploy?
5. What did the author not touch that this change implies should also change?

End with a single line: BLOCK or PASS, and if BLOCK, the one finding that
justifies it. Do not hedge, and do not pad the list to look thorough.

Where to go next with software factories

tmux for agent fleets The layer below the factory: seeing which agent is blocked, surviving a closed laptop, passing work between six

herdr: an agent multiplexer Agent state detection and a socket API, for when screen-scraping the fleet stops scaling

Continuous delivery Wiring the gate: headless reviewers, pipeline YAML, deploy gates, and red-build triage

Human-AI collaboration patterns Where the human stays in the loop, and what changes when they step out of it

Quality gates and enforcement The deterministic half of the gate, and how to make it cheap enough to run first

AI code review checklist What the review station should actually look for once agents write most of the diff