Skip to content

@wondelai/harness-audit is an open-source CLI that scores any repository from Level 0 to Level 5 across the six Software Factory stations—Intent, Harness, Loop, Graph, Verification, and Release—by inspecting committed files, CI gates, test oracles, and agent configurations locally in under two seconds. Developers, tech leads, and CTOs use it to replace subjective self-assessment with verifiable repository evidence.

Most engineering teams overestimate their autonomy level because questionnaires measure aspirations rather than what actually sits in Git. A team lead may believe strict guardrails are live while CLAUDE.md has bloated past 400 lines, .codex/config.toml still specifies the retired untrusted approval policy, and CI runs zero protection against agents weakening test files.

Subjective forms record how a team hopes it works. @wondelai/harness-audit reads the exact environment Claude Code, Codex, and Cursor encounter when opening a session, mapping every finding onto Dan Shapiro’s Autonomy Ladder (January 2026) and the six Software Factory stations:

StationWhat deterministic repository checks verifyCore canon guide
1. IntentConcise AGENTS.md (≤ 150 lines), thin @AGENTS.md delegation in CLAUDE.md, scoped rules, skills, and spec/PR templatesAGENTS.md & context
2. HarnessExplicit permission/sandbox posture (flagging retired Codex untrusted), lifecycle hooks with stop_hook_active, MCP hygiene, and secret protectionPermissions & sandboxes · Agent hooks
3. LoopUnified single-command verification (lint, typecheck, test, build), strict static typing, deterministic formatter, committed lockfileAgent-ready codebase
4. GraphWorktree workflow scripts or .worktreeinclude, isolated local ports/state, and scoped subagent definitionsGit worktrees · Ephemeral environments
5. VerificationReal unit and E2E suites, CODEOWNERS/CI oracle protection, low .skip/suppression slop, automated PR review (e.g. .cursor/BUGBOT.md), fitness functionsOracle strength · Protecting the oracle
6. ReleasePR CI gates, evidence-bundle PR template, automated dependency checks, and production error/telemetry observabilityEvidence bundle · Codebase health

For the complete per-check inventory, see the harness audit checks reference.

Why hard level caps prevent inflated Level 4 scores

Section titled “Why hard level caps prevent inflated Level 4 scores”

A repository with a spotless AGENTS.md and twenty Agent Skills—yet no test suite, no oracle protection, and weak CI gates—still operates at Level 2. Averaging raw points hides structural weaknesses, so @wondelai/harness-audit applies non-negotiable level caps:

Claimed LevelRequired minimum evidence in the repository
Level 1–2Baseline instruction file and reproducible build/test commands
Level 3Single-source instructions, deterministic verification loop, explicit permission posture, and CI gates
Level 4Strong Verification station (real tests, oracle protection, automated PR review) plus Graph isolation
Level 5Full six-station coverage including evidence bundles, architectural fitness functions, and release observability

Level caps make the audit useful for CTOs and platform leads comparing dozens of repositories: a high Intent score cannot disguise an unguarded test oracle or missing sandbox boundary.

  1. Run the zero-config local scan. Execute from any repository root—zero API keys, zero network calls, zero code uploaded:

    Terminal window
    npx @wondelai/harness-audit
  2. Capture workflow signals files cannot show. Add the optional 7-question terminal interview for team adoption, PR review queue age, and cost visibility:

    Terminal window
    npx @wondelai/harness-audit --interview --role lead
  3. Generate a tailored fix prompt or minimal baseline. Export a repository-specific remediation prompt for Claude Code, Codex, or Cursor—or preview a clean non-destructive scaffold before writing anything:

    Terminal window
    npx @wondelai/harness-audit --prompt --out HARNESS-FIX-PROMPT.md
    npx @wondelai/harness-audit init --dry-run
  4. Share reports and enforce a regression floor in CI. Export a standalone HTML report for leadership reviews and add a CI gate so harness scores never silently decay on future PRs:

    Terminal window
    npx @wondelai/harness-audit --format html --out harness-report.html
    npx @wondelai/harness-audit --ci --min-score 65

Verify that audit remediation actually improved safety

Section titled “Verify that audit remediation actually improved safety”

Raising a score only counts when the underlying controls catch real defects. Pair every audit-driven pull request with three verification checks:

  • Re-run the deterministic scan in CI: Confirm --ci --min-score passes without weakening thresholds and that no level cap remains active on the station you fixed.
  • Run live canary probes: Adding a hook file or sandbox block satisfies static detection; prove refusal with hook fixtures, a sandbox canary, and a codebase defect canary.
  • Track multi-repo baselines over time: Store periodic HTML or JSON outputs across core repositories so engineering leadership tracks real structural progress rather than raw AI code volume (metrics frameworks).
  • Teams game the score with empty boilerplate. Adding placeholder AGENTS.md files, dummy hooks, or tautological tests ticks static checkboxes without improving safety. Pair static scans with --deep inspection, slop detection, and canary tests.
  • Engineers chase points in strong stations while ignoring level caps. Polishing Intent from 85 to 95 does not raise an organization capped at Level 2 by weak Verification. Fix the lowest-scoring gating station first.
  • Running init without auditing existing configs. Always preview with npx @wondelai/harness-audit init --dry-run first so generated scaffolds align with your team’s existing AGENTS.md, package manager, and CI layout.
  • Treating one audit as a permanent certification. Tool conventions, model capabilities, and CI configs drift every quarter. Keep --ci --min-score in pull-request workflows and re-evaluate your threshold after every major upgrade.

Interactive guide, books, and team support

Section titled “Interactive guide, books, and team support”

Explore the interactive breakdown at /en/harness-audit/, browse the open canon editions at /en/books/, or consult official vendor documentation for tool-specific remediation syntax: Claude Code docs, OpenAI Codex docs, Cursor docs, Agent Skills, and MCP.

Did this guide help you?

One tap, no sign-in.

not at all a lot