Metrics for an AI-native SDLC

Metrics for an AI-native SDLC pair one leading indicator per lifecycle stage, which moves within days of a process change, with one lagging quality or reliability outcome that must not get worse. Both come from data a team already has: the git history of intent.md, spec.md and plan.md, pull request metadata, CI runs, deploys and the incident tracker.

Your team adopted the artifact chain two months ago. Pull requests merge faster, the tech lead says review feels heavier, and nobody can say which stage is slow or whether quality is holding. The DORA dashboard moves once a quarter, which is far too late to tell you that specs are being rewritten halfway through the build.

This page is for the developer who runs the AI-native lifecycle on a repository and the tech lead who reports on one team. It covers the per-stage signals. The organization-level definitions they feed, such as accepted change rate and cost per accepted change, live on one canonical page: DORA, SPACE, DX Core 4 and AI measurement.

What you get from per-stage lifecycle metrics

• A table with one leading and one lagging indicator for each stage and play, and where each number lives.
• Four repository conventions that make every stage metric computable with git log and your code host’s API.
• A decision table that turns a week’s numbers into one action.
• Three copy-paste prompts: one writes the extraction script, one diagnoses a week of data, and one measures whether your instruction file is working.
• A way to prove the numbers are right, and the failure modes that make them lie.

How do leading and lagging indicators work together?

A leading indicator moves within hours or days of a change to one play: commit plan.md before implementing, and time from plan to merged pull request shifts the same week. A lagging indicator is the quality or reliability outcome that takes weeks to show: rework, escaped defects, change fail rate.

Read them as a pair, never alone. A play works when its leading indicator improves and its lagging indicator holds or improves. The evidence for pairing is DORA’s 2025 report (Google Cloud, 2025-09-23), which found “a positive relationship between AI adoption on both software delivery throughput and product performance”, and in the next sentence: “AI adoption does continue to have a negative relationship with software delivery stability.” A panel that shows only speed reports half of that finding.

Leading indicator	Lagging indicator	What it means	What to do this week
Improves	Holds or improves	The play works	Keep it; instrument the next stage
Improves	Gets worse	You automated past the gate instead of through it	Stop expanding agent scope at this stage; tighten the gate’s evidence check
Flat	Holds	The play is not in use, or the data does not see it	Check marker coverage (see the verification section) before you blame the team
Gets worse	Improves	The gate is strict but slow	Automate the evidence check so a human reads findings, not the artifact

Which indicator belongs to each lifecycle stage?

Each row is one play from the lifecycle. The source column names the system that already holds the timestamps, so none of these needs a new analytics product.

Stage and play	Leading indicator	Lagging indicator	Where the data lives
Plan: capture intent.md	Time from the first request to a committed intent.md	Share of intents accepted into design versus closed; edits to intent.md after the first spec.md commit	Git history of the intent folder
Design: write spec.md	Time from the intent.md commit to the accepted spec.md commit	spec.md commits dated after the first plan.md commit for the same change (requirements rework)	Git log
Build: plan mode, then implement	Share of changes that merge from the first implementation pass; time from plan acceptance to merged pull request	Rework cycles per change; how often the merged diff still matches the file list in plan.md	Pull request metadata, drift-review findings
Build: instruction file (CLAUDE.md, AGENTS.md, Cursor Rules)	Review comments that repeat a mistake the file should prevent	Time to first merged pull request for a new teammate	Review comments; git history of the instruction file
Build: skills	Time from a policy owner’s approval to the updated skill merging	Review findings that cite that policy, which should fall toward zero	Pull requests on the skills folder
Build: parallel sessions	Sessions started per engineer per day (a team average, never broken out by person), and peak concurrent sessions, while first-pass merge share holds	Merged changes per repository per week, read beside rework cycles	Agent telemetry; pull request history
Test: session feedback loop	First-pass CI success rate for agent-assisted changes	Time in review per pull request; change fail rate	CI runs; pull request metadata
Test: continuous evals	Eval pass rate after each configuration change; time for an incident to become an eval	Regressions caught in CI versus in production	Eval suite results; incident tracker
Deploy: agent review	Time to first review; share of review threads the authoring agent closed without a human commit	Defects and vulnerabilities caught before merge versus in production	Code host; incident tracker
Deploy: approval-gate hooks	Time each change waits at each gate	Gate violations that reach production, before versus after the hooks	Hook logs; incident tracker
Deploy: CI/CD	Share of pipeline failures triaged without paging a human	DORA’s delivery metrics, per service	CI and deploy tooling
Maintain: close the loop	Time from a control-band breach to an intent.md in triage	Share of findings that become merged fixes; repeat incidents of the same class	Detection script log; pull request history
Maintain: scheduled scans	Share of repositories on a schedule; time from finding to a fix in review	Vulnerabilities found by the scan versus in production or by outside report	Scan history; incident tracker

The last Deploy row deliberately defers to DORA. Its five delivery metrics (change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate) are defined on the metrics frameworks page, and this page does not redefine them.

Make lifecycle metrics computable from git

Most stage metrics are the time between two commits, or a count of commits after a date. They are computable only if the artifacts are committed at the moment they are accepted, in a predictable place.

1. Give every change one folder. Keep intent.md, spec.md and plan.md for a change together, for example intent/notifications-digest/. The artifact chain’s note on where to put the files covers monorepos and multi-repository intent.

2. Commit each artifact when it is accepted, not at the end. The timestamp of the commit that adds the file is the stage’s clock. A spec.md committed together with the implementation makes the design lead time zero and the metric meaningless. Squash merging does the same after the fact: it folds the intent.md, spec.md, plan.md and code commits of a branch into one commit on main, and git log on main cannot recover the separate timestamps. If you squash merge, either merge each accepted artifact in its own small pull request, use merge commits or rebase merging for change branches, or read the per-commit timestamps from the pull request’s commit list through the code host API (on GitHub, GET /repos/{owner}/{repo}/pulls/{pull_number}/commits) instead of from main.

3. Put the change ID in the pull request. A Change: notifications-digest line in the pull request description joins code host data to the artifact folder. A CI check that fails when the line is missing keeps the join from rotting.

4. Mark agent-assisted changes and follow-ups. Label agent-assisted pull requests (agent-assisted, or Claude Code’s own claude-code-assisted label where it is enabled). Reverts use git revert, whose message names the reverted commit. Fix-forward commits carry a Follow-up: #PR trailer that names the pull request they repair.

With those conventions in place, the git side of the table is three commands, all run in a terminal at the repository root:

# When each artifact was first committed: the stage clock for every change
# (%cI is the committer date, the same clock that --since reads)
git log --diff-filter=A --reverse --format='%h %cI' --name-only -- \
  'intent/*/intent.md' 'intent/*/spec.md' 'intent/*/plan.md'

# Requirements rework for one change: spec.md commits after the first plan.md commit
CHANGE=notifications-digest
FIRST_PLAN=$(git log --diff-filter=A --format=%H -- "intent/$CHANGE/plan.md" | tail -1)
git log --oneline "$FIRST_PLAN"..HEAD -- "intent/$CHANGE/spec.md"

# Reverts and fix-forwards in the last 30 days
git log --since=30.days --grep='^This reverts commit' --format='%h %cI %s'
git log --since=30.days --format='%h %(trailers:key=Follow-up,valueonly,separator=%x2C)' | awk 'NF>1'

All three commands read committer dates, so a rebase moves a commit’s clock to the moment it was rewritten; read the numbers from the history you merge, not from a branch that is still being rebased. The rework command counts only spec.md commits that come after the first plan.md commit in history: a spec.md edit made in the same commit that adds plan.md is not rework, and the range $FIRST_PLAN..HEAD leaves it out.

Pull request, CI and deploy data come from your code host’s API. Rather than hand-writing that part, let an agent write the whole extraction as one script, with the prompt below, and commit it.

Copy-paste prompt: write the per-stage extraction script

Write a read-only script, scripts/lifecycle-metrics.py, that computes our AI-native SDLC stage metrics per ISO week for the last 26 weeks. Do not write to git or to GitHub. Read the GitHub token from the GH_TOKEN environment variable.

Conventions in this repository:
- Each change has a folder intent/<change-id>/ holding intent.md, spec.md and plan.md. The commit that first adds a file is that stage's timestamp. If main is squash-merged, read those commit timestamps from each PR's commit list (GET /repos/{owner}/{repo}/pulls/{pull_number}/commits), not from git log on main.
- Pull requests carry a "Change: <change-id>" line in the description, and agent-assisted ones carry the label agent-assisted or claude-code-assisted.
- Reverts are `git revert` commits; fix-forward commits carry a "Follow-up: #<pr>" trailer.

Compute per week, split into agent-assisted and other pull requests:
1. Design lead time: intent.md commit to spec.md commit (median, p90).
2. Requirements rework: spec.md commits dated after the first plan.md commit, per change.
3. First-pass merge share: merged PRs with no follow-up or revert within 14 days and no more than one review round.
4. Time to first review and time in review (median, p90).
5. First-pass CI success rate: PRs whose first CI run on the first pushed commit passed.
6. Marker coverage: share of merged PRs that have a Change line and share that have an agent label.

Output a CSV with one row per week and metric, and a short Markdown summary. Never group by author. Mark every heuristic (such as "one review round") at the top of the summary. Add unit tests that run the script against a small fixture repository you create in a temp directory, and show the test output. Then spot-check five real PRs by hand against the CSV and show the comparison.

Where does each tool record session data?

Every stage metric above comes from git, the code host, CI and the incident tracker, so it is identical whichever agent wrote the code. Only two rows depend on the tool: parallel sessions, and the agent-assisted marker. Full instrumentation, including cost, is on the metrics frameworks page.

Claude Code

Claude Code exports OpenTelemetry metrics and events when telemetry is on. Set it for the team through managed settings. claude_code.session.count counts sessions started, not sessions running at once: count sessions started per engineer per day from it, and derive concurrency from overlapping session.id activity in the exported events:

{
  "env": {
    "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
    "OTEL_METRICS_EXPORTER": "otlp",
    "OTEL_LOGS_EXPORTER": "otlp",
    "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc",
    "OTEL_EXPORTER_OTLP_ENDPOINT": "http://collector.example.com:4317"
  }
}

On Team and Enterprise plans with the Claude GitHub app installed and GitHub analytics turned on, merged pull requests with Claude Code-assisted lines get the claude-code-assisted label. Contribution metrics are in public beta and are not available with Zero Data Retention; use your own agent-assisted label there (checked against the Claude Code analytics docs on 2026-09-26). An Owner turns on the GitHub analytics toggle in Claude Code admin settings.

Codex

Codex exports logs, traces and metrics through an [otel] table in ~/.codex/config.toml, with otlp-http or otlp-grpc exporters. Keep log_user_prompt = false unless your privacy review approves storing prompts:

[otel]
environment = "prod"
log_user_prompt = false
exporter = { otlp-http = { endpoint = "https://otel.example.com/v1/logs", protocol = "binary" } }
metrics_exporter = { otlp-http = { endpoint = "https://otel.example.com/v1/metrics", protocol = "binary" } }

exporter sends logs only and trace_exporter sends traces. Leaving out metrics_exporter does not mean no metrics: release builds then fall back to Codex’s built-in Statsig exporter, which sends usage metrics to OpenAI (ab.chatgpt.com). Point metrics_exporter at your own collector as above, or set it to "none". enabled = false under [analytics] also turns the metrics exporter off, including one aimed at your collector. The field names and this default come from the Codex source (OtelConfigToml, OtelConfig::default and core/src/otel_init.rs, checked 2026-10-02).

Codex CLI 0.157.1 adds no pull request label that we could verify on 2026-09-26, so mark agent-assisted pull requests with your own label. Before you trust session counts from CI, confirm that your collector receives data from codex exec runs: issue openai/codex#12913 reported that codex exec emitted no OpenTelemetry metrics (status not confirmed for 0.157.1 on 2026-09-26), so if exec metrics are missing, count those sessions from log events instead.

Cursor

Cursor offers team usage analytics to team admins, but we could not confirm from a primary source on 2026-10-02 which plans include its analytics dashboard and APIs, or what each one counts as AI-generated code. Check Cursor’s admin docs for your plan before you build on them, and keep your own agent-assisted label as the marker.

For a collector and dashboards, see agent observability; ccusage (npx ccusage@latest) gives per-machine session and cost counts without a server.

Instrument the stages in this order

Start where the data already exists and the leverage is highest, then move upstream. Before you change any play, reconstruct at least three months of history for its indicators, so the first week after the change has something to be compared with.

1. Test and Deploy first. First-pass CI success, time to first review and time in review need no new convention: your CI and code host already hold them. They also show the most common bottleneck once agents write code, which is review.

2. Build next. Add the change ID line and the agent label, and start committing plan.md. First-pass merge share and rework cycles become computable.

3. Design and Plan after that. Once intent.md and spec.md are committed at acceptance, the design lead time and requirements rework rows fill in.

4. Maintain last. The close-the-loop rows need a deterministic detection script and a triage queue first; the Maintain stage sets those up.

5. Review weekly in the retrospective. The tech lead brings the decision table and one week of data, and the team picks one stage to change. The prompt below does the first read.

Copy-paste prompt: diagnose one week of lifecycle metrics

Read metrics/lifecycle-weekly.csv (the output of scripts/lifecycle-metrics.py) for the last eight weeks.

For each lifecycle stage (Plan, Design, Build, Test, Deploy, Maintain):
1. State whether its leading indicator improved, stayed flat or got worse over the last four weeks compared with the four before, and the same for its lagging indicator. Treat changes smaller than the week-to-week spread of the first four weeks as flat.
2. Apply this rule: leading better and lagging worse means we automated past a gate; leading flat means the play is not in use or not measured; leading worse and lagging better means a slow gate.
3. Check marker coverage first. If fewer than 80% of merged PRs carry a Change line, say that the stage numbers are unreliable and stop there for that stage.

Then name the single stage we should change next, the one concrete change you propose, and which indicator should move within two weeks if it works. Do not mention individual engineers. Quote the CSV rows you relied on.

Copy-paste prompt: measure whether the instruction file is working

Use the GitHub API (token in GH_TOKEN, read-only) to collect every review comment on pull requests merged into main in the last 30 days.

Group the comments into recurring mistakes: the same kind of problem raised on at least three different PRs (for example "uses the old API client", "missing tenant check", "test mocks the database"). For each group, show the count, three example PR numbers, and whether CLAUDE.md or AGENTS.md in this repository already has a rule that should have prevented it.

Output a table: mistake, count, covered by an instruction (yes/no, quote the line), proposed new or changed rule. Do not edit any file. Do not name the authors of the PRs or comments.

How do you know the lifecycle metrics are right?

A metric that nobody checks becomes the least-reviewed artifact in the repository. Four checks keep these honest without anyone re-reading the data by hand every week.

• No model computes a number. The extraction script is deterministic, committed and unit-tested against a fixture repository, the same rule the control-band detection follows. Agents write the script and interpret its output; they never produce the numbers.
• Report marker coverage beside every metric. If only half the merged pull requests carry a change ID, a design lead time computed from them describes half the team. Treat a row with coverage below 80% as a warning, not a result.
• Spot-check ten changes a month. Pick ten merged pull requests at random and compare the script’s timestamps, labels and follow-up links with the history by hand. More than one disagreement in ten means the extraction logic is wrong, not the team.
• Keep the pair rule and the unit. Every leading indicator appears on the same view as its lagging one, and every number is per repository or service (team averages are fine), never a view of one engineer.

The tech lead owns the team’s weekly view and signs off each change to a play. The definitions themselves belong to the organization’s metrics owner named on the metrics frameworks page; a team does not change a formula locally.

What goes wrong with lifecycle metrics, and how to recover

A leading indicator becomes a target. Someone sets “specs within one day”, and specs get thinner. Requirements rework then rises in the Build stage. Recovery: remove the target, show design lead time only beside requirements rework, and discuss the pair in the retrospective.

Artifacts are committed in bulk. intent.md, spec.md and plan.md land in the same commit as the code, so every stage lead time reads zero. Recovery: add a CI check that fails a pull request when its change folder has no plan.md in an earlier commit, and reconstruct history only from changes that pass it.

Squash merge erases the stage clock. Each artifact was committed on time on the branch, but the squash put them all into one commit on main, so every stage lead time reads zero again and git log on main cannot tell. Recovery: merge each accepted artifact in its own small pull request, switch change branches to merge commits or rebase merging, or have the extraction script read per-commit timestamps from each pull request’s commit list through the code host API rather than from main.

Rework hides as new work. A fix for last week’s agent change arrives as a new feature pull request with no link, so first-pass merge share looks perfect. Recovery: make the Follow-up: #PR trailer part of the review checklist, and have the review agent flag fixes to recently merged code that lack it.

Parallel sessions rise and rework rises with them. Throughput looks better while each change needs more rounds. Recovery: cap concurrent sessions per engineer at the last level where first-pass merge share held, and raise it again only when the Test stage’s first-pass CI success improves.

Review load moves to a few seniors. Time in review grows, but only for pull requests that need two particular reviewers. Recovery: split time in review by reviewer group (not by person), set pull request size budgets, and see how to keep the review queue moving.

Numbers turn into a leaderboard. Per-engineer views appear, and the team starts optimizing the counts. Recovery: delete the per-person views, state in the team’s working agreement that these metrics describe the process, and keep individual performance out of them.

Where to go next with lifecycle metrics

Start from the lifecycle overview if the stages are new to you. When the per-stage signals are in place, roll them up with the organization-level definitions.

DORA, SPACE, DX Core 4 and AI measurement Canonical AI metric definitions and the minimal panel per level of the organization.

Maintain: close the loop Turn a metric breach into an intent.md with deterministic detection.

Observing agents in operation Telemetry pipelines and dashboards for agent sessions, cost and traces.

Tool map: Claude Code, Cursor, and Codex Which lifecycle capabilities each tool provides today.