Team adoption — measure effective workflow use
AI adoption is effective use of an approved workflow, not seats purchased, prompts sent, or a universal percentage target. A mature team knows which roles and tasks are eligible, whether people can complete those workflows, whether accepted delivery improves, and whether quality or control health deteriorates. People who should not use a workflow are not “non-adopters.”
Q1 · Adoption and tooling Max-score evidence: broad eligible-team use paired with workflow completion, accepted outcomes, quality, and control evidence.
Define adoption before counting it
Section titled “Define adoption before counting it”Use a denominator tied to the workflow:
effective adoption = eligible engineers completing the approved workflow successfully / eligible engineers with access and trainingA successful completion must reach the workflow’s evidence boundary—for example, an accepted plan, a reviewed PR with passing checks, or a resolved incident record. Opening the tool is not completion.
Measure four layers
Section titled “Measure four layers”| Layer | Question | Example evidence |
|---|---|---|
| Access | Can eligible people start safely? | managed account, policy acceptance, working setup |
| Capability | Can they complete the workflow? | onboarding fixture and sampled run |
| Outcome | Does accepted delivery improve? | lead time or task success for matched work |
| Guardrail | Is harm controlled? | defects, reverts, denied actions, review load |
- Select one workflow. Examples: planning a bounded feature, repairing a failing test, or reviewing a PR.
- Define eligibility and success. Exclude roles and tasks for which the workflow is inappropriate.
- Run a baseline and pilot. Compare like-for-like work; record team, risk, sample size, and confounders.
- Remove observed friction. Fix access, repository context, examples, verification, or review queues based on failed runs.
- Expand only with guardrails. Broaden access when the outcome improves without violating quality and control thresholds.
Prompts for the adoption review
Section titled “Prompts for the adoption review”Define eligibility, successful completion, accepted outcome, and quality guardrails for this AI workflow. Do not use logins, prompt count, or seat count as success.Analyze failed or abandoned runs. Classify friction as access, context, capability, tool reliability, review queue, or unsuitable task. Recommend the smallest intervention and how to test it.Compare the pilot with matched baseline work. Report sample size, completion, accepted lead time, quality, control events, and confounders. Recommend expand, narrow, retrain, or stop.Acceptance evidence
Section titled “Acceptance evidence”- Definitions and denominators are published and stable across the comparison period.
- Telemetry is proportionate, privacy-reviewed, and does not store sensitive prompts unnecessarily.
- Qualitative interviews explain failures that aggregate usage cannot.
- The organization does not penalize people for avoiding an unsuitable or unsafe workflow.
- Expansion has a named owner, threshold, review date, and rollback decision.
Failure pattern: adoption theater
Section titled “Failure pattern: adoption theater”Purchasing licenses for everyone, mandating a weekly prompt count, or celebrating a fixed percentage can increase activity while delivery quality falls. Treat adoption as a diagnostic layer. The goal is a better accepted outcome from eligible work, with lower friction and controlled risk.
Continue with onboarding
Section titled “Continue with onboarding”Use Developer onboarding time to test capability, then connect adoption to the AI metrics panel.