CTO Scorecard Guide

The CTO Scorecard maps your engineering org against the 2026 AI tooling maturity ladder: Reactive → Coordinated → Optimized → Strategic Leader. This guide is the implementation companion: 25 playbooks — one per question — covering adoption, governance, parallelism, compliance, and ROI.

Section 1 · Adoption & spend

Q1 · Team adoption rate Crossing the 70% active‑user threshold where compounding effects kick in.

Q2 · Tooling policy Standardized stack + exception process — the middle ground between Shadow AI and forced single‑tool.

Q3 · Team billing Team/Enterprise plans (Anthropic Teams, Cursor Teams, OpenAI Business) with centralized audit.

Q4 · Cost visibility Full FinOps: per‑dev + per‑repo + per‑PR + alerts + model tagging.

Section 2 · Shared infrastructure

Q5 · Shared agent rules Multi‑tier governance: managed policy + per‑repo CLAUDE.md + .claude/rules/.

Q6 · Shared skills Company skills repo + bootstrap script + onboarding hook.

Q7 · Internal MCP servers MCP platform team — multiple servers, dedicated owner, docs, monitoring.

Q8 · MCP security model Allowlist + scoped tokens + log audit + periodic red‑team.

Section 3 · Quality gates

Q9 · Team PR review automation Layered: CodeRabbit / Greptile + Claude Code Action / Codex / Sentry Seer + ultrareview.

Q10 · Gates vs guardrails Design‑time policy beats per‑PR auto‑review for review bandwidth.

Q11 · AI‑PR labelling Auto‑label + extra gates for AI‑authored PRs (full test suite + security scan + 2× reviewer).

Q12 · E2E policy E2E required + agent runs browser tests itself before merge.

Q13 · AI in CI/CD Full pipeline: agent generates, agent reviews, agent tests, agent deploys — with hard gates.

Section 4 · Parallelism at team scale

Q14 · Team parallelism (Tier 2) Conductor, Claude Squad, Cursor + worktrees — and how to keep merge conflicts solved.

Q15 · Tier 3 overnight runs Curated 'AI eligible' backlog, scheduled runs, review‑on‑arrival in the morning.

Q16 · Team vibe‑coding policy Per‑tier policy: Lovable / Bolt / v0 for MVPs, Cursor / Claude Code for production.

Section 5 · Org enablement

Q17 · Plan mode policy Hard‑enforced Plan mode for sensitive changes — hook + managed policy.

Q18 · Shared hooks governance Repo + auto‑install via dotfiles bootstrap + signed/audited hook scripts.

Q19 · Dev onboarding time Sub‑day setup: bootstrap script + CLAUDE.md + hooks + skills + MCP + sample sessions.

Q20 · Knowledge sharing ai‑toolkit‑internal repo + regular brown‑bag sessions.

Q21 · Compliance policy Allowlist + log audit + PII scrubbers + DPAs + GDPR/HIPAA mapping.

Section 6 · Strategy & ROI

Q22 · AI metrics panel Spend, throughput, quality, adoption, review‑to‑merge, cost‑per‑feature.

Q23 · ROI measurement Annual dashboard: pre‑AI baseline vs current, $/dev saved, headcount equivalent.

Q24 · AI tooling roadmap 6–12 month roadmap + dedicated AI tooling lead/team with budget.

Q25 · Vendor risk management Multi‑vendor + abstraction (router, gateway, OpenRouter / Bedrock / Vertex) + DR plan.

Not sure where you stand?

Take the CTO Scorecard 25 questions, ~12 minutes. Tier rating + 30/60/90 plan + risk register.

Developer Scorecard Guide Building your own setup as an IC? Jump to the developer playbook set.