Knowledge sharing — version evidence, not tricks

Knowledge sharing for AI-assisted engineering, as CTO Scorecard Q20 scores it, means turning lessons from accepted changes, failed agent runs, review findings, and incidents into versioned artifacts with an owner and a source. Top score needs those artifacts updated from real evidence and exercised in recurring sessions.

This page is for the tech lead who keeps the team’s agent setup working and the CTO who answers Q20. The situation: in June an engineer spent an afternoon finding out why the agent kept writing migrations with the wrong timestamp format, and posted the fix in a chat thread. In September a new engineer hits the same failure, the thread is gone, and the team wiki still recommends two commands the tools have since renamed.

What you get from a versioned knowledge loop

• The Q20 scoring table, and how it relates to the knowledge-sharing question on the Tech Lead Scorecard.
• A routing table that sends each lesson to one home: an instruction file, a skill, a hook, a test, a runbook, or a decision record.
• A lesson record template with owner, provenance, evidence, and a retirement trigger, ready to commit.
• The per-tool traps that keep lessons in one person’s memory, and a headless probe that proves a fresh agent session finds the shared version.
• A 30-minute recurring session agenda, three copy-paste prompts, the measures that show the loop works, and its failure modes.

How does CTO Scorecard Q20 score knowledge sharing?

The scorecard asks “How does the team share AI tooling know-how (skills, prompts, MCPs, tricks)?” and scores four answers.

Points	Answer	Evidence that earns it
0	None, ad-hoc Slack	None.
1	Slack or Discord channel, no archive	A channel people scroll back through.
2	Internal doc or wiki with patterns	A page of prompts and configurations, usually without owners or dates.
3	Versioned patterns with owners and evidence, updated from reviews and incidents and exercised in recurring team sessions	The knowledge index, lesson records linked to the failures that produced them, discovery-probe results, and the minutes of the last two sessions.

The step from 2 to 3 is the one that matters. A wiki records what somebody believed once. A versioned artifact with an owner, a source, and a test is something the team can check, change through review, and delete when a tool release makes it wrong.

The Tech Lead Scorecard asks the same question from the team side (“Do you run internal knowledge-sharing on AI workflows?”) and gives its top score to “Regular cadence + captured artifacts”. The loop on this page satisfies both.

Route each lesson to one canonical home

Most knowledge rot starts with a lesson stored in the wrong place: a hard rule written as advice in a Markdown file, or a one-person preference copied into every tool’s instruction file. Route each lesson by what has to happen for it to take effect.

When the lesson is…	Its home is…	Canonical page	Proof that it works
A repository fact every session needs (commands, protected paths, conventions)	The shared instruction core, AGENTS.md or CLAUDE.md	Shared agent rules	A fresh session quotes it correctly
A procedure for a recurring task	A skill (SKILL.md package)	Shared skills	Positive and negative trigger fixtures
Something that must never happen	A hook, permission rule, or CI gate	Shared hooks governance	A blocked-action test
A defect class that escaped review	A test, property check, or lint rule	Failure taxonomy	The check fails on the defective commit
An operational response (rollback, credential rotation)	A runbook	Maintain loop	A dry run by someone who did not write it
A choice between options (tool, model, workflow)	A decision record	Architecture decisions	Baseline, result, and a review date
A new idea nobody has tested	The learning stream: a demo or a session topic	This page, “Run a recurring AI knowledge-sharing session”	Becomes one of the rows above, or is dropped

Two rules keep the table honest. First, prefer the earliest control that catches the problem: a lesson that can be a test should not be a sentence in AGENTS.md, because the model may ignore a sentence and CI cannot ignore a test. Second, each lesson has exactly one home; other files link to it instead of copying it.

Write a lesson record the team can check

A lesson record is the metadata that lets anyone decide in one minute whether a piece of knowledge still holds. Keep one per lesson in docs/agent-knowledge/, and list them in docs/agent-knowledge/INDEX.md so there is one place to search.

---
id: KN-014
title: Migrations must use UTC timestamps in the file name
owner: "@payments-lead"
status: active            # draft | active | superseded | retired
home: AGENTS.md#migrations   # the one canonical artifact this lesson lives in
source: incident INC-2291 (2026-06-12), PR #4410 review finding
evidence: tests/migrations/test_filename_format.py fails on commit 3f9c2e1
tools_checked: Claude Code 2.1.283, Codex CLI 0.157.1
last_verified: 2026-09-26
review_by: 2026-12-26
retire_when: the migration generator enforces UTC itself
---

## What happened
The agent generated migrations named with local time; two ran out of order in staging.

## What changed
Added the rule to AGENTS.md#migrations and a filename test that fails on local-time names.

## How we know it holds
The test runs in CI on every PR. The discovery probe asks a fresh session
for the migration naming rule and checks the answer quotes AGENTS.md.

Four fields do the work. source ties the lesson to a real failure, so nobody keeps a rule whose reason is lost. home enforces the single canonical location. evidence names the check that fails when the lesson is violated. retire_when states in advance what makes the lesson obsolete, which is what lets the team delete it without a debate.

Add a CODEOWNERS entry for docs/agent-knowledge/ and for each instruction and skill path, so every change to shared knowledge gets a review from its owner. For the dated tool facts that lessons depend on (versions, flags, defaults) and the CI check that fails on retired terms, reuse the setup in keeping a team current rather than building a second one.

Run the AI knowledge-sharing loop in five steps

1. Capture the signal where it appears. Give the team one intake: a knowledge label on issues, or a short form that files one. Signals worth capturing are a repeated question, a failed or abandoned agent run, a review finding the agent repeats, an escaped defect, an onboarding block, and an experiment result. A signal is one line plus a link to the run, PR, or incident.

2. Route it to one home within a week. The owner of the intake applies the routing table above, or closes the signal as “not durable” with a reason. Unrouted signals older than two weeks are the first item at the next session.

3. Change the artifact through a pull request with evidence. The PR updates the home artifact and adds or updates the lesson record. It carries a reproducer or a failing check, and CI runs any skill fixtures or hook tests the change touches. The owner in CODEOWNERS approves.

4. Prove discovery in a fresh session. Run the discovery probe (next section) in each approved tool. A lesson that a fresh agent session cannot find does not exist for the next engineer, whatever the wiki says.

5. Review and retire on schedule. On or before review_by, the owner re-checks the evidence and either renews the date or sets status: retired and removes the text from its home. Superseded lessons point to their replacement for one release, then go.

Keep personal memory from becoming the team’s source

Claude Code and Codex both keep some knowledge per person, and that is where team lessons quietly get stuck. The rule is the same in all three tools: personal memory is a draft, and anything the team needs moves into the repository through review.

Claude Code

Checked against Claude Code 2.1.283.

Claude Code reads CLAUDE.md files and can “accumulate learnings automatically with auto memory” (memory docs). Auto memory is machine-local: it lives under ~/.claude/projects/<project>/memory/ and is not shared across machines or through the repository, so a lesson it captured helps only the person who hit the failure. /memory lists the CLAUDE.md files, toggles auto memory, and opens its folder; review what auto memory learned and promote the team-relevant parts into the repository’s instruction file or a skill through a PR.

Claude Code reads AGENTS.md only when the project has no CLAUDE.md, and only from v2.1.277, which is on both release channels since stable reached v2.1.287 (checked 2026-10-12). If your shared core is AGENTS.md, keep a thin CLAUDE.md that points to it, as shared agent rules describes.

Codex

Checked against Codex CLI 0.157.1.

Codex’s memories feature is stable but off by default (codex features list), and /memories configures its use and generation. If your team enables it, treat it like auto memory in Claude Code: a personal draft that feeds pull requests, not the shared source.

The bigger trap is trust. Since Codex 0.150.0 an untrusted project does not supply its project-level AGENTS.md, so an engineer who declines the trust prompt runs without any of the shared lessons and nothing fails loudly. Run the discovery probe in a trusted checkout.

Cursor

Cursor configures project instructions through Rules (rules docs, checked 2026-10-12). Keep team lessons in committed project rules, not personal settings. Cursor also reads AGENTS.md in the project root and in subdirectories, and CLAUDE.md the same way (Cursor docs, checked 2026-10-12), but a fresh Agent chat still has to find the lesson. Paste the probe question from the next section (the quoted string in the bash block) into a new Agent chat and record the answer with the other tools’ results.

Prove a fresh agent session finds the lesson

Reading the lesson record proves nothing about whether an agent will apply it. The discovery probe asks a fresh, read-only session a question the lesson answers, and checks the answer against the record. Run it from the repository root in a terminal, and keep the output as the evidence for step 4.

# Claude Code 2.1.283: plan mode does not edit files; the session is not saved to disk
# needs CLAUDE.md, or AGENTS.md (v2.1.277+)
claude -p --permission-mode plan --no-session-persistence \
  "How must new database migration files be named in this repository? Quote the file and line that says so. If no instruction covers it, answer NOT FOUND." \
  > probe-KN-014-claude.txt

# Codex CLI 0.157.1: read-only sandbox, no session files, final answer to a file
# run in a trusted checkout: untrusted projects don't load AGENTS.md (0.150.0+)
codex exec --sandbox read-only --ephemeral -o probe-KN-014-codex.txt \
  "How must new database migration files be named in this repository? Quote the file and line that says so. If no instruction covers it, answer NOT FOUND."

A pass is an answer that states the rule and quotes the home named in the record, or the file it is imported from (a CLAUDE.md that points to AGENTS.md counts). NOT FOUND, a paraphrase from the code, or a quote from a stale wiki page is a fail, and the fix belongs to the owner. Run the probes for every lesson touched in a release, and for the whole index once a quarter.

Copy-paste prompts for knowledge maintenance

Run these in the approved tool from the repository root. They work the same in Claude Code, Codex, and Cursor.

Copy-paste prompt: turn a failed run into the smallest durable artifact

Read the failed run, review comment, or incident I paste below, then read AGENTS.md (or CLAUDE.md), .agents/skills/, docs/agent-knowledge/INDEX.md, and the CI configuration.
1. State the lesson in one sentence.
2. Say whether existing guidance already covers it; if so, quote it and explain why it did not prevent the failure.
3. Choose the single home for the lesson: instruction file, skill, hook or permission rule, test or lint rule, runbook, or decision record. Prefer the earliest automated control. Explain the choice in two sentences.
4. Draft the change to that home and a lesson record with id, owner, source, evidence, tools_checked, review_by, and retire_when.
Do not copy the lesson into more than one file. Do not edit anything until I accept the plan.

Copy-paste prompt: audit the knowledge index for rot

Read docs/agent-knowledge/INDEX.md and every lesson record it lists. For each record, report in a table:
- review_by in the past, or last_verified older than 90 days
- a home that no longer exists or no longer contains the lesson
- the same rule stated in more than one file (quote both locations)
- an evidence check that no longer exists in the test suite or CI
- a tool command, flag, or model name, with the tool version the record says it was checked against
- no source, or a source that is only a chat message
Do not change any file. End with the five records most likely to mislead a new engineer, and why.

Copy-paste prompt: test whether a new engineer can find and apply a lesson

Act as an engineer who joined today and has only this repository. Without reading docs/agent-knowledge/, work out how to add a database migration that backfills a new column.
List every file you read, in order, and the rule you took from each. Then compare your answer with docs/agent-knowledge/KN-014.md and report: whether you found the rule, how many files it took, any instruction that contradicted it, and the link or sentence that would have led you there sooner.
Do not edit anything.

Replace KN-014 and the migration task with a lesson from your own index. The third prompt tests the path a person follows, which the headless probe does not.

Run a recurring AI knowledge-sharing session

The scorecard’s top answer requires that patterns are “exercised in recurring team sessions”. Run a 30-minute knowledge session every two weeks with a rotating facilitator, separate from any demo slot.

Minutes	Item	Output
5	Open signals older than one week	Each one routed or closed with a reason
10	One lesson from the last two weeks, shown on the real failure: the run, the fix, the check	Team agrees the home is right, or it moves
10	One lesson due for review: rerun its probe and its evidence check live	Renewed, rewritten, or retired
5	One new tool behaviour someone saw this fortnight	A signal filed, or “nothing” recorded

Write the minutes into docs/agent-knowledge/sessions/ with the date and the artifacts changed. A session with no artifact change and no explicit “nothing” did not count. Skills training belongs in review-to-learn sessions; this session is about the knowledge base itself.

How do you verify knowledge sharing works without reading every page?

The tech lead should not reread the whole knowledge base each month. The loop is verified from what it produces.

• Repeat failures fall. Tag agent failures by class using the failure taxonomy. A failure class that has a lesson record and still recurs means the lesson is in the wrong home or is not found.
• Probes pass. The discovery probe results for every lesson touched in a release are attached to the release PR. The quarterly full run is kept with the session minutes.
• Evidence checks run in CI. Every evidence field names a test, a fixture, or a hook test that CI runs; a record whose check has been deleted fails the audit prompt.
• Nothing is overdue. A scheduled job lists records past review_by. The count at each session is a number the team can see go down.
• Retirements happen. A knowledge base where nothing has been retired in a quarter is growing without being maintained.
• A named owner signs off. The tech lead owns the index and accepts each quarter’s audit; the CTO’s Q20 evidence is the index, the last two sets of session minutes, and the latest probe results.

What breaks when teams share AI knowledge?

The tips archive. A wiki page of 60 prompts and copied configurations that nobody owns, half of it written for tool versions that are gone. Recovery: import only items that have a source and a current owner into lesson records, and archive the rest with a date. Items nobody claims within two weeks stay archived.

The same rule in five files. A lesson copied into CLAUDE.md, AGENTS.md, a Cursor rule, a skill, and a README drifts apart within a month, and the agent follows whichever it read last. Recovery: choose one home, replace the copies with links, and add the “more than one file” check from the audit prompt to the quarterly run. If you must keep per-tool files, generate them from one source (rules sync) instead of editing each copy.

Advice where a control belongs. “Never edit the generated client” as a sentence fails the first time the model ignores it. Recovery: move it to a permission rule, hook, or CI check, and keep the sentence only as the explanation.

Lessons live in personal memory. Auto memory or a private rules file fixed the problem for one engineer, and everyone else still hits it. Recovery: make “promote personal memory” a standing question in the session, and add the probe to developer onboarding so a new engineer surfaces the gap.

Stale vendor facts. A lesson says a flag or default that a release changed, and the agent follows it. Recovery: record tools_checked in every lesson, date-stamp tool facts in one file, and fail CI on retired terms as keeping a team current describes.

Sessions turn into demos. The meeting shows new features every fortnight and no artifact changes. Recovery: keep the agenda above, and record “nothing” explicitly when nothing changed so the pattern is visible.

Where to go next with knowledge sharing

The prerequisite for this loop is a shared instruction core and a skills setup that already work; the next step is feeding production evidence into it automatically.

Q5 · Shared agent rules The instruction core most lessons land in.

Shared skills Package procedures with fixtures and pinned distribution.

Maintain Turn production findings into new intent automatically.

Upskilling a team Katas and review-to-learn sessions for the people side.

For every other question on the scorecard, start from the CTO Scorecard answer key. For the tech lead’s version of this question, see the Tech Lead Scorecard answer key.