Using Documentation as Effective AI Context

Level 2 How-to Updated 2026-10-12

Documentation as context is project knowledge written into files that a coding agent loads on its own: CLAUDE.md for Claude Code, AGENTS.md for Codex, and Rules for Cursor. The file carries what the code cannot show, such as commands, conventions, and deliberate exceptions. It only helps while it stays short, specific, and checked against the repository.

This page is for developers who set up an agent on a repository and for tech leads who own the shared instruction file. You recognise the problem: the agent runs npm test when your repository uses pnpm vitest, rebuilds a helper that already exists, or “fixes” a pattern the team chose on purpose. Each session starts with no memory of the last one, so the agent pays to rediscover the same facts, and it sometimes rediscovers them wrongly. New to context budgets? Start with Understanding context windows. For the short version of what a repository file should contain, see AGENTS.md and CLAUDE.md: concise repository context.

What a self-checking instruction file gives you

A clear split between what belongs in the always-loaded file and what should load only on demand.
How Claude Code, Codex, and Cursor find and combine instruction files, including the size limits that cut files silently.
Three copy-paste prompts: draft a file, audit it, and record a lesson after a repeated mistake.
A CI check and a fresh-session probe that show the file is still true, without anyone rereading it line by line.

What documentation should an agent load, and when?

Agent-facing documentation comes in three layers. The mistake that makes files long is putting all three into the first one.

Layer	What goes in it	When it loads	Where it lives
Always loaded	Commands, conventions that differ from defaults, “never do X” constraints	Every session, every turn	`CLAUDE.md`, `AGENTS.md`, Cursor Rules that always apply
Loaded on demand	Rules for one directory or file type, multi-step procedures, design decisions	When the agent touches matching files, or when it decides a skill is relevant	Path-scoped rules, nested instruction files, skills, docs the file points to
Fetched live	Third-party library APIs, internal docs outside the repository	When the agent calls a tool	MCP servers such as Context7

The always-loaded layer is expensive, because every line is paid for on every turn. It is worth a line only when removing that line would make the agent do the wrong thing. The other two layers cost context only when the agent uses them.

How do Claude Code, Codex, and Cursor load instruction files?

The three tools share the idea but differ in file names, merge order, and limits. These details decide whether your most specific instructions reach the model.

Claude Code reads CLAUDE.md at the start of every session (memory docs, read 2026-09-26).

repo/
  CLAUDE.md            # team instructions, committed (or .claude/CLAUDE.md)
  CLAUDE.local.md      # personal notes, add to .gitignore
  .claude/rules/
    testing.md         # no `paths:` frontmatter, loads at launch
    api.md             # `paths: ["src/api/**/*.ts"]`, loads only for matching files
  packages/web/
    CLAUDE.md          # loads when Claude reads a file in packages/web/

Hierarchy. Files in your working directory and every directory above it load at launch and are concatenated root first, so the file closest to where you started is read last. Files in subdirectories load when Claude reads a file there.
Path-scoped rules. A rule in .claude/rules/ with a paths list in its frontmatter loads only when Claude reads a matching file. paths is the only frontmatter field Claude Code reads from a rule.
Imports do not save context. @path/to/file expands at launch, so an import organises a long file without making it cheaper.
Free maintainer notes. Block-level HTML comments () are stripped before the file reaches the model.
AGENTS.md. From v2.1.277 (v2.1.281 on Bedrock, Google Cloud, Foundry, LLM gateways and telemetry-off sessions; on both release channels since stable reached v2.1.287, checked 2026-10-12), Claude Code reads AGENTS.md when there is no CLAUDE.md, .claude/CLAUDE.md, or CLAUDE.local.md in the working directory or above it. If both exist, it reads only the CLAUDE.md files. To keep one shared file, put @AGENTS.md at the top of CLAUDE.md and add Claude-only lines below it, or set Project instructions in /config to claude-md-and-agents-md to load both.
Size. The docs target under 200 lines per CLAUDE.md. The 200-line (or 25 KB) cut-off applies to auto memory’s MEMORY.md, not to CLAUDE.md, which loads in full up to 4 MiB.

Codex reads AGENTS.md before it does any work. The loader below was checked in the Codex 0.157.1 source (codex-rs/core/src/agents_md.rs).

repo/                      # project root = nearest ancestor with .git
  AGENTS.md
  services/payments/
    AGENTS.override.md     # used instead of AGENTS.md in this directory
    AGENTS.md              # ignored while the override exists

One file per directory. Codex walks from the project root down to the directory you started in, never past the root and never below your starting directory. In each directory it takes the first file it finds, in this order: AGENTS.override.md, AGENTS.md, then any names in project_doc_fallback_filenames. An override therefore replaces the AGENTS.md beside it; it does not add to it.
Concatenated root first. The files found are joined in root-to-start order.
A 32 KiB budget. project_doc_max_bytes defaults to 32 KiB across all project files. The file that crosses the budget is cut, and every file after it is skipped. Because the most specific files come last, they are the ones lost. The warning goes to Codex’s log, not the terminal UI.
Trust. Since 0.150.0, an untrusted project supplies no project-level AGENTS.md at all.
/init writes a starting AGENTS.md.

Cursor’s mechanism is Rules: “system-level instructions to Agent” (Rules docs, fetched 2026-08-28).

The rule file format, as Cursor’s docs give it (checked 2026-10-12):

Project rules live in .cursor/rules as .mdc files; a plain .md file there is ignored.
The frontmatter fields are description, globs and alwaysApply; rules apply to Agent (Chat), not to Tab completion or Inline Edit.
Cursor supports AGENTS.md in the project root and in subdirectories, with the more specific nested file taking precedence, and it reads CLAUDE.md the same way, always applied to every conversation.
How to confirm a scoped rule loaded: ask the agent which rules it applied after it opens a matching file.

Project rules in Cursor walks through writing and scoping rules on this site.

The content advice on this page applies unchanged: one topic per rule, commands and exceptions rather than general advice, and scoping for anything that matters in only one part of the codebase.

What belongs in an agent instruction file?

Use one test for every line: would the agent get this wrong without the line? If not, delete it.

Keep	Example
Commands the agent cannot guess	`pnpm vitest run src/billing`, `make db-reset` before integration tests
Conventions that differ from defaults	“Money is integer cents; never use floats”
Deliberate exceptions, with the reason	“We do not use an ORM in `src/reports/`; raw SQL is faster for these queries”
Hard constraints	“Never edit files under `generated/`; run `pnpm codegen` instead”
Where to look, not what is there	“The canonical API handler is `src/api/users.ts`”

Cut	Why
A project overview or architecture summary	The agent reads the code faster than it reads your summary, and the summary drifts
A file-by-file map	Stale within weeks, and a stale map actively misleads
Language conventions and “write clean code”	The model already follows them; the line is pure cost
Long procedures, such as the release checklist	Move them to a skill or a linked doc that loads on demand
Anything that changes weekly	Put it where it is generated, not where it is remembered

Specific beats emphatic. Compare these two versions of the same intent:

# Weak: nothing here changes what the agent does
We care about quality. Write clean, well-tested code and handle errors properly.

# Strong: each line prevents a specific mistake
- Run one test file: pnpm vitest run <path>
- API errors return { error: string, code: number }; never throw from a handler
- Integration tests use the real database (docker compose up db); never mock it
- Never commit to main; open a branch and a pull request

Build an instruction file the agent follows

Generate a draft, then cut it. Run /init in Claude Code or Codex, or use the first prompt below in any tool. Treat the output as raw material. Generated files tend to repeat what the code already shows, so expect to delete much of it.
Run every command in it. A command the agent copies from the file and that then fails costs more than a missing command. Keep only commands that pass on a clean checkout.
Move scoped rules out of the root file. Anything that matters in only one directory goes into a path-scoped rule (.claude/rules/ with paths), a nested AGENTS.md, or a Cursor rule scoped as the Rules docs describe. Procedures become skills.
Choose one source of truth for mixed teams. If the team uses more than one tool, keep the shared content in AGENTS.md and make CLAUDE.md a one-line @AGENTS.md import plus any Claude-only lines. Cursor reads AGENTS.md in the project root and subdirectories (Cursor docs, checked 2026-10-12), so it needs no copy; confirm it with the probe below. Keep .cursor/rules for rules only Cursor needs, not for a copy of the shared file. Where you do keep per-tool copies, use a sync tool, as described in One rules source for every agent. Two hand-maintained copies drift apart quickly.
Add the drift check and the probe from the next section, and commit the file with an owner in CODEOWNERS.
Add a rule only after the second occurrence. When the agent makes the same mistake twice, add one specific line. After a single failure, fix the code and move on. Pruning CLAUDE.md and AGENTS.md covers the other direction: removing lines that no longer earn their place.

Draft an AGENTS.md for this repository that I will cut down. It is a draft,
not a finished file.

Include only:
1. Build, test, lint, and single-test commands. Run each one and report
   which pass on this checkout. Mark any command I could not guess from
   package.json, the Makefile, or the CI config.
2. Conventions that differ from the language's defaults, each with one
   file path that shows it.
3. Setup steps a new clone needs before the tests pass.
4. Places where this repo deliberately does not do the obvious thing,
   with the reason if the code or commit history shows it.

Do not include a project overview, an architecture summary, or a map of
directories. Stay under 80 lines. Put a "(?)" after any line you are not
sure about instead of padding the file.

Audit our agent instruction files (CLAUDE.md, CLAUDE.local.md, AGENTS.md,
.claude/rules/, and Cursor rules if present). Do not edit anything yet.

For each line, classify it as:
- KEEP: you would get this wrong without it
- CUT: you would do this correctly anyway, or it describes code you can read
- STALE: it names a command, path, script, or dependency that no longer
  exists (check package.json, the lockfile, and the file tree)
- MOVE: it matters for only one directory or task; name the path-scoped
  rule, nested file, or skill it should move to
- CONFLICT: it contradicts another line; quote both

Then report the current line count of each file and show the proposed
version of the root file as a diff.

You have now made this mistake twice: you ran the whole test suite with
`pnpm test` when the change only touched src/billing, and it took 9 minutes.

Add one rule to AGENTS.md that prevents it. The rule must:
- be one line, with the exact command to use instead
- go in the existing Commands section, not a new section
- replace any existing line it overlaps with

Show me the diff. If a line of similar intent already exists, tell me
why it did not work instead of adding a second one.

In Claude Code, /doctor prompt-audit (v2.1.283 and later, both release channels on 2026-10-12) runs a similar audit over the same files and proposes edits without applying them. It runs through the bundled /claude-api skill, so it is unavailable when disableBundledSkills is set or that skill is turned off in skillOverrides.

How do you prove the instruction file still matches the repository?

Nobody rereads a 90-line file every week, so make the file check itself. Two checks cover most drift: a CI script that fails when the file names a path or script that no longer exists, and a probe that asks a fresh agent session questions only the file can answer.

1. A drift check in CI. Save this as scripts/check-agent-docs.sh and run it in the same job as your linter. It fails when a file goes over its line budget, names a src/, scripts/, tests/, docs/, or packages/ path that does not exist, or names an npm run script that package.json does not define. Extend the patterns to pnpm, make, or whatever your file uses.

#!/usr/bin/env bash
# Fails when an agent instruction file drifts from the repository.
set -uo pipefail
status=0
for f in AGENTS.md CLAUDE.md; do
  [ -f "$f" ] || continue
  n=$(wc -l < "$f")
  if [ "$n" -gt 200 ]; then echo "$f: $n lines, budget is 200"; status=1; fi
  # Every backticked repo path must exist (globs and <placeholders> are skipped).
  for p in $(grep -oE '`(src|scripts|tests|docs|packages)/[^` ]*`' "$f" | tr -d '`' | grep -v '[*<]' | sort -u); do
    [ -e "$p" ] || { echo "$f: path not found: $p"; status=1; }
  done
  # Every "npm run X" must be a script in package.json.
  for s in $(grep -oE 'npm run [A-Za-z0-9:_-]+' "$f" | awk '{print $3}' | sort -u); do
    node -e "process.exit(require('./package.json').scripts?.[process.argv[1]] ? 0 : 1)" "$s" \
      || { echo "$f: npm script not found: $s"; status=1; }
  done
done
exit $status

Against a file that names a deleted src/routes/ directory and an npm run lint script that was renamed, it prints both lines and exits with 1.

2. A fresh-session probe. Write five questions whose answers live only in the instruction file, such as “Which command runs one test file?” and “Where do generated files come from?” Run them in a new session after every change to the file.

--tools "" disables every tool, so Claude can answer only from what it loaded at startup. --tools takes a list, so put the prompt before it (or after --); otherwise the prompt is read as a tool name:

claude -p "From your project instructions only: which command runs a single test file, and which directory must you never edit by hand?" --tools ""

codex exec has no switch that turns off file reading, so tell it not to open files and run it read-only. Run it in a project Codex already trusts; in an untrusted checkout Codex loads no AGENTS.md and every answer will be wrong.

codex exec --sandbox read-only --ephemeral "Do not open any files. From your AGENTS.md instructions only: which command runs a single test file, and which directory must you never edit by hand?"

A wrong or hedged answer means the line is missing, buried, or contradicted by another file. Compare the answers with the expected ones in the pull request that changes the file. The file’s owner in CODEOWNERS signs off on that pull request; the CI check and the probe are the evidence, so the reviewer reads the diff and the probe answers, not the whole file.

When should documentation come from an MCP server instead of the file?

Library documentation does not belong in your instruction file. It is long, it changes with every release, and the agent needs only the page for the API it is calling. Fetch it live instead.

Context7 (Upstash) serves version-specific library docs over MCP. It is the most-starred single-purpose MCP server in our catalogue at 62.4k GitHub stars (upstash/context7, read 2026-09-26). The API key is optional and raises rate limits.

claude mcp add --scope user --transport http context7 https://mcp.context7.com/mcp

codex mcp add context7 --url https://mcp.context7.com/mcp

Add this to .cursor/mcp.json in the project, or to ~/.cursor/mcp.json for every project (paths and shape from the Context7 README):

{ "mcpServers": { "context7": { "url": "https://mcp.context7.com/mcp" } } }

Context7 also ships a CLI + Skills mode with no MCP server: npx ctx7 setup --claude (or --cursor) installs a skill instead (npm ctx7 0.5.12, checked 2026-09-26). The trade-off: the skill adds no tool definitions to context and fetches docs through the CLI only when it runs; the MCP server exposes resolve-library-id and query-docs as tools the agent can call in any session.

Then add one line to the instruction file, for example: “For any Next.js or Drizzle API, fetch the docs with context7 before writing code.” The file stays short and the docs stay current. Internal documentation that lives outside the repository, such as runbooks and architecture decision records, follows the same rule: link or serve it, do not paste it. Documentation MCP servers compares the options.

What breaks when documentation is used as context?

The agent ignores a rule that is clearly in the file. The file is usually too long, or another line contradicts it. Do not add “IMPORTANT” or bold: Anthropic’s memory docs note that longer files reduce adherence, so every added line weakens the rest. Delete other lines to make room, move scoped rules out, and run the probe again. If the rule must hold every time, enforce it with a hook or a CI gate instead of a sentence.

Codex silently loses the most specific instructions. In a monorepo, the root and intermediate AGENTS.md files use up the 32 KiB budget and the file nearest your working directory is cut. Check the combined size with wc -c on each file in the chain, shrink the root file first, or raise project_doc_max_bytes in config.toml. If Codex loads no project AGENTS.md at all, check trust first: since 0.150.0 an untrusted project, such as a fresh clone or a CI checkout, supplies none.

Claude Code does not pick up your AGENTS.md. Any CLAUDE.md, .claude/CLAUDE.md, or CLAUDE.local.md in the directory tree takes precedence, and a build older than v2.1.277 predates the feature. Add @AGENTS.md to the top of CLAUDE.md, which works on every version, or set Project instructions in /config to claude-md-and-agents-md to load both. Note that a personal CLAUDE.local.md alone is enough to switch AGENTS.md off.

A committed file tells the agent to do something harmful. Instruction files are part of your supply chain: a pull request can add a line that tells the agent to run a script or send data somewhere. Review instruction-file changes like code, require the CODEOWNERS owner, and never let a CI agent with secrets load instruction files from an untrusted branch.

The file and the repository disagree after a migration. The drift check catches renamed scripts and deleted paths. It does not catch a changed convention, so add “update AGENTS.md” to the checklist for any pull request that changes tooling, and rerun the probe questions.

The file grows back after every incident. Each postmortem adds a line and nothing is ever removed. Set a line budget in the drift check and prune on a schedule, and whenever you change models, with the ablation protocol.

Where to go next with documentation as context

Pruning CLAUDE.md and AGENTS.md The other direction: find which lines still change the agent's behaviour and delete the rest.

Understanding context windows Why every always-loaded line costs you on every turn, and where the limits are.

Memory patterns Auto memory and cross-session notes: what the agent records on its own versus what you write.

AGENTS.md and CLAUDE.md: concise repository context The short checklist for what a repository instruction file must contain.

One rules source for every agent Keep AGENTS.md, CLAUDE.md and Cursor rules in step with Ruler or rulesync.

Making a codebase agent-ready Tests, scripts, and structure that let the agent verify its own work, so the file can stay short.