The Human's Job in Agentic Engineering

The human’s job in agentic engineering is the set of decisions no verification oracle can make for an agent: taste, architecture, product direction, the design of the evals and stop conditions, the permissions agents run with, and the call on which loops must not run unattended. Accountability for each decision stays with a named person.

Your team’s agents now open more pull requests in a day than the team used to write in a week. Review has quietly turned into approval, an overnight loop ran with whatever permissions the last session left behind, and nobody can say who decided that. This page is for the developer who wants to know what their job is now, the tech lead who has to write it down for a team, and the CTO or executive who has to answer for it.

What you get from mapping the human’s job

• The six responsibilities that do not transfer to an agent, each traced to a named source.
• A four-question test that decides, per loop, whether it may run unattended.
• An autonomy register row you can paste into your handbook, so every unattended loop has an owner, a reason and a review date.
• The per-tool commands that turn a permission decision into configuration.
• Four checks that prove the human decisions still hold without anyone reading every diff.
• Three copy-paste prompts, and a recovery step for each way the human’s job quietly disappears.

Why does the human’s job move instead of shrinking?

The work changes shape. Andrej Karpathy’s published summary of his Sequoia Ascent talk (30 April 2026) puts it as “The unit of programming changed from typing lines of code to delegating larger ‘macro actions’.” BCG Platinion, writing as consultants in March 2026, says the same in a sentence a board understands: “The defining shift is not the absence of humans; it is the relocation of human effort.” Simon Willison proposed vibe engineering for experienced engineers who work faster with agents while staying accountable for the software; he later said agentic engineering was winning as the name.

At Level 2 on the autonomy ladder, all six responsibilities below hide inside the act of writing code, so they feel automatic and nobody names them. From Level 3 upward, each one has to be written into a file, a gate or a policy. An agent does not infer them, and a passing test suite does not report their absence.

Which six responsibilities stay human?

Responsibility	The decision	Where it is written down	Source
Taste	This working implementation is the wrong one	Architecture brief, review rubric	Karpathy (transcript), April 2026
Architecture	Boundaries, layering and dependency direction	Instruction file, fitness functions	Karpathy (transcript), April 2026; DORA, September 2025
Product direction	What to build, and what should not exist	Spec, acceptance criteria	Karpathy (transcript), April 2026; Shapiro, January 2026
Eval and stop-condition design	What “done” means, as commands that exit 0	Test suite, eval suite, CI config	Shapiro; Stripe, February 2026; Naresh, July 2025
Permissions	What an agent may run without asking	Settings files, sandbox profile	Claude Code and Codex docs
The call not to automate	Which loops run unattended, and who answers for them	Autonomy register	Linear, August 2025

Taste

In his published transcript, Karpathy is direct about the agents’ present standing: “Right now the agents are like interns. You still have to be in charge of aesthetics, judgment, taste, and oversight.” Taste is the judgement that a working implementation is the wrong one: the abstraction that will cost a year, the dependency that solves today’s problem and owns tomorrow’s, the API that is correct and unusable. No test fails on any of that, which is why it stays with a person.

Architecture

“You still have to understand the fundamentals,” Karpathy says in his talk (published transcript), and “you cannot be a good director if you do not understand.” Architecture has the slowest feedback loop in the job: wrong answers surface months after the run that made them. DORA’s 2025 report (September 2025) found that “Teams working in loosely coupled architectures with fast feedback loops see gains, while those constrained by tightly coupled systems and slow processes see little or no benefit.” The architecture decides how much of the acceleration a team keeps.

Product direction

In his published transcript, Karpathy names the bottleneck: “I am becoming the bottleneck of even knowing what we are trying to build, why it is worth doing, and how to direct my agents.” Dan Shapiro’s Level 4 human (The Five Levels, January 2026) writes a spec and works more like an engineering or product manager than a coder. An agent builds what the spec says with more patience than any team. It does not tell you the feature should not exist.

Eval and stop-condition design

This responsibility grows as the others shrink. Shapiro’s Level 4 is “leave for 12 hours, and check to see if the tests pass”, which is only safe if somebody designed the tests. Naresh (DEV Community, July 2025) states the case for determinism: “A model can argue that its work is complete. A deterministic validator can prove that a required field is missing.” Stripe’s Minions, described by Alistair Gray in February 2026, run against “over three million” existing tests for “at most two rounds of CI”, after which “we send the branch back to its human operator for manual scrutiny.”

The oracle is the one part of the system an agent must never author. Choosing it is a human job; keeping it out of the agent’s reach is a design constraint. Protect the oracle shows how.

Permissions

What an agent may run unsupervised is a decision every tool asks you to make. Codex documents an approval policy for “when Codex must ask you before it executes an action (for example, leaving the sandbox, using the network, or running commands outside a trusted set)”. Claude Code’s auto mode routes “tool calls through a classifier that blocks anything irreversible, destructive, or aimed outside your environment”. Cursor documents run modes for the same choice.

Defaults are not a policy. From Claude Code v2.1.283 on the latest channel, interactive sessions on supported models start in auto mode, while claude -p starts in Manual; Codex’s default approval policy is on-request. An unattended run at 3 a.m. inherits whichever of those applies, and that is a decision somebody made by not making it.

The call not to automate

Linear’s write-up on its Agent Interaction SDK (Leela Senthil Nathan, August 2025) states the principle plainly: “an agent cannot be held accountable.” The product enforces it: “issues can only be assigned to humans, and only delegated to agents”, so “the issue still has a human assignee—someone accountable for the result—but it also has a delegated agent responsible for taking action.”

Accountability does not delegate. Somebody decides what runs unattended and answers for it, and that decision has a shape: a named person per loop, a written reason, and a date to revisit it. The loops that hurt teams are rarely the ones somebody chose to automate. They are the ones that became unattended because nobody said otherwise.

Decide which loops may run unattended

The four questions of the autonomy ledger, from software factories, decide it per loop. A loop is a named workflow, such as “dependency upgrades in the payments service”, not a whole repository.

1. Name the oracle that decides “done”. “npm test and the contract suite exit 0” is an oracle. “It looks right” is not.

2. Check whether the agent can edit that oracle. If the agent can change the tests that judge its work, the oracle is a suggestion. Move the tests under code-owner review or make the decisive check predate the change.

3. Measure how long a wrong answer takes to surface. Seconds in CI is a green light. A customer report three weeks later is a red one.

4. State the blast radius. Reversible and contained, such as a copy change behind a flag, or irreversible, such as a schema migration.

5. Record the answers in the autonomy register with an owner, a reason and a review date (template below). A loop that answers cleanly on all four may run unattended. A loop that fails question 2 or 3 stays supervised, however good the harness is.

Five classes of work fail those questions often enough to name up front:

• Schema, auth, pricing and migrations. The oracle is production behaviour, feedback takes days, and the blast radius is customers. A human reads the diff every time.
• Work with no oracle that runs in CI. Architectural refactors are the standard case: nothing you can run says the new shape is better. Put the effort into the plan, not the automation.
• Work where the agent owns both sides. A feature in a well-covered module looks safe until you notice the agent wrote the feature and the tests that pass it. Either a human reads the plan, or the decisive check predates the change.
• Loops with slow feedback. If a wrong answer surfaces when a person notices rather than when a command exits non-zero, the loop accumulates errors quietly.
• Anything that reads text an outsider can write. Issue bodies, pull request comments, webhook payloads, scraped pages and inbound email are attacker-controlled input. An unattended agent with tools and a prompt-injection surface is a remote-execution path. If the loop must run, treat the input as data, and give the loop the narrowest permission set in the system; the agent threat model covers the rest.

Record the decision in the autonomy register

The autonomy register holds one row per loop; the operating model defines who owns it and governance and autonomy defines the risk classes. Paste this row format into your handbook and fill one row per unattended loop:

AUTONOMY REGISTER ROW — one per loop

Loop:            dependency upgrades, services/payments
Ladder level:    4 (agent opens PR; human reads the evidence, not the diff)
Risk class:      medium
Oracle:          unit + contract suites in CI; lockfile audit
Oracle owner:    @payments-leads (CODEOWNERS on tests/ and contracts/)
Agent can edit
the oracle:      no — tests/ and contracts/ are denied to the agent identity
Feedback time:   under 15 minutes (CI)
Blast radius:    one service; revert is a single PR
Permission set:  workspace-write sandbox, registry network only, no prod secrets
Outside input:   none (reads the lockfile and advisories only)
Accountable:     @payments-tech-lead (one named person, not a team)
Reason:          120 upgrade PRs a quarter; CI catches the failures we have seen
Review by:       2026-12-31

The review date matters most. A row whose date has passed is a loop running on an expired decision, so treat it the way you treat an expired certificate.

Turn the permission decision into configuration

The register says what a loop may do; the tool configuration is where that decision becomes true. These are the unattended CI shapes from permissions and sandboxing, checked against Claude Code 2.1.286 and Codex CLI 0.157.1 on 2026-09-26.

Claude Code

dontAsk denies anything that would prompt, so the allowlist is the whole policy, and the Edit(/tests/**) deny keeps the oracle out of reach:

claude -p "Run the test suite and fix failing tests in src/. Do not edit tests." \
  --permission-mode dontAsk \
  --permission-prompts none \
  --settings '{"sandbox":{"enabled":true,"allowUnsandboxedCommands":false}}' \
  --allowedTools "Read" "Edit" "Bash(npm test)" "Bash(npm run lint)" \
  --disallowedTools "Edit(/tests/**)"

For interactive sessions that run in auto mode, claude auto-mode config prints the effective auto mode configuration as JSON, so you can diff the policy you approved against what is running.

Codex

codex exec has no -a of its own in 0.157.1, so the approval flag goes before the subcommand. The :workspace permission profile (beta) grants workspace writes and no network:

codex -a never exec -c 'default_permissions=":workspace"' \
  "Run the unit tests and fix failing tests in src/. Do not edit tests."

Do not combine a permission profile with --sandbox: OpenAI says the two systems “do not compose”.

Cursor

Cursor documents run modes for the same decision; set them in the editor per your register row. For scripted runs, the @cursor/sdk options local.autoReview and local.sandboxOptions make the posture explicit in code. The full example is in permissions and sandboxing.

How do you prove the human’s decisions still hold?

Nobody on a Level 4 team reads every diff, so the decisions above need evidence of their own. Run these four checks, and have the register owner sign off on them each quarter:

Check	What goes red	Where it runs
Oracle protection	An agent-authored pull request changes files under tests/, evals/ or the CI config without a code-owner approval	Branch protection with Require review from Code Owners
Permission drift	A settings file, sandbox profile or allowlist changes without a matching register update	Code-owner review on .claude/, .codex/ and .cursor/
Unreviewed merges	The share of merged pull requests with no review rises in a risk class that requires review	A weekly query against your Git host, per risk class
Expired decisions	A register row’s review date has passed	A scheduled job that reads the register

Unreviewed merges are the leading signal. Faros AI’s AI Engineering Report 2026 (April 2026, telemetry from 22,000 developers) found 31.3% more pull requests merging with no review at all. Nobody decided that; it happened one busy afternoon at a time. What you read instead of the diff is covered in reading evidence instead of code.

Copy-paste prompts for keeping the human’s job

Copy-paste prompt: write the architecture brief the agents must obey

Read this repository and draft the architecture brief my agents must not
violate. Output four sections and nothing else.
Invariants: the boundaries, layering rules and dependency directions this
codebase actually enforces today. Cite a file for each one.
Earned exceptions: the places where the code breaks its own rule on purpose,
and the reason, from git history or comments.
Off limits: the modules where a change requires a human decision, with globs.
Open questions: the three architectural decisions that are currently implicit
and would be expensive to get wrong, phrased as questions for me to answer.
Do not propose refactors. Do not edit files.

Copy-paste prompt: sort the backlog into loops that may run unattended

Take the open issues in this backlog and sort them into three buckets using the
autonomy ledger: runs unattended, human reads the plan, human reads the diff.
For each issue, answer the four questions in one line each: what oracle decides
done; can the agent edit that oracle; how long until a wrong answer surfaces;
what the blast radius is. Then give the bucket and the single check that would
move it one bucket safer.
Flag separately any issue whose input includes text an outsider can write
(issue bodies from external reporters, webhook payloads, scraped content).
Output a table sorted by bucket. Do not start work on any of them.

Copy-paste prompt: design the permission set an unattended run gets

Design the permission policy for unattended runs in this repository, for each of
the three tools I use (Claude Code, Codex, Cursor). For each tool state: the
mode an unattended run must use; the commands allowed without asking; the
commands that must always prompt or be blocked outright; the network domains
reachable; the paths writable and the paths read-only; and the secrets that must
not be present in the environment at all.
Derive every entry from what the repository actually needs. Read the package
scripts, CI config and any existing settings files first, and list, at the end,
every permission you granted that you could not justify from a file.
Do not edit any settings file; output the policy as a table.

Where does the human’s job quietly disappear, and how do you get it back?

The reviewer becomes an approver. Approval stops being a decision and becomes a click. To recover, write down which risk classes may merge on evidence alone, require review for the rest in branch protection, and track the unreviewed-merge share per class.

The oracle becomes the spec. When tests are the only statement of what “correct” means, product decisions migrate into fixtures where nobody reviews them. To recover, keep acceptance criteria in the spec, map each one to the check that proves it, and review changes to that mapping like code; see writing specs at Level 4.

Accountability drifts to the tool. “The agent did it” is not an answer in an incident review. To recover, add a named accountable person to every register row before the loop runs again, as Linear does with its human assignee.

Permissions drift through convenience. Someone widens an allowlist to unblock one run, and every later run inherits it. To recover, put settings files under code-owner review and treat every change as a register update with a reason.

Understanding goes last. A line Karpathy quotes approvingly in his published transcript is the warning: “You can outsource your thinking, but you can’t outsource your understanding.” A team that can no longer explain its own system has lost something no throughput number shows. To recover, have a human explain each architectural change in the pull request description, and rotate who does it; craft and career covers which skills to keep in practice.

Level 5 gets mistaken for the destination. Shapiro’s dark factory, where “humans are neither needed nor welcome”, describes “a handful of people” in teams of fewer than five. The six responsibilities above describe Level 4, which is where the published evidence sits.

Where to go next with the human’s job

Before this page, read one map of the ladder for where each loop sits. Next, read reading evidence instead of code for what replaces the diff.

Reading evidence instead of code What a human reads when nobody reads every diff.

Human judgment at the gates The four gates where a person decides, run on one task.

Governance and autonomy Risk classes, guardrails and gates, and an autonomy policy template.

Permissions and sandboxing The per-tool configuration behind every permission decision.

Level 5: You Run the Software Factory Six stations, and which of them may run unattended.

The State of Agentic Engineering The sourced evidence behind the numbers on this page.