The Human's Job in Agentic Engineering
The human’s job in agentic engineering is the set of decisions no verification oracle can make for an agent: taste and architecture, product direction, the design of the evals and stop conditions the agents run against, the permissions they run with, and the call on which loops must not run unattended. A line Karpathy quotes approvingly draws the boundary: thinking can be outsourced, understanding cannot.
The uncomfortable version of the question is not “will there be work” but “which work”. Andrej Karpathy’s answer, in his Sequoia Ascent summary of 30 April 2026, is that the unit changed rather than the responsibility: “The unit of programming changed from typing lines of code to delegating larger ‘macro actions’.” BCG Platinion, writing as consultants in March 2026, put the same thought in a sentence a board understands: “The defining shift is not the absence of humans; it is the relocation of human effort.”
Six responsibilities that do not transfer
Section titled “Six responsibilities that do not transfer”They are not evenly distributed across the ladder. At Level 2 all six hide inside the act of writing the code, which is why they feel automatic and unnamed. From Level 3 upward each one has to be made explicit — written into a file, a gate or a policy — because an agent will not infer it and a passing test suite will not report its absence. What follows is that list, with the source for each.
Karpathy is direct about the agents’ present standing: “Right now the agents are like interns. You still have to be in charge of aesthetics, judgment, taste, and oversight.” Elsewhere on the same page: “you are in charge of taste, engineering, design, and whether the system makes sense.”
Taste is not decoration. It is the judgement that a working implementation is the wrong one — the abstraction that will cost a year, the dependency that solves today’s problem and owns tomorrow’s, the API that is correct and unusable. No test suite fails on any of that, which is exactly why it stays with a person.
Architecture
Section titled “Architecture”“You still have to understand the fundamentals,” Karpathy writes, “you cannot be a good director if you do not understand.” The architectural decision is also the one with the worst feedback loop in the business: wrong answers surface in months, long after the run that made them is forgotten.
There is third-party evidence that this responsibility is load-bearing rather than sentimental. DORA’s 2025 report found that “Teams working in loosely coupled architectures with fast feedback loops see gains, while those constrained by tightly coupled systems and slow processes see little or no benefit.” The architecture decides how much of the acceleration a team gets to keep.
Product direction
Section titled “Product direction”Karpathy names the bottleneck without softening it: “I am becoming the bottleneck of even knowing what we are trying to build, why it is worth doing, and how to direct my agents.” Shapiro’s Level 4 human (The Five Levels, January 2026) writes a spec and works closer to an engineering or product manager than to a coder.
An agent will build what the spec says with more patience than any team. It will not tell you the feature should not exist.
Eval and stop-condition design
Section titled “Eval and stop-condition design”This is the responsibility that grows as the others shrink. At Level 4 the stop condition is the product of your judgement: what “done” means, expressed as commands that exit 0. Shapiro’s version of the level is “leave for 12 hours, and check to see if the tests pass” — which is only safe if somebody designed the tests.
The argument for determinism is put well by Naresh (DEV Community), writing about graph engineering in July 2025: “A model can argue that its work is complete. A deterministic validator can prove that a required field is missing.” Stripe’s factory is the same idea at scale: the Minions run against “over three million” pre-existing tests, bounded to “at most two rounds of CI”, after which “we send the branch back to its human operator for manual scrutiny.”
The oracle is the part of the system that an agent must never be able to author. Choosing it is a human job, and keeping it out of the agent’s reach is a design constraint, not a preference.
Permissions
Section titled “Permissions”What an agent may run unsupervised is a decision, and every tool asks you to make it. Codex documents an approval policy for “when Codex must ask you before it executes an action (for example, leaving the sandbox, using the network, or running commands outside a trusted set)”, with named modes. Claude Code’s auto mode routes “tool calls through a classifier that blocks anything irreversible, destructive, or aimed outside your environment”, and its sandbox asks you to “define which files and network domains commands can touch”. Cursor exposes run modes for the same choice.
Defaults are not a policy. The permission set an unattended run inherits at 3am is whichever one the last session left behind, and that is a decision somebody made by not making it.
The call not to automate
Section titled “The call not to automate”Linear’s engineering write-up on delegating to agents, August 2025, states the principle more plainly than any vendor doc: “an agent cannot be held accountable.” Its product enforces it — “issues can only be assigned to humans, and only delegated to agents”, so “the issue still has a human assignee—someone accountable for the result—but it also has a delegated agent responsible for taking action.”
Accountability does not delegate, which means somebody has to decide what runs unattended and be answerable for the answer. In practice that decision has a shape: a named person per loop, a written reason, and a date to revisit it. The loops that hurt teams are rarely the ones somebody chose to automate — they are the ones that became unattended because nobody said otherwise.
When a loop must not run unattended
Section titled “When a loop must not run unattended”The four questions from software factories decide it, per loop, in writing:
- What oracle decides “done”? Name the specific thing. “Tests pass” is an oracle. “It looks right” is not.
- Can the agent fake it? If the agent can edit the oracle, the oracle is a suggestion.
- How long until a wrong answer surfaces? Seconds is a green light. Three weeks is a red one.
- What is the blast radius if it is wrong? Reversible and contained, or a schema migration.
Answer them in writing, per loop, and keep the answers where the next person will find them. A loop that answers cleanly on all four can run dark today; one that fails question 2 or 3 stays lit no matter how good the harness is. Five classes of work fail those questions often enough to be worth naming up front.
Schema, auth, pricing and migrations. The oracle is production behaviour, feedback takes days to weeks, and the blast radius is customers. These stay lit: a human reads the diff, every time.
Anything with no oracle that runs in CI. Architectural refactors are the standard example. There is nothing to run that says the new shape is better, and the answer arrives in months. Front-load the planning instead of the automation.
Work where the agent owns both sides. Feature work in a well-covered module looks like a green-light loop until you notice the agent wrote the feature and the tests that pass it. Either a human reads the plan, or the decisive check predates the change.
Loops whose feedback is slow. Anything where a wrong answer surfaces after a human notices, rather than when a command exits non-zero, is a loop that will accumulate wrongness quietly.
Anything that reads text an outsider can write. Issue bodies, pull request comments, webhook payloads, scraped pages and inbound email are attacker-controlled inputs. An unattended agent with tools and a prompt-injection surface is a remote-execution path with a friendly name. If such a loop has to run, wrap the input as data, strip its authority to issue instructions, and give it the narrowest permission set in the system.
Copy-paste prompts for keeping the human’s job
Section titled “Copy-paste prompts for keeping the human’s job”Where the human’s job quietly disappears
Section titled “Where the human’s job quietly disappears”The reviewer becomes an approver. Faros AI measured 31.3% more pull requests merging with no review at all across 22,000 developers in April 2026. Nobody decided that; it happened one busy afternoon at a time. The honest version is a written rule about which classes may merge unread.
The oracle becomes the spec. When the tests are the only statement of what “correct” means, product decisions migrate into fixtures, where no one reviews them and no one can find them later.
Accountability drifts to the tool. “The agent did it” is not an answer to an incident review. Linear’s rule — the human assignee stays — is the cheap version of a fix that is expensive to retrofit after the first outage.
Understanding goes last, and quietly. The line Karpathy quotes approvingly is the warning: you can outsource your thinking, but you cannot outsource your understanding. A team that stops being able to explain its own system has lost something that no throughput number will show.
Level 5 gets mistaken for the destination. Shapiro’s dark factory, where “humans are neither needed nor welcome”, describes a handful of teams of under five people. The six responsibilities above describe Level 4, which is where the published evidence actually sits, and where the interesting work is.