DevOps with AI

DevOps with AI means letting Claude Code, Codex, or Cursor read live production state through read-only MCP servers, then draft the diagnosis, the fix, or the infrastructure change. A human or a deterministic gate still applies it. This section covers incidents, observability, performance, security, compliance, recovery, cost, and the health of the agents themselves.

It is written for developers on call and for tech leads who own a service. Your staging deploy is green, production returns 500s under real traffic, the runbook lives in three people’s heads, and the one engineer who knows the Helm values is on leave. The agent can read the manifests and query the metrics faster than you can. The question is how you let it help without letting it touch production unchecked.

Which operations guide fits your problem?

Your situation	Start with
An alert is firing and you need the root cause and a postmortem	Incident response
Problems reach you through users, not dashboards	Monitoring and observability
Latency, memory, or autoscaling is wrong	Performance tuning
A scanner blocks the deploy or a CVE needs triage	Security operations
An auditor wants evidence for SOC 2, HIPAA, GDPR, or PCI	Compliance automation
Nobody has restored a backup in a year	Disaster recovery
The cloud bill grew and nobody knows why	Cloud cost management
You need to see what your agents did, what it cost, and who approved it	Agent observability
Agent-written code is drifting, duplicating, or eroding the architecture	Codebase health
You want to classify what went wrong in an agent change and which control should have caught it	Failure taxonomy
A metric drifts and you need a deterministic anomaly check before the agent diagnoses	Deterministic anomaly detection
You want security scans on a schedule with evidence and PR gates	Scheduled security scans
You want an agent in the incident channel that gathers evidence under a read-only identity	Scoped AI incident triage

If you work in one tool, its operate section covers the same ground with that tool’s commands: Claude Code, Codex, or Cursor.

Operations start where delivery ends. If the problem is in the pipeline itself, go to CI/CD pipelines, containerization, or infrastructure as code.

Connect read-only MCP servers before anything that can write

An agent that sees your real signals gives specific answers; an agent that sees your pasted summary guesses. Model Context Protocol (MCP) servers give the agent that access. The server definitions are the same for all three tools; only the place you register them differs. Start with observability and documentation servers, and turn off write tools where the server allows it.

Claude Code

# Terminal. Sentry (OAuth on first use via /mcp in a session)
claude mcp add --transport http sentry https://mcp.sentry.dev/mcp
# Grafana, with write tools disabled
claude mcp add grafana -e GRAFANA_URL=https://grafana.example.com \
  -e GRAFANA_SERVICE_ACCOUNT_TOKEN=YOUR_TOKEN -- uvx mcp-grafana --disable-write

Codex

# Terminal. Writes [mcp_servers.<name>] into ~/.codex/config.toml
codex mcp add sentry --url https://mcp.sentry.dev/mcp && codex mcp login sentry
codex mcp add grafana --env GRAFANA_URL=https://grafana.example.com \
  --env GRAFANA_SERVICE_ACCOUNT_TOKEN=YOUR_TOKEN -- uvx mcp-grafana --disable-write

Cursor

In .cursor/mcp.json (project) or ~/.cursor/mcp.json (global):

{ "mcpServers": {
  "sentry": { "url": "https://mcp.sentry.dev/mcp" },
  "grafana": { "command": "uvx", "args": ["mcp-grafana", "--disable-write"],
    "env": { "GRAFANA_URL": "https://grafana.example.com",
             "GRAFANA_SERVICE_ACCOUNT_TOKEN": "YOUR_TOKEN" } } } }

Replace YOUR_TOKEN with a Grafana service-account token that has the Viewer role. Datadog runs a managed server (https://mcp.datadoghq.com/v1/mcp for US1; change the host for other sites), AWS now recommends the Agent Toolkit for AWS over the older awslabs/mcp servers, and HashiCorp’s Terraform MCP server keeps the operations that need approval off unless you set ENABLE_TF_OPERATIONS=true. The install commands and traps for each are in observability MCP servers and infrastructure MCP servers.

Prompts for your first on-call shift with an agent

Copy-paste prompt: read-only triage of a live alert

Checkout is returning 5xx since roughly 14:10 UTC. Investigate read-only: do not
change code, config, or infrastructure. Use the Grafana and Sentry MCP tools to
(1) chart 5xx rate and p95 latency for service checkout over the last 2 hours,
(2) list Sentry issues first seen or spiking after 13:45 UTC with their top frames,
(3) list deploys and config changes since 13:00 UTC from git log and the CI history.
Give me the three most likely causes ranked, the evidence for each as query or
issue links, and the single cheapest check that would rule each one out.

Copy-paste prompt: a rollback-ready fix proposal

Cause confirmed: the 13:52 deploy changed the connection pool size in
config/database.yml from 20 to 5. Prepare a fix on a new branch but do not
merge or deploy. Include: the one-line config change, a test or load check that
fails before and passes after, the exact rollback command, and the metric and
threshold that should show recovery within 10 minutes of the deploy. Put all of
it in the PR description as a checklist.

How do you prove an agent’s operational change is safe?

You do not read every line of generated HCL or YAML. You read the evidence that a machine produced:

• A diff of reality, not of text. terraform plan, kubectl diff, or a dry-run shows what will change in the live system. A plan that destroys or replaces a resource blocks the merge.
• Policy and scanner gates in CI. Policy-as-code checks, image and IaC scanners, and cost checks run on every PR, so a hardening mistake fails the build before anyone reviews it.
• A recovery signal and a rollback path. Every change names the metric that proves it worked and the command that undoes it; progressive delivery and a rehearsed rollback make both routine.
• A named approver for production. The agent proposes, a human applies. The evidence bundle defines what the PR must carry for that sign-off.

What breaks when agents work on production systems?

Failure modes and how to recover

• The agent holds write credentials it never needed. A “quick fix” becomes an unreviewed change to live state. Recovery: revoke and re-issue the token with read-only scope, then check the provider’s audit log for the session’s actions.
• Hardening advice from an older Kubernetes era. PodSecurityPolicy was removed in Kubernetes 1.25, so a manifest that uses it does not apply. Recovery: ask for Pod Security Admission levels plus NetworkPolicy, and validate against the cluster version.
• Stale MCP pins. The terraform plugin in claude-plugins-official pinned image 0.4.0 while HashiCorp’s README showed 1.3.0 tags (checked 2026-09-26). Recovery: register the server yourself with a current image tag.
• Tool output floods the context. A broad log query fills the window and the agent’s reasoning degrades. Recovery: narrow the time range and service in the prompt, and see reducing MCP token cost.
• A confident root cause without evidence. Recovery: require query links for every claim, as the triage prompt does, and treat an unlinked claim as a hypothesis.

Where to go next with AI in operations

If nothing is burning, start with monitoring and observability: every other workflow here gets faster once the agent sees real signals. Then set up agent observability so you can audit what the agents themselves do, and read MCP security before you grant any server write access.