# Open-weight and self-hosted models for coding agents

URL: https://developertoolkit.ai/en/ecosystem/models/open-weight/
Description: GLM, Qwen, Kimi, DeepSeek and Mistral models for air-gapped work: licences, which harnesses accept them, how to serve them, and how to measure the gap on your own evals.

Open-weight coding models (GLM-5.3 and GLM-5.3-Flash, Qwen3.8, Kimi K3, DeepSeek V4.x, and Mistral's Devstral 2 and Medium 3.5) can drive Claude Code, Codex and several vendor CLIs from a server you control. They fit air-gapped networks, strict data rules or a cost floor, and a team should route work to them only after its own evals show the quality gap is acceptable.

Your security team rules that one product line's code may not leave the building. Procurement wants a second model supplier before a seven-figure renewal. Finance asks why a nightly job that labels 40,000 test failures runs on your most expensive model. Someone proposes "just run Qwen on our own GPUs", and nobody can say how much worse the agent gets, which tools still work, or what the licence allows.

This page is for the **CTO** deciding whether to fund a self-hosted model and for the **developer** asked to wire one into the team's agent and prove it works. It gives you an October 2026 shortlist with licence status, the harness matrix, verified vLLM and Ollama configuration, an eval protocol with a runner and decision rule, a cost formula, a data-rules checklist, and the traps that silently send code off the machine. Model versions, prices and context windows live on [the models hub](/en/ecosystem/models/).

## Which open-weight coding models matter in October 2026?

Five families cover almost every serious evaluation. The sizes and vendor agents were verified on 2026-09-26 against each vendor's GitHub repository or package registry, except where a cell says otherwise; the weights licences were read on each model's Hugging Face repository on 2026-10-12, and "check the model card" marks a licence that was not read.

| Model | Vendor | Size (total / active) | Weights licence | Vendor's own agent | Evidence for coding |
| ----- | ------ | --------------------- | --------------- | ------------------ | ------------------- |
| GLM-5.3 and GLM-5.3-Flash | Z.ai | 744B / 40B (Flash: 320B / 18B) | GLM-5.3: custom "GLM-5.3 License", an MIT-style grant including weights, but a MaaS operator with group revenue above 10 billion US dollars over any consecutive 12 months must pass Z.ai's security review before commercial use. GLM-5.3-Flash and GLM-5.2: MIT. The repo code is Apache-2.0 (read 2026-09-26) | ZCode (Apache-2.0, repository created 2026-09-20); also runs in Claude Code | The only two open-weights entries on the official Terminal-Bench 4.0 board |
| Qwen3.8 (2.4T-A95B and 27B) | Alibaba | 2.4T / 95B, and a 27B model | Qwen3.8-27B: Apache-2.0. Qwen3.8-2.4T-A95B: "Qwen3.8-Max License" (the model name must be displayed in products above 100M monthly active users or $20M monthly revenue; a separate licence is needed by MaaS or "AI Work Assistant" businesses above $50M revenue over 12 months). Qwen3.8-Flash-Next: "Qwen Community License 1.0" | Qwen Code | Vendor: "a Qwen-Max-class model to open release"; scores not verified |
| Kimi K3 (launched July 2026) | Moonshot AI | 2.8T / 104B | Custom "Kimi K3 License", not OSI open source: MaaS operators above $20M revenue over 12 months need a separate agreement with Moonshot AI, and "Kimi K3" must be displayed in products above 100M monthly active users or $20M monthly revenue; internal use is exempt | Kimi Code | Vendor-reported only (Terminal-Bench 2.1) |
| DeepSeek V4 (V4.1 Flash, V4-Pro) | DeepSeek | V4.1 Flash: 552B backbone, 8B active in prefill and 16B in decode; Pro: 1.6T / 49B (secondary: SitePoint) | MIT (V4.1 Flash, V4-Pro-0813, V4-Flash-0731) | DeepSeek Harness `dsh` (developer preview, npm `latest` 0.2.0-rc.2) | V4.1 Flash (2026-09-10): DeepSeek reports 31.2 on Terminal-Bench 4.0 in the minimal mode of its own harness; not on the official board |
| Devstral 2 · Devstral Small 2 · Mistral Medium 3.5 | Mistral | Devstral 123B and 24B (secondary: VentureBeat) | Medium 3.5: Modified MIT, with no rights for a company whose global consolidated monthly revenue exceeds $20 million; Devstral: check the model card | Mistral Vibe | Devstral 2 self-reports 72.2% on SWE-bench Verified (secondary: VentureBeat) |

- **GLM-5.3 and GLM-5.3-Flash are the two open-weights models with independent evidence.** On the official Terminal-Bench 4.0 leaderboard (read 2026-10-12), GLM-5.3 at `max` effort in Claude Code scores 41.8% ± 3.2 and GLM-5.3-Flash (effort `none`) 35.8% ± 3.5. Grok 4.7 in Grok Build scores 37.6% ± 3.5 and GPT-5.6 Sol in Codex 37.3% ± 3.8; the 95% intervals overlap GLM-5.3's, so the board does not separate them. For scale, Claude Opus 5.5 at `max` in Claude Code scores 64.8% ± 3.1 and Claude Fable 5.1 57.9% ± 3.8.
- **Qwen3.8-Max-0902 is a hosted model.** Alibaba's Model Studio notice describes it as the 2026-09-02 snapshot of the `qwen3.8-max` endpoint and does not call it open weights, so its 27.0% ± 3.8 on the official board (in Claude Code) is a hosted-model result, not evidence for the downloadable Qwen3.8 weights.
- **Mistral Large 4 has no weights yet.** Mistral's changelog (2026-10-06) calls it "an open-weight, general-purpose multimodal model" in public preview, with 1.05T total and 52B active parameters and a 1M context window, and adds: "Open weights are coming soon." No licence is published (checked 2026-10-12), so it is a hosted model until the weights and licence appear.
- **Two more licences worth reading before you shortlist them.** MiniMax M3 (about 428B total and 23B active, 1M context) uses the "MiniMax Community License": the grant is for non-commercial purposes, commercial use requires "Built with MiniMax M3" and a notice to MiniMax, and written authorization is needed above $20M yearly revenue. The weights of Xiaomi's MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL are MIT.
- **Vendor numbers do not cross benchmark versions.** Kimi K3 reports 88.3 on Terminal-Bench 2.1, an older, near-saturated version with no conversion to 4.0.
- **The SWE-bench Verified board is frozen** (newest entry 2026-02-26, checked 2026-10-12), so any 2026 figure for these models is a self-report.

None of this predicts results on your repository; the eval section below does. See also [reading coding-agent benchmarks](/en/ecosystem/models/benchmarks/).

## When is an open-weight model the right call?

An open-weight model answers a constraint, not a quality need. The model rule from [the models hub](/en/ecosystem/models/) still holds: start on the tool's default model (Claude Opus 5.5 in Claude Code; GPT-6.1 Sol in Codex), tune effort before switching model, and switch only when your own evals say so.

| Your situation | Open-weight model? | Why |
| -------------- | ------------------ | --- |
| Code may not leave your network (defence, some banks, some public sector) | **Yes, self-hosted** | No third party processes the code. First check whether a cloud route under your contract satisfies the rule ([where the model runs](/en/org/model-hosting/)) |
| Data rules require a named region or zero retention, but a cloud provider is allowed | Usually no | A hosted frontier model through your cloud contract is simpler |
| High-volume, narrow jobs (labelling failures, generating fixtures, first-pass triage) | **Maybe** | Pays off at volume only if the eval pass rate on that job is close to the hosted model's |
| Hard, open-ended feature work on a large codebase | No, as the default | The board's open-weights entries (GLM-5.3 at 41.8%, GLM-5.3-Flash at 35.8%) sit well below the best Claude entry (Opus 5.5 at 64.8%) |
| A second supplier to avoid lock-in | **Yes, as a tested fallback** | Keep one open-weight setup passing your eval set ([avoiding lock-in](/en/org/lock-in-and-portability/)) |
| A laptop without network on a plane | Maybe, for small tasks | A 24B–27B model runs locally, with a clear quality drop on multi-file work |

## Which harnesses accept a self-hosted model?

The harness matters as much as the model: the board entry is GLM-5.3 **in Claude Code**. Each harness speaks one API format, and the serving stack must match it.

| Harness | How it reaches your model | API it needs | What you lose |
| ------- | ------------------------- | ------------ | ------------- |
| Claude Code 2.1.283 | `ANTHROPIC_BASE_URL` plus a token variable; `--model` or `ANTHROPIC_MODEL` | Anthropic Messages (`/v1/messages`) | Remote Control (off when the base URL is not `api.anthropic.com`); `/voice` (needs a claude.ai login); MCP tool search, off by default (set `ENABLE_TOOL_SEARCH=true` if your server forwards `tool_reference` blocks) |
| Codex 0.157.1–0.162.1 | `--oss` with `--local-provider ollama` or `lmstudio`, or a `[model_providers.<id>]` block in `config.toml` | Responses API only (`wire_api = "responses"`); `"chat"` is rejected | Features tied to a ChatGPT or OpenAI login, such as `codex cloud` tasks |
| Cursor | Not verified for a model served inside your network (checked 2026-10-12) | — | See the Cursor tab below |
| Qwen Code, Kimi Code, Mistral Vibe, DeepSeek `dsh` | Each vendor's CLI, built for its own models | Vendor-specific | Portability of your prompts, hooks and skills |
| OpenCode, Cline, Aider, Goose, Crush | Provider settings in each tool | Mostly OpenAI-compatible | Your team's Claude Code or Codex configuration |

Serving stacks that implement those APIs, verified on 2026-09-26:

- **vLLM** (PyPI `vllm` 0.30.0): OpenAI-compatible "plus Anthropic Messages API", with tool calling on both. The production choice for a shared GPU server.
- **Ollama**: "a subset of the Anthropic Messages API"; `ollama launch claude` and `ollama launch codex` wire the agents for you.
- **LM Studio**: `POST /v1/messages` locally; Codex reaches it through `--local-provider lmstudio`.
- **llama.cpp** `llama-server`: Anthropic Messages compatible; tool use requires `--jinja`.
- **LiteLLM** or **Claude Code Router**: a gateway with virtual keys and budgets in front of several backends ([gateways and local models](/en/ecosystem/agent-tools/gateways-local-models/)).

**Popularity as of 2026-09-26:** Ollama has 181,740 GitHub stars (GitHub, ollama/ollama, read 2026-09-26); stars measure attention, not fitness for a regulated network.

## How do you connect Claude Code or Codex to a self-hosted model?

Start with a single-machine Ollama trial to learn the failure modes, then move to vLLM on a shared server.

**Claude Code**

**Trial on one machine with Ollama.** Ollama's default context is set by VRAM (4k under 24 GiB), which silently truncates an agent's system prompt, so raise it first:

```bash
# Terminal 1: start Ollama with an agent-sized context
OLLAMA_CONTEXT_LENGTH=64000 ollama serve

# Terminal 2: let Ollama launch Claude Code wired to a local model (interactive session)
ollama launch claude --model LOCAL_MODEL

# Terminal 3, while the session runs: confirm the loaded context
ollama ps          # CONTEXT should read 64000; PROCESSOR should read 100% GPU
```

**Team server with vLLM.** Map every model alias to the served model: Claude Code also calls the `haiku` alias for background work, and without the mapping those calls fail with "model not found":

```bash
# Developer machine or CI runner. VLLM_API_KEY comes from your secret store, never a literal.
export ANTHROPIC_BASE_URL=http://gpu-01.internal:8000
export ANTHROPIC_AUTH_TOKEN="$VLLM_API_KEY"   # sent as "Authorization: Bearer ..."
export ANTHROPIC_API_KEY=""                   # stop a saved key from overriding the gateway
export ANTHROPIC_MODEL=SERVED_MODEL
export ANTHROPIC_DEFAULT_OPUS_MODEL=SERVED_MODEL
export ANTHROPIC_DEFAULT_SONNET_MODEL=SERVED_MODEL
export ANTHROPIC_DEFAULT_HAIKU_MODEL=SERVED_MODEL
export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1   # no auto-updates, telemetry or error reporting
claude
```

Run `/status` to confirm the base URL, and `/context` to see how much of the window the system prompt and tools take before you type anything (the context cost). For a team, put the variables in the `env` block of a managed settings file ([LLM gateway](/en/claude-code/team/llm-gateway/)).

**Codex**

**Trial on one machine.** Codex has a built-in switch for Ollama and LM Studio:

```bash
OLLAMA_CONTEXT_LENGTH=64000 ollama serve                   # terminal 1
codex --oss --local-provider ollama -m LOCAL_MODEL         # terminal 2
codex --oss --local-provider lmstudio -m LOCAL_MODEL       # or LM Studio after `lms server start`
```

**Team server with vLLM.** Put the provider in a profile file, `$CODEX_HOME/selfhosted.config.toml` (`~/.codex/selfhosted.config.toml` by default). `-p selfhosted` layers it on top of the base config (`codex --help`, 0.157.1), so your hosted setup stays the default. The keys below are read from the codex-cli 0.157.1 source:

```toml
model = "SERVED_MODEL"
model_provider = "vllm"

[model_providers.vllm]
name = "vLLM (internal)"
base_url = "http://gpu-01.internal:8000/v1"
env_key = "VLLM_API_KEY"      # the variable Codex reads the key from; never put the key here
wire_api = "responses"        # "chat" fails with "no longer supported" in 0.157.1
```

Then run a smoke test that forces a tool call:

```bash
codex -p selfhosted exec "List the three largest files under src/ and say which one has no tests."
```

**Cursor**

Whether Cursor's agent can use a model served inside your network is not verified here (checked 2026-10-12). Cursor's self-hosted machines (changelog, 2026-09-02) "keep tool execution entirely in your own network", which is tool execution and not model hosting. Do not plan an air-gapped rollout on Cursor until Cursor confirms the data path in writing. Until then, use Cursor for code that may leave the network, and Claude Code or Codex in its integrated terminal for the restricted repositories.

## How do you set up a vLLM server and prove the agent works?

One GPU server, one agent, one smoke task with a pass or fail answer. It assumes a Linux host with enough GPU memory (see the sizing rule below) and Python with `uv`.

1. **Check the licence before you download anything.** Record the licence name and use restrictions from the Hugging Face model card in your model register.

2. **Serve the model with tool calling on**, which a coding agent needs. Use the tool-call parser the model card names: vLLM lists parsers by family (for example `glm47`, `kimi_k3`, `deepseek_v4`, `qwen3_xml` / `qwen3_coder`, `mistral` in vLLM 0.30.0), and a newer model may need a newer vLLM release.

   ```bash
   # GPU server. VLLM_API_KEY comes from your secret store.
   uv venv && uv pip install vllm==0.30.0
   vllm serve MODEL_REPO_OR_PATH \
     --served-model-name SERVED_MODEL \
     --enable-auto-tool-choice --tool-call-parser PARSER \
     --max-model-len 131072 \
     --api-key "$VLLM_API_KEY" \
     --port 8000
   ```

   `MODEL_REPO_OR_PATH` is the Hugging Face repository or, air-gapped, the local directory holding the weights.

3. **Point the agent at it** with the Claude Code or Codex configuration above.

4. **Run a smoke task that needs tools, edits and tests.** In a throwaway branch of a real repository:

   ```bash
   claude -p "Add a failing unit test for parseDuration('90m') returning 5400 seconds, run it, then fix src/time.ts until it passes. Report the test command and its final output." \
     --permission-mode acceptEdits --allowedTools "Bash(npm *)" --output-format json
   ```

5. **Read the result, not the prose.** Run the test yourself (`npm test -- time`) and check `git diff --stat`.

**What you should see:** a JSON `result` that names the test command, a diff touching `src/time.ts` and one test file, and a test run you reproduce as green. Prose without edits means tool calling is broken; a truncated plan means the context is too small.

**Sizing rule.** Weight memory ≈ total parameters × bytes per parameter: 744B at 8 bits is about 744 GB, at 4 bits about 372 GB, before the KV cache. Mixture-of-experts helps speed, not memory. The 24B–27B class (Devstral Small 2, Qwen3.8-27B) fits one workstation.

## How do you measure the gap on your own evals?

Public boards rank model and harness pairs on someone else's tasks. Your question is whether the model is good enough for **your** work in **your** harness, so build a small eval set from your own history. The full method is on [evals for coding agents](/en/ecosystem/observability/evals-coding-agents/); this is the version for a model switch.

1. **Collect 20 to 30 tasks from merged pull requests.** Pick the kind of work you plan to route, each with tests the pull request added. Keep the parent commit, the issue text as the prompt, and the pull request's test files, at their repository paths, as the hidden checker.

2. **Freeze the setup.** Same harness version, `CLAUDE.md` or `AGENTS.md`, MCP servers, skills and effort level. Record them next to the results.

3. **Run each task in a disposable worktree**, on the baseline (the tool's default model) and on the candidate, three times per setup if you can afford it; one run on 25 tasks cannot separate models a few points apart.

4. **Score with the hidden tests**, not with the agent's own report. A task passes when the hidden tests pass and the type check and linter are clean.

5. **Compare four numbers**, then apply the decision rule below.

A minimal runner, for a repository where each task lives in `evals/tasks/<id>/` with `prompt.md`, `base_commit` and a `hidden-tests/` directory:

```bash
#!/usr/bin/env bash
# evals/run.sh SETUP_NAME  (run on an isolated runner with no production credentials)
set -euo pipefail
setup="$1"; mkdir -p "evals/results/$setup"
for dir in evals/tasks/*/; do
  id=$(basename "$dir")
  wt="$(mktemp -d)/$id"
  git worktree add --detach "$wt" "$(cat "$dir/base_commit")" >/dev/null
  ( cd "$wt"
    # A fresh worktree has no node_modules. Offline, point npm at your internal registry mirror.
    npm ci --silent || { echo "$id SETUP-FAIL"; exit 0; }
    claude -p "$(cat "$OLDPWD/$dir/prompt.md")" --permission-mode acceptEdits \
      --allowedTools "Bash(npm *)" --output-format json > "$OLDPWD/evals/results/$setup/$id.json" || true
    cp -r "$OLDPWD/$dir/hidden-tests/." .   # hidden tests are stored at their repository paths
    if npm test --silent && npx tsc --noEmit && npm run lint --silent; then echo "$id PASS"; else echo "$id FAIL"; fi
  ) | tee -a "evals/results/$setup/summary.txt"
  git worktree remove --force "$wt"
done
```

Use your repository's own install, test, type-check and lint commands in place of the `npm` ones. Run it twice: once with the default environment, once after exporting the self-hosted variables from the Claude Code tab. For Codex, replace the two-line `claude` command with the line below; `--json` writes JSONL events, so the result file is `.jsonl`:

```bash
codex -a never exec -p selfhosted --sandbox workspace-write --json "$(cat "$OLDPWD/$dir/prompt.md")" > "$OLDPWD/evals/results/$setup/$id.jsonl" || true
```

| Metric | Definition | Why it matters |
| ------ | ---------- | -------------- |
| Pass rate | Tasks whose hidden tests, type check and lint pass ÷ tasks run | The quality gap, in your terms |
| Cost per solved task | (GPU-hours × hourly cost, or API spend) ÷ tasks passed | A model that fails half the time is not cheap |
| Intervention rate | Runs that asked a question, looped or hit the turn limit ÷ runs | A person has to step in |
| Tool-call error rate | Malformed or rejected tool calls ÷ tool calls (JSON transcripts) | First sign of a wrong parser |

**Decision rule.** Route a job type to the open-weight model when its pass rate is within an agreed margin of the baseline (for example, five points on 25 tasks × 3 runs), cost per solved task is lower, and the intervention rate is no higher. Otherwise keep the hosted model and re-run on the next release ([the new-model playbook](/en/ecosystem/models/new-model-playbook/)).

**Copy-paste prompt: build the eval set from your history**

```text
Build an evaluation set for comparing coding models on this repository. Do not change application code.
1. Use git log and gh pr list --state merged --limit 200 to find merged pull requests from the last 6 months
   that (a) fix a bug or add a small feature, (b) add or change tests, and (c) touch fewer than 8 files.
2. Pick 25, spread across at least 4 different directories. Skip anything that changes CI, migrations,
   lockfiles or needs secrets to test.
3. For each, create evals/tasks/<pr-number>/ with:
   - prompt.md: the linked issue text, or the PR description rewritten as a task with no hint of the solution
   - base_commit: the parent commit of the merge
   - hidden-tests/: the test files the PR added or changed, at their repository paths, as of the merge commit
4. Verify each task: check out base_commit, copy hidden-tests/. into the repository root, run the tests, and confirm they FAIL.
   Drop any task whose hidden tests already pass.
Output a table: task id, directory, one-line task, test command, and whether the fail-first check passed.
```

**Copy-paste prompt: compare two eval runs**

```text
Compare evals/results/baseline and evals/results/selfhosted. Do not edit files.
For each setup report: pass rate, the number of runs that ended without a final result,
and the number of tool calls that returned an error (read the JSON transcripts).
Then list every task that passed on baseline and failed on selfhosted, with the first point
where the selfhosted transcript went wrong (wrong file, malformed tool call, gave up, truncated context).
Group those failures by cause. End with one line: ROUTE (job types where selfhosted is within
5 points of baseline), HOLD (not yet), or INVESTIGATE (the failures look like setup errors, not model quality).
```

## What is the cost floor of a self-hosted model?

Self-hosting replaces a per-token price with a fixed cost that you pay whether the GPUs are busy or not. Put two numbers in front of finance instead of a per-token comparison:

- **Self-hosted cost per solved task** = (GPU amortisation or reserved rental + power + engineer time to run the server) ÷ (monthly tasks × eval pass rate).
- **Hosted cost per solved task** = API or plan spend for the same job ÷ tasks solved on the baseline. Break-even is the monthly volume at which the two are equal.

Below break-even, a hosted model ([the models hub](/en/ecosystem/models/) lists cheaper tiers) wins; above it, self-hosting wins only if the pass rate holds. Easy to miss: re-running the eval for every release, and the reviewer time a lower pass rate adds. See also [the economics of agentic engineering](/en/strategy/economics/).

**Copy-paste prompt: air-gap readiness audit**

```text
Audit this repository and my local agent configuration for an air-gapped run against a self-hosted model.
Do not change anything; report only.
1. List every network destination the agent setup would reach: MCP servers in .mcp.json and ~/.claude.json,
   hooks in .claude/settings.json, plugins, skills that call external URLs, and package installs in scripts.
2. List the environment variables that control the model endpoint and which ones are set now
   (ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_API_KEY, ANTHROPIC_MODEL, ANTHROPIC_DEFAULT_*_MODEL,
   CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC). Show names only, never values.
3. Flag any model name ending in ":cloud" and any endpoint that is not on the internal network.
Output a table: destination, where it is configured, needed offline (yes/no), and the fix.
```

## Which data rules apply to open-weight models?

"Open weights" answers where the model runs, not what the licence allows. Use this checklist before the model touches production code.

- **Licence named in the register.** Record the weights licence from the model card, not the GitHub repository's code licence (GLM's code is Apache-2.0; the weights of GLM-5.3 use the custom GLM-5.3 License, and GLM-5.3-Flash is MIT). Read the commercial strings too: revenue thresholds, name-display rules and MaaS clauses decide whether your use is allowed.
- **Self-hosted, not the vendor's API.** A vendor's hosted API or coding plan sends your code to that vendor, under its terms and jurisdiction. DeepSeek's API, for example, offers an Anthropic-format base URL (`https://api.deepseek.com/anthropic`) and the model name `deepseek-flash` (the legacy name `deepseek-v4-flash` is still accepted, but the model it named is retired); pointing Claude Code at it is hosted use, not self-hosting.
- **Local tags only.** Ollama tags ending in `:cloud` run on Ollama's cloud, not your machine.
- **Egress blocked, not trusted.** Block outbound traffic from agent runners at the network layer, and set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1`.
- **Hosted open-weight models count as hosted.** Kimi K3 in GitHub Copilot, or GLM-5.3 on Mistral's platform (`zai-glm-5-3`, generally available 2026-09-28), is not self-hosting.

Residency and zero retention for frontier models: [where the model runs](/en/org/model-hosting/).

## How do you keep quality provable on a weaker model?

Keep every gate you have for the hosted model and add the eval set as a gate. Output is verified the same way whichever model wrote it; only the catch rate changes.

| Gate | What it proves | Owner |
| ---- | -------------- | ----- |
| Eval set pass rate, re-run on every model or harness upgrade | The model still solves your kind of task | The platform team |
| Tests, type check and lint in CI on every pull request | This change works | CI, required status checks |
| A failing-test-first rule in the agent's instructions | The test would catch a regression | The developer who dispatched the task |
| A reviewer agent on the hosted baseline, where data rules allow | A second model checks every diff | The tech lead picks the paths |
| Revert and incident rate per model, monthly | Weaker output is not reaching production | The CTO, with the platform team |

Sign-off stays with people: the developer owns each merge; the platform owner routes job types to models on the eval numbers. When the pass rate drops after an upgrade, roll back first and investigate second. Guardrails for the self-hosted box: [permissions and sandboxing](/en/workflows/harness/permissions-and-sandboxing/).

## What breaks when an agent runs on an open-weight model?

**Traps with open-weight models and local serving**

- Ollama's default context is 4k under 24 GiB VRAM, which cuts off the agent's system prompt. Set `OLLAMA_CONTEXT_LENGTH=64000` and check `ollama ps`.
- Ollama model tags ending in `:cloud` run on Ollama's cloud: code leaves the machine.
- PyPI `litellm` 1.82.7 and 1.82.8 were malicious (March 2026, since removed). Pin the version and install with hashes.
- Codex rejects `wire_api = "chat"` (0.157.1–0.162.1): a Chat-Completions-only server cannot be a Codex provider.
- "Qwen4" does not exist: no Qwen4 release was on QwenLM's GitHub or Hugging Face on 2026-10-12 (Qwen3.8-Flash-Next previews its architecture); "Qwen 4 Coder" scores are unsourced.
- `opencode-ai/opencode` and the Python `kimi-cli` are archived; use `anomalyco/opencode` and `@moonshot-ai/kimi-code`.

**The agent answers in prose and never edits a file.** Tool calling is off or the parser is wrong. Recovery: restart vLLM with `--enable-auto-tool-choice` and the model card's parser (on llama.cpp, add `--jinja`), then re-run the smoke task.

**Claude Code still calls Anthropic.** A saved login or API key, or a settings file, overrides the shell variables. Recovery: set `ANTHROPIC_API_KEY=""` and `ANTHROPIC_AUTH_TOKEN`, run `/status`, and move the variables into the settings file's `env` block.

**Long tasks collapse halfway.** `--max-model-len` is too small or the KV cache does not fit. Recovery: check `/context`, raise `--max-model-len` if memory allows, and split the task with a plan file.

**The eval looks great and production does not.** The eval set holds public code the model trained on, or only easy tasks. Recovery: use private code, check that hidden tests fail first, and add harder tasks from recent incidents.

**The server rejects requests with "Unexpected value(s) for the `anthropic-beta` header" or "Extra inputs are not permitted".** It does not implement Anthropic's beta headers and tool-schema fields. Recovery: set `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1`, which strips them (environment-variable reference, checked against v2.1.283), and re-run the smoke task.

**The licence blocks the use you planned.** The expensive failure. Recovery: route the job type back to the baseline and pick a model legal has approved. Prevent it with step 1 of the server setup.

## Where to go next with open-weight models

- [Models hub](/en/ecosystem/models/) — Current versions, prices and defaults.
- [Reading coding-agent benchmarks](/en/ecosystem/models/benchmarks/) — What Terminal-Bench 4.0 and SWE-bench measure, and what they miss.
- [Gateways and local models](/en/ecosystem/agent-tools/gateways-local-models/) — LiteLLM, Claude Code Router, CC Switch, Ollama and LM Studio in detail.
- [Where the model runs](/en/org/model-hosting/) — Cloud routes, residency and zero retention for the frontier models.
- [OpenCode](/en/comparison/opencode/) — The open-source agent that routes across providers.
- [When a new model ships](/en/ecosystem/models/new-model-playbook/) — Re-run your eval set within 48 hours of a release.
