Local models (colibri)
Itervox runs agent CLIs, and most of them can talk to a model server other than their vendor’s. Point one at a server on your own machine and that profile’s runs cost nothing per token, and your code and prompts never leave the machine.
This guide uses colibri, an
open-source engine for very large open models. It serves an
OpenAI-compatible API and an Anthropic-compatible /v1/messages. The steps
are the same for any server with those APIs.
1. Start the server
Section titled “1. Start the server”Follow colibri’s own setup. It checks your hardware and recommends a model. Then start its API server:
coli serve # OpenAI-compatible /v1 and Anthropic-compatible /v1/messages on 127.0.0.1:80002. Point a profile at it
Section titled “2. Point a profile at it”Set the agent CLI’s base-URL variable at the start of the profile’s command.
Itervox skips NAME=value assignments when it works out the backend, so this
profile is still the claude backend, with the same flags, resume and
parsing.
agent: profiles: deps-analyzer-local: command: ANTHROPIC_BASE_URL=http://127.0.0.1:8000 claude --model <model name colibri serves> soul_file: .itervox/agents/deps-analyzer/SOUL.md instructions_file: .itervox/agents/deps-analyzer/INSTRUCTIONS.md- Claude Code reads
ANTHROPIC_BASE_URL, without/v1. If the server wants a key, putANTHROPIC_AUTH_TOKENin.itervox/.env, not in the command. - OpenCode talks to any OpenAI-compatible provider you define in
opencode.json(base URLhttp://127.0.0.1:8000/v1). Choose it with--model <provider>/<model>. See Agent backends. - Codex reads custom providers from
~/.codex/config.toml, not from the command.
To keep a local profile on this machine, don’t combine it with
agent.ssh_hosts. A run on a worker host would look for the server on that
host.
3. Give slow runs time
Section titled “3. Give slow runs time”A local model is slower than a vendor’s, so raise the timeouts that would otherwise kill a healthy run:
| Setting | Default | What to set |
|---|---|---|
agent.turn_timeout_ms | 3600000 (1 h) | the longest a whole run may take |
agent.stall_timeout_ms | 300000 (5 min) | longer than the slowest gap between two events |
agent.read_timeout_ms | 30000 | longer than the slowest single response chunk |
4. Check it
Section titled “4. Check it”itervox doctor probes every model server that a profile’s command points
at. It lists which profiles use each one, and warns when one does not
answer:
Model servers ANTHROPIC_BASE_URL=http://127.0.0.1:8000, used by deps-analyzer-local: not answeringWARNING: deps-analyzer-local point their agent at http://127.0.0.1:8000, which did not answer (…); their runs will fail until it does — start the model server (for colibri: `coli serve`) or fix ANTHROPIC_BASE_URLThe warning does not change doctor’s exit code, because the server may just
not be started yet. A URL taken from a variable ($LOCAL_MODEL_URL) is
listed but not checked. Doctor sees only assignments in the command itself,
not a CLI’s own config file.
5. Fall back to it when a vendor’s limit is hit
Section titled “5. Fall back to it when a vendor’s limit is hit”Itervox pauses dispatch to a backend whose usage limit was hit, and can
reroute work through
agent.backend_fallback.
Give the local profile its own pool, so a limit on Anthropic doesn’t hold
it, and add the pool as a step in the chain:
agent: profiles: coder: command: claude coder-local: command: ANTHROPIC_BASE_URL=http://127.0.0.1:8000 claude --model <model> pool: local backend_fallback: chain: [claude, codex, claude/local] profile_map: coder: { "claude/local": coder-local }When claude is limited and nothing else in the chain is healthy, coder
issues run on coder-local. They switch back once the limit resets, using
the usual switch_back policy.
In the desktop app
Section titled “In the desktop app”The desktop app watches these servers for you. It
reads each running daemon’s list of model servers
(GET /api/v1/model-servers) and merges
them across projects. The tray menu gets a Model servers section:
Model serverslocal 127.0.0.1:8000 — answering · used by 2 projectsGPU: NVIDIA GeForce RTX 4090 · 37% · 10.2/24.0 GBA server is named after its profiles’ pool, or “model server” when they
set none. The end of the project tabs shows the same list (hover it). The app
probes each server at most every 30 seconds, with a 3-second timeout, and
any HTTP answer counts, as in doctor. It notifies you when a server that was
answering misses two probes in a row, and again when it answers.
The app only probes 127.0.0.1, localhost and ::1. It shows these
servers as not checked, with the reason:
- a URL from a variable;
- any other host;
- a project with
agent.ssh_hosts, whose runs go to the worker hosts.
The GPU line comes from nvidia-smi when it is on your PATH. On Apple
silicon the app shows the integrated GPU, without its load. Otherwise it
shows “not reported”. CPU, memory and disk are on the dashboard’s
machine status.
Safety
Section titled “Safety”A local model does not make a run safer. The agent CLI runs with the same
permissions whichever model it talks to. A run with isolation: process
keeps network access, so it can still reach the server on 127.0.0.1. See
Safety.