Manual Testing Guide
This guide walks through scenarios that automated tests cannot fully cover — anything involving a real agent CLI, real tracker behavior, real dashboard UI, or the interplay between the orchestrator and an actual filesystem. Use it when:
- Validating a feature before tagging a release
- Reproducing a bug report from an issue
- Onboarding to a part of the codebase you have not touched before
- Sanity-checking a change that touched multiple subsystems
Each scenario is independent. Pick the ones relevant to what you changed. Scenarios marked (local) use the local Markdown tracker (tracker.kind: local, issues in .itervox/issues/) and need no tracker credentials. Scenarios marked (LLM) require a real claude or codex CLI; if you do not have one installed, skip them. Scenarios marked (offline) need neither a daemon nor an agent.
Prerequisites
Section titled “Prerequisites”# Build a fresh binary (embeds the dashboard)make build
# Use it by absolute path in the scenarios belowexport ITERVOX="$PWD/itervox""$ITERVOX" --helpYou will create throwaway project directories under /tmp/itervox-test-* for each scenario. Clean them up afterwards with rm -rf /tmp/itervox-test-*.
For LLM scenarios you also need either claude (with claude login completed) or codex (with codex login) on your PATH.
Where the settings go
Section titled “Where the settings go”itervox init writes the layered sections (agent, automations, dependencies, hooks, polling, server, tracker, workspace, …) to .itervox/settings.jsonc, and that file wins over the WORKFLOW.md front matter. A front-matter agent.max_retries: 1 in a project made by init is overridden by the "max_retries": 5 in settings.jsonc. So:
- put the settings in the scenarios below in
.itervox/settings.jsonc(or.itervox/settings.local.jsonc, which is gitignored and wins over it); - put profiles (
agent.profiles) inWORKFLOW.md, whereinitwrites them; - run
"$ITERVOX" config showto see each value and the layer it came from.
The daemon reloads when WORKFLOW.md or a settings file changes.
Finding the dashboard
Section titled “Finding the dashboard”The dashboard is not on a fixed port. init scaffolds server.port: 0, so the operating system picks a free port at every start. (A hand-written WORKFLOW.md without server.port still uses 8090.) To find the URL:
cat .itervox/dashboard_urlwhile the daemon runs (the file is removed when it stops);"$ITERVOX" doctorprintsdashboard URL: http://127.0.0.1:<port>/ (reachable);- the daemon’s log line
dashboard URL: on a terminal it carries the token (?token=…); otherwise the token is withheld and the line names thetoken_file(<logs-dir>/api-token) that holds it.
Smoke test (5 minutes, local)
Section titled “Smoke test (5 minutes, local)”The fastest path to confirming a build is healthy.
Setup:
mkdir /tmp/itervox-test-smoke && cd /tmp/itervox-test-smokegit init -q"$ITERVOX" init --tracker local --runner claudeinit --tracker takes linear, github or local. The in-memory tracker is no longer offered (init --tracker memory exits with unknown tracker "memory" (supported: linear, github, local)); use itervox demo for a throwaway board with fake issues and a scripted agent.
Expect from init:
WORKFLOW.md(profiles and the prompt) and.itervox/settings.jsonc(everything else, with"server": {"port": 0}).itervox/agents/<name>/SOUL.mdandINSTRUCTIONS.mdforimplementer,reviewer,input-responderanddeps-analyzer(plus amerge-botdirectory).itervox/issues/ITX-1.md, an example issue inBacklog, and six playbooks in.itervox/playbooks/.itervox/.gitignorelisting the runtime files (HEARTBEAT.md,daemon.pid,dashboard_url,logs/,runtime/,/*.json,/attest.key, …)- The printed next steps end with “Find the dashboard URL: check .itervox/dashboard_url or run “itervox doctor""
Run:
"$ITERVOX"Expect:
- No terminal UI starts (the daemon is always headless); the logs stream on stderr
- A
HTTP server listening addr=127.0.0.1:<port>line, then adashboard URLline (see Finding the dashboard) "$ITERVOX" watch --oncein another shell starts withitervox · 0/3 running(max_concurrent_agentsis 3 in the scaffold).itervox/HEARTBEAT.mdexists, names the dashboard URL, and refreshes after state changes (at most every ~15 s)
Verify:
cat .itervox/dashboard_url # http://127.0.0.1:<port>/"$ITERVOX" doctor | grep 'dashboard URL'head .itervox/HEARTBEAT.md # Daemon: running, Dashboard: http://127.0.0.1:<port>/"$ITERVOX" status # this project's running daemon(s)Quit with Ctrl+C. The daemon drains: in-flight agent turns get up to --shutdown-grace (30 s) to finish; with nothing running it exits at once and removes daemon.pid, dashboard_url and HEARTBEAT.md.
Schema 2 validation (local)
Section titled “Schema 2 validation (local)”Goal: verify the daemon hard-fails on legacy schemas and prints actionable guidance, and that the migration gets you to a config the daemon loads.
mkdir /tmp/itervox-test-schema && cd /tmp/itervox-test-schemagit init -q
cat > WORKFLOW.md <<'EOF'---tracker: kind: localagent: profiles: implementer: prompt: "You are an implementer."---Just a body.EOF
"$ITERVOX" --workflow WORKFLOW.mdExpect: the daemon exits with a non-zero code and the stderr message contains:
WORKFLOW.md is missing itervox_schema_version.Run: itervox init --update --workflow WORKFLOW.mdThen: run the migration and verify:
"$ITERVOX" init --update --workflow WORKFLOW.mdWORKFLOW.md.bakis createdWORKFLOW.mdnow starts withitervox_schema_version: 2- Inline
prompt:is gone fromagent.profiles.implementer .itervox/agents/implementer/SOUL.mdexists.itervox/agents/implementer/INSTRUCTIONS.mdcontains the migrated prompt text under a “Migrated Instructions” headingtracker.kindmoved to.itervox/settings.jsonc(“moved N setting(s) from WORKFLOW.md to .itervox/settings.jsonc”)
Check: start the daemon again and read its first error, if any:
"$ITERVOX" --workflow WORKFLOW.mdA legacy profile that had no command: (like the one above) can come out of the migration still without one. The daemon then refuses to start with config: agent.profiles.implementer.command is required in schema 2. If you see that, the migration left the profile incomplete: note it, add command: claude to the profile, and start again; the daemon then starts. A migration that fills in command makes the second start succeed without the edit.
Migration warning for workspace.auto_clear (local)
Section titled “Migration warning for workspace.auto_clear (local)”Goal: verify the v0.2.0 semantic-change notice fires for a workflow that still has workspace.auto_clear: true in its front matter.
mkdir /tmp/itervox-test-autoclear && cd /tmp/itervox-test-autocleargit init -q
cat > WORKFLOW.md <<'EOF'---itervox_schema_version: 2tracker: kind: localworkspace: auto_clear: trueagent: profiles: implementer: command: claude soul_file: .itervox/agents/implementer/SOUL.md instructions_file: .itervox/agents/implementer/INSTRUCTIONS.md---EOF
mkdir -p .itervox/agents/implementertouch .itervox/agents/implementer/SOUL.md .itervox/agents/implementer/INSTRUCTIONS.md
"$ITERVOX" init --update --workflow WORKFLOW.md 2>&1Expect: the output contains the warning line:
itervox init --update: warning: workspace.auto_clear: true semantics changed in v0.2.0 — workspace is now cleared only when the issue reaches a terminal tracker state ...and, in the same run, workspace.auto_clear among the settings moved to .itervox/settings.jsonc.
The warning fires once. The migration reads the front matter, and the same run moves workspace.auto_clear out of it, so a later init --update (after you remove or move WORKFLOW.md.bak, which it refuses to overwrite without --force) prints no warning. To see it again, put auto_clear: true back in the front matter.
Agent handoff chain (local + LLM)
Section titled “Agent handoff chain (local + LLM)”Goal: end-to-end validation of the v0.2.0 handoff convention. This is the load-bearing v0.2.0 scenario.
Setup: create a project with two profiles (researcher, implementer) and dispatch the same issue through both in sequence.
mkdir /tmp/itervox-test-handoff && cd /tmp/itervox-test-handoffgit init -q && git commit -q --allow-empty -m init"$ITERVOX" init --tracker local --runner claude
# Add a researcher profile alongside the default implementermkdir -p .itervox/agents/researchercat > .itervox/agents/researcher/SOUL.md <<'EOF'You are a researcher. You map relevant code and flag risks.EOFcat > .itervox/agents/researcher/INSTRUCTIONS.md <<'EOF'# Researcher INSTRUCTIONS## Workflow- Inspect the issue.- Write findings to `run.handoff_path` (see Run Context block).- Do NOT edit code.
## Handoff Protocol- Read the prior handoffs above (if any).- Write your deliverable to `run.handoff_path` before exiting.EOFRegister the researcher under agent.profiles in WORKFLOW.md (command, soul_file, instructions_file, like the implementer). Then start the daemon:
"$ITERVOX"Step 1 — researcher run: create a task for the researcher and start it:
"$ITERVOX" task "Map the code for the login flow" --profile researcher(or set the issue’s profile on the dashboard and move it to Todo). Watch the worker complete.
Verify after the researcher exits: handoffs live in the issue’s worktree, not in the project directory. With the scaffold’s workspace.worktree: true the worktree is <workspace.root>/worktrees/<identifier> (workspace.root is in .itervox/settings.jsonc), on branch itervox/<identifier in lower case>:
WT="<workspace.root>/worktrees/ITX-2"ls "$WT/.itervox/handoff/"# Expect: 2026-XX-XXTHH-MM-SS.SSSZ_researcher.mdgit -C "$WT" log --oneline -3# Expect: a commit "chore(itervox): record agent handoff"If the agent wrote no handoff, the worker synthesizes one from the session summary (log: worker: agent wrote no handoff — synthesized from session summary) rather than counting the run as a success without one.
Step 2 — implementer run: switch the issue’s profile to implementer on the dashboard and move the issue back to Todo ("$ITERVOX" dispatch ITX-2, or edit state: in its file).
Verify the implementer’s prompt: check the issue’s log on the dashboard (or "$ITERVOX" logs ITX-2). The rendered prompt should contain a ## Prior Agent Handoffs block with the researcher’s deliverable as a sub-heading.
After the implementer completes:
ls "$WT/.itervox/handoff/"# Expect both:# 2026-XX-XXTHH-MM-SS.SSSZ_researcher.md# 2026-XX-XXTHH-MM-SS.SSSZ_implementer.mdgit -C "$WT" log --oneline | grep 'record agent handoff'Itervox commits each worker run’s handoff on the issue branch itself (chore(itervox): record agent handoff, only .itervox/handoff/), so git status in the worktree does not list them as new files. A read-only reviewer’s handoff is the exception: it stays uncommitted for the next implementer run to commit.
Partial handoff rename on failure (local + LLM)
Section titled “Partial handoff rename on failure (local + LLM)”Goal: verify a worker that fails mid-deliverable has its handoff renamed to .partial.md.
Setup: as above. In .itervox/settings.jsonc set "max_retries": 1 under agent: the run is retried once, then the issue moves to tracker.failed_state (or is paused when none is set). Do not use 0: agent.max_retries: 0 means unlimited retries, so the issue would be retried forever. Use a profile whose INSTRUCTIONS ask the agent to write its handoff and then fail, or let a run go silent past agent.stall_timeout_ms for a stall.
Steps:
- Start a worker that will fail after writing to
run.handoff_path(for example, INSTRUCTIONS that tell the agent to write a draft handoff and then runexit 1via Bash). - Watch the worker exit failed (or stalled).
Verify:
ls "$WT/.itervox/handoff/"# If the agent wrote a handoff before failing:# 2026-XX-XXTHH-MM-SS.SSSZ_<profile>.partial.md# (note .partial.md suffix)If the agent never wrote anything, there is no file to mark partial — that is also correct behavior.
Negative case (regression guard): plant a handoff from an earlier run, then run a worker that fails without writing its own (e.g. a broken profile command). The predecessor file must NOT be renamed, because only files written since the failed run started are eligible:
echo "predecessor work" > "$WT/.itervox/handoff/2020-01-01T00-00-00.000Z_implementer.md"# After the failure:ls "$WT/.itervox/handoff/"# Expect: 2020-01-01T00-00-00.000Z_implementer.md (NOT renamed)AutoClear semantics: success path (local + LLM)
Section titled “AutoClear semantics: success path (local + LLM)”Goal: verify workspace.auto_clear fires when no further work is pending on the workspace.
Setup (.itervox/settings.jsonc):
"workspace": { "auto_clear": true /* keep root, worktree, clone_url, base_branch */ },"agent": { "auto_review": false /* for this scenario */ }Steps:
- Dispatch a worker that succeeds. With no reviewer to run after it, the worker’s success is the end of the issue’s work on this workspace.
- Watch the orchestrator log for
orchestrator: workspace auto-clearedright after the worker’s success.
Verify:
ls "<workspace.root>/worktrees/"# Expect: the issue's worktree is GONE# Logs are preserved separately under the logs directoryAutoClear semantics: deferred behind reviewer (local + LLM)
Section titled “AutoClear semantics: deferred behind reviewer (local + LLM)”Goal: verify the v0.2.0 change that lets auto_clear coexist with auto_review.
Setup (.itervox/settings.jsonc):
"workspace": { "auto_clear": true },"agent": { "reviewer_profile": "reviewer", "auto_review": true }Steps:
- Dispatch a worker (non-reviewer profile). It succeeds.
- Watch the orchestrator log —
orchestrator: dispatching reviewershould fire AFTER the main worker but BEFORE the workspace clear. - The reviewer runs. It succeeds.
- The workspace clear fires now, after the reviewer’s success.
Verify:
- The reviewer’s prompt logs show it could read the workspace (not empty)
- After the reviewer’s success, the worktree is removed
- No
skipping auto-review because workspace auto-clear is enabledlog line — that line was removed in v0.2.0
AutoClear semantics: failure path (local + LLM)
Section titled “AutoClear semantics: failure path (local + LLM)”Goal: verify auto_clear also fires when an issue moves to failed_state after max retries.
Setup (.itervox/settings.jsonc):
"agent": { "max_retries": 1 },"tracker": { "failed_state": "Failed" },"workspace": { "auto_clear": true }Steps:
- Dispatch a worker that always fails (a broken profile
command). - After the retry, the issue moves to “Failed”.
Verify:
- Orchestrator log:
worker: max retries exhausted→orchestrator: workspace auto-cleared - Worktree removed
- The issue file’s
state:isFailed
Negative case (regression guard): with max_retries: 5, verify the worktree persists between failed retries — only cleared on the final terminal transition.
Automations: where to put them
Section titled “Automations: where to put them”Automations are a list section: put the whole automations list in .itervox/settings.jsonc (the snippets below are that list’s entries). At startup, and on every reload, the daemon logs automations: compiled total_configured=N registered=N dropped=0 … with a count per trigger type; check it after each change. A rule the loader cannot use fails startup with startup: config invalid … error="automation \"<id>\": …", except a rule without id, profile or trigger.type, which is currently skipped without an error (it is missing from total_configured).
Automation: cron trigger (local + LLM)
Section titled “Automation: cron trigger (local + LLM)”Goal: verify cron-driven dispatch fires on schedule.
{ "id": "every-minute", "enabled": true, "trigger": { "type": "cron", "cron": "* * * * *", "timezone": "UTC" }, "filter": { "states": ["Todo"], "limit": 1 }, "profile": "implementer", "instructions": "A cron-driven test fired. No-op."}The schedule is trigger.cron; a schedule: key is not read, and startup fails with cron automations require trigger.cron. Put one issue in Todo.
Verify after ~1 minute:
- The startup log counts it:
automations: compiled … cron=1 - An automation run starts for the
Todoissue (Running sessions, or the run history) - The dashboard’s Live Ops strip shows the automation activity
Automation: input_required trigger (local + LLM)
Section titled “Automation: input_required trigger (local + LLM)”Goal: verify input-required automations dispatch a helper.
Setup: a profile whose agent asks a question that triggers input_required (e.g. “Which option?\n1. …”). Configure an input-responder automation:
{ "id": "input-responder", "enabled": true, "trigger": { "type": "input_required" }, "filter": { "input_context_regex": "Which option" }, "profile": "input-responder", "policy": { "auto_resume": true }}input_context_regex belongs under filter. Under trigger it is not read, so the rule matches every question.
Steps:
- Dispatch the main worker. It asks a question.
- The orchestrator records the input request.
- The input-responder automation matches and dispatches.
- The responder produces a reply (its profile has
provide_inputinallowed_actions). - With
auto_resume: true, the daemon resumes the main worker with the response.
Verify:
- The Live Ops strip shows the input-required → resuming → running progression
- The issue log shows the responder’s reply text
- The main worker eventually finishes (or asks another question)
Automation: pr_opened trigger (local + LLM, GitHub remote)
Section titled “Automation: pr_opened trigger (local + LLM, GitHub remote)”Goal: verify the pull-request detection that dispatches in-process.
Setup: a repository with a GitHub remote, gh installed and logged in, and a worker profile whose agent pushes its branch and opens a pull request. Configure:
{ "id": "pr-followup", "enabled": true, "trigger": { "type": "pr_opened" }, "profile": "reviewer", "instructions": "A PR was opened. Run a quick review pass."}How detection works: after a worker run (not an automation run) succeeds, Itervox runs gh pr view <branch> --json url,state in the worktree for the run’s own branch, and takes the URL of an open PR. It does not scan the agent’s output, so printing a PR URL is not enough: the PR must exist for that branch. The trigger fires the first time Itervox comments the PR link on the issue (🔗 Pull request created: <url>); a PR whose link is already in the issue’s comments does not fire it again.
Steps:
- Dispatch a worker that opens a pull request for its branch.
- When it exits, Itervox comments the PR link on the issue.
- The
pr_openedautomation dispatches after the worker’s exit.
Verify:
- The issue log has
worker: pr_opened url=https://github.com/<owner>/<repo>/pull/<n> - An automation run with the
reviewerprofile starts for the issue, and its trigger context carries the PR URL and branch
Automation: rate_limited trigger and profile fallback (local + LLM)
Section titled “Automation: rate_limited trigger and profile fallback (local + LLM)”Goal: verify the rate-limit auto-switch path.
Setup: a custom error pattern that the test agent will print, and the fallback rule:
"agent": { "max_retries": 1, "rate_limit_error_patterns": ["test_rate_limit_marker"]},"automations": [ { "id": "rl-fallback", "enabled": true, "profile": "implementer", "trigger": { "type": "rate_limited" }, "policy": { "auto_switch": true, "switch_to_profile": "implementer-fallback", "switch_to_backend": "codex" } }]Create a profile implementer-fallback in WORKFLOW.md (for switch_to_backend: codex, one whose command runs codex). Make the primary profile’s agent print test_rate_limit_marker to stderr.
What the loader requires, each verified at startup:
profile— every automation needs one. Arate_limitedrule without it is currently dropped without an error (it is missing fromautomations: compiled total_configured=…); a fix is in progress, so check that count rather than relying on an error.policy.switch_to_profilenaming an existing, enabled profile.policy.auto_switch: true(orauto_resume: true); without it startup fails withrate_limited automations require policy.auto_switch: true or policy.auto_resume: true.
Steps:
- The primary worker dispatches and fails with the marker on stderr.
- A text-pattern match switches when retries are exhausted (a structured vendor limit, such as Claude’s quota error, switches on the first failure without using a retry).
- The rate_limited automation fires; the issue’s profile switches to the fallback.
Verify while the fallback run is in progress (through the API, or the issue card on the dashboard). Give the fallback profile a slow agent if you need time to look:
GET /api/v1/state:autoSwitches[]has a row for the issue with"source": "automation","toProfile": "implementer-fallback"and"toBackend": "codex"GET /api/v1/issues: the issue’sagentProfileisimplementer-fallback, with anautoSwitchobject- The fallback run uses that profile
After the fallback run succeeds, the switch is cleared (a rate_limited switch lasts until a successful run): autoSwitches[] has no row for the issue and agentProfile is back to what it was, the operator’s pinned profile if there was one. A backend_fallback switch follows its switch_back setting instead.
Automation: issue_entered_state chain (local + LLM)
Section titled “Automation: issue_entered_state chain (local + LLM)”Goal: verify state-transition-driven chaining (the foundation of the handoff factory pattern).
Setup: add Researching and Implementing to tracker.active_states, give the researcher and implementer profiles move_state in allowed_actions, and define:
[ { "id": "kick-research", "trigger": { "type": "issue_entered_state", "state": "Todo" }, "profile": "researcher", "instructions": "When your handoff is written, run: itervox action move-state --state Researching" }, { "id": "kick-implement", "trigger": { "type": "issue_entered_state", "state": "Researching" }, "profile": "implementer", "instructions": "When done, run: itervox action move-state --state Implementing" }]The trigger takes one state, trigger.state (singular); states: [...] is not read and startup fails with issue_entered_state automations require trigger.state. The rule cannot move the issue itself: policy.move_to_state here fails startup with policy.move_to_state is only meaningful on blockers_resolved triggers. The agent moves it with itervox action move-state --state <state>, which needs move_state in its profile’s allowed_actions.
Steps:
- Create an issue in
Todo. - Watch the orchestrator log dispatches as each automation fires.
Verify:
- The researcher runs first and writes a handoff to
.itervox/handoff/<ts>_researcher.mdin the worktree - The issue moves to
Researching(by the agent’smove-state) - The implementer runs; its prompt contains the researcher’s handoff
- The issue moves to
Implementing
Automation: blockers_resolved trigger (local + LLM)
Section titled “Automation: blockers_resolved trigger (local + LLM)”Goal: verify dependency-driven dispatch.
Setup: two issues, A blocks B (in B’s file: blocked_by: [ITX-A]), B in Backlog. Give the automation’s profile move_state in allowed_actions, then configure:
{ "id": "unblock-handler", "enabled": true, "trigger": { "type": "blockers_resolved" }, "profile": "implementer", "policy": { "move_to_state": "Todo" }}policy.move_to_state requires the profile to allow move_state: without it startup fails with policy.move_to_state requires profile "implementer" to allow "move_state".
The trigger comes from the dependency audit, which runs every agent.dependency_audit_refresh_interval_ms — 10 minutes (600000) by default. Lower it for the test, for example "dependency_audit_refresh_interval_ms": 60000 under agent.
Steps:
- Wait for an audit to record B as blocked by A (the Deps tab shows the edge).
- Move issue A to
Done. - The next audit sees B’s blocker resolved; the blockers_resolved automation fires.
Verify:
- The snapshot’s dependency audit shows B as unblocked
- The automation dispatches a run for B
- B moves to
Todo
Daemon-backed agent actions (local + LLM)
Section titled “Daemon-backed agent actions (local + LLM)”Goal: verify the /api/v1/agent-actions/* permission gate.
For each action (comment, comment_pr, move_state, create_issue, provide_input):
Positive case: create a profile with the action in allowed_actions. Have the agent invoke it via itervox action ... (comment, create-issue, move-state, provide-input). Verify the action succeeded (tracker comment posted, issue moved, etc.).
Negative case: remove the action from allowed_actions. Verify the daemon’s HTTP layer rejects the request with 403 and the agent’s stdout/stderr contains the denial.
Token isolation check: cat .itervox/runtime/* — bearer grants should be ephemeral, scoped to a single run. They should NOT contain ITERVOX_API_TOKEN (the main dashboard token).
Guardrails: a held pipeline push (lanes) (local)
Section titled “Guardrails: a held pipeline push (lanes) (local)”Goal: a push that changes a guardrails.lanes.human file (.github/**, Makefile, WORKFLOW.md, …) is held for approval; approving lets that push through once, any other reply refuses it. See Lanes.
Fastest path, no agent CLI: the pipeline-guard demo runs a scripted agent behind the real guardrails, with a stub gh, so nothing leaves the machine:
scripts/pipeline-guard-demo.sh --part guarded --bin "$ITERVOX"Expect (it answers as the operator through the API):
the push is held in the human lane: .github/workflows/ci.yml- the operator’s rejection, then
signed check: `test` exited with code 1 - after the code fix:
DEMO-1 is In Review,the pushed branch leaves .github/ unchanged, and two signed-check records that verify - the last line
DEMO_RESULT=guarded ok
By hand (LLM): in a project with a remote, ask the agent to edit .github/workflows/ci.yml and push.
- The push waits. The dashboard shows Pipeline change in the Needs-attention row and the issue’s Now panel, with the files (
Pipeline change: the agent wants to push changes to .github/workflows/ci.yml …)."$ITERVOX" watchlists it underNEEDS YOU. - Approve:
"$ITERVOX" approve <issue>(or replyapproveon the dashboard). That push goes through; the next commit asks again. - Reject: reply anything else, for example
"$ITERVOX" reply <issue> "fix the code, not the pipeline". The push is refused and the agent receives your reply. - While the push waits, the run is not failed for being silent;
agent.turn_timeout_msstill bounds the wait. merge_prlater refuses the pull request withlane_human, even after the approval.
Also check: "$ITERVOX" doctor prints a Lanes section (human: … (default), second_opinion: off).
Signed checks (verify, itervox attest verify) (local + LLM, or offline)
Section titled “Signed checks (verify, itervox attest verify) (local + LLM, or offline)”Goal: the daemon runs a profile’s verify checks itself after the agent’s run, signs the records, and a failing check holds the run. See Signed checks.
With the daemon (LLM): add to a profile in WORKFLOW.md:
verify: - name: test run: make test-
Start the daemon.
.itervox/attest.key(0600) and.itervox/attest.pubappear; a project with noverifygets no key files. -
Run an issue whose tests fail after the agent’s run. The run is held input-required with
Signed checks failed (verify): …and`test` exited with code 1 (`make test`). -
Reply with a fix instruction; the checks run again when that run ends.
-
The issue panel’s Signed checks section lists each run’s checks;
GET /api/v1/issues/<issue>/attestationsreturns them withverified. -
Verify a record file offline:
Terminal window "$ITERVOX" attest verify .itervox/attestations/<issue>/<run>_<profile>.jsonl --tree <worktree>Expect
ok check test …,ok seal …andall 2 records verify (key …), exit 0.
Offline, no daemon: the referee signs check runs in the same format:
cd "$(mktemp -d)" && git init -q && echo hi > a && git add a && git commit -qm init"$ITERVOX" referee keygen --out /tmp/itervox-test-keys"$ITERVOX" referee run --name test --key /tmp/itervox-test-keys/attest.key --out /tmp/itervox-test-checks.jsonl -- 'true'"$ITERVOX" attest verify /tmp/itervox-test-checks.jsonl --pubkey /tmp/itervox-test-keys/attest.pub --tree .- Expect
all 2 records verify, exit 0. - Change
"exit_code":0to1in the file: each altered record showsFAIL … (attest: signature does not verify), exit 1. - Restore it and add an untracked file:
--tree .fails withrecord is for a different tree, exit 1.
itervox hook in your own sessions (offline)
Section titled “itervox hook in your own sessions (offline)”Goal: the guardrail policy answers a Claude Code or Codex PreToolUse hook. See Interactive sessions (itervox hook).
In a project made by init (it has a guardrails: block):
hook() { printf '{"hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"%s"},"cwd":"%s"}' "$1" "$PWD" | "$ITERVOX" hook claude-pretooluse; echo; }hook 'git push --force origin main' # "permissionDecision":"deny" — push to main is refusedhook 'gh pr merge 12' # "deny" — agents do not merge their own pull requestshook 'terraform apply' # "ask" — needs a human approval (approval_commands)hook 'ls -la' # no output: allowed"$ITERVOX" hook install claude --print # the .claude/settings.json entry; writes nothing- An
Editof.github/workflows/ci.ymlanswersask(“pipeline file: … is in guardrails.lanes.human”). codex-pretoolusedenies what Claude would be asked about (Codex has no ask).hook install claudemerges the entry into.claude/settings.json; running it again changes nothing.
itervox referee (offline)
Section titled “itervox referee (offline)”Goal: the CI referee sorts a branch’s files into lanes, from the policy on the base ref. See referee.
cd "$(mktemp -d)" && git init -q -b mainecho hi > a.txt && git add . && git commit -qm initgit checkout -qb featmkdir -p .github/workflows && echo 'x: 1' > .github/workflows/ci.yml && echo b > b.gogit add . && git commit -qm change"$ITERVOX" referee lanes --base main; echo "exit=$?"Expect: lane: human, policy: default (.referee.yml not on the base ref), .github/workflows/ci.yml under human (1), b.go under auto (1), and exit=20. A branch that changes only b.go exits 0; --json prints the same as one object.
Also:
"$ITERVOX" referee audit --base mainprintsaudit: no findings (main...HEAD)here (exit 0). Addcontinue-on-error: trueto a workflow step on the branch: it reports[ci_continue_on_error] failure ignored in a CI workflowand exits 20.- A
.referee.ymlcommitted only on the branch (evenhuman: []) does not change the result: the output still sayspolicy: default (.referee.yml not on the base ref), because the policy is read from the base ref.
Patches only (push: daemon) (local + LLM, remote)
Section titled “Patches only (push: daemon) (local + LLM, remote)”Goal: the agent gets no push ability; Itervox pushes after the gates. See Patches only.
Setup: on a profile in WORKFLOW.md, push: daemon (keep guardrails.enabled: true, the default). Startup refuses it without guardrails (push: daemon needs guardrails.enabled: true) and with agent.ssh_hosts.
Steps and expected results:
- Ask the agent to
git pushorgh pr create. The shims refuse withthis profile emits patches; the daemon pushes after its checks. - Let the run succeed with a commit. After
hooks.after_run_required,verifyandrequire_evidence, Itervox pushes the branch with its own credentials and opens the pull request (withgh; on GitLab it pushes but opens no merge request). - With
verifyon the profile, leave uncommitted changes after the checks: nothing is pushed, and the run waits for input with “the tree changed after the checks”. - A human-lane change is held as a Pipeline change, as in the lanes scenario.
Model servers and local models (local)
Section titled “Model servers and local models (local)”Goal: a profile whose command points at a model server on this machine is probed by doctor. See Local models.
Setup: add a profile to WORKFLOW.md:
coder-local: command: ANTHROPIC_BASE_URL=http://127.0.0.1:8000 claude pool: local soul_file: .itervox/agents/implementer/SOUL.md instructions_file: .itervox/agents/implementer/INSTRUCTIONS.mdSteps:
- With nothing on port 8000, run
"$ITERVOX" doctor. Its Model servers section readsANTHROPIC_BASE_URL=http://127.0.0.1:8000, used by coder-local: not answering, with aWARNING:namingcoli serve. The exit code does not change because of it. - Start any HTTP server on that port (your model server, or
python3 -m http.server 8000 --bind 127.0.0.1) and run doctor again:… used by coder-local: answering. - (LLM, with a real model server) run an issue on
coder-local. A limit on Anthropic does not hold it: its breaker is theclaude/locallane. - In the desktop app, the tray menu’s Model servers section lists the server for the project.
tmux sessions: New session, Stop, re-adoption (local + LLM)
Section titled “tmux sessions: New session, Stop, re-adoption (local + LLM)”Goal: sessions itervox starts in tmux are controllable, stop properly and survive a daemon restart. Needs tmux on the daemon’s PATH. See Sessions in tmux.
Steps:
- Dashboard → Sessions → New session: choose Claude Code, the project folder and an optional first prompt. A tab appears (started in tmux) with the pane’s screen, the attach command
tmux attach -t itervox-<id>, Stop session and Force stop.tmux lslistsitervox-<id>. - Type into it from the tab (Send) and watch the screen update.
- Stop session (admin token, asks first): Itervox presses Ctrl-C (twice for Claude Code), waits up to 5 seconds, then kills the tmux session if it is still there. The tab goes away; the feed records
session_stoppedwith outcomestoppedorkilled. - Start another session, then Force stop:
tmux kill-sessionat once (outcomekilled). - Re-adoption after a restart: start a session, then stop the daemon (
Ctrl+C) and start it again. The tmux session keeps running (tmux show-options -t itervox-<id> @itervox-projectprints this project’s directory). Within one discovery pass (15 s) the tab is back, says re-adopted after restart, and Stop works again. It shows Guardrails from the previous daemon run when guardrails were prepared for it. - Negative case: a tmux session named
itervox-<something>that you start by hand (no@itervox-projecttag) is not adopted; it stays a hand-started session.
With tmux missing, the panel shows tmux not found — install it to control sessions from here and New session is disabled.
Profile evals (itervox evals) (offline; live is LLM)
Section titled “Profile evals (itervox evals) (offline; live is LLM)”Goal: the released binary runs the profile evals. See Repo Analysis → Evals.
cd /tmp/itervox-test-smoke # any project with a WORKFLOW.md"$ITERVOX" evals recordedExpect: [evals] fixtures: built-in scenarios, then [evals] 8/8 pass with one PASS <profile>/<scenario> line each, and [evals] report saved to .itervox/evals/<timestamp>.json. No agent runs and no tokens are spent.
- With
.itervox/evals/fixtures/present, the scenarios come from there instead (or from-fixtures DIR). - The dashboard’s Repo Analysis → Evals shows the newest saved report.
- (LLM)
ITERVOX_EVALS_LIVE=1 "$ITERVOX" evals liveruns the live suite with real agents;recordandpromoteneed-fixturesor.itervox/evals/fixtures.
Dashboard surfaces (local + LLM)
Section titled “Dashboard surfaces (local + LLM)”Walk through each dashboard view and confirm visual coverage.
Live Ops strip (top of dashboard):
- Counters for Running, Paused, Retrying, Input Required, Automations, Capacity
- Red warning when automation queue saturated (force by dispatching >
max_automation_queue_lengthautomations)
Automation Queue panel:
- Queued entries show profile, backend, trigger, queued age
- Detail drawer (click an entry) shows trigger metadata, policy, profile permissions, dependency audit state, worker capacity
- Local search filters by issue identifier or trigger type
Deps tab:
- React Flow graph renders blocker → blocked relationships
- Badges: running, queued, terminal, blocked, unblocked, unknown
- Clicking a node opens the existing issue detail panel
- Graph does NOT edit dependencies (read-only)
Issue detail timeline:
- “Status changes” section shows recent transitions
- Source chips: tracker observation, dashboard action, worker lifecycle, automation move, system cleanup
- Automation-sourced rows include profile/backend/worker metadata
Settings:
- Automation editor: all 14 trigger types selectable, IANA timezone typeahead works
- Profile editor: SOUL/INSTRUCTIONS files visible, daemon actions checkboxes mutable
- Skills Inventory moved: Settings shows “Skills inventory moved to Repo Analysis” linking to
/repo#capabilities
Repo Analysis (/repo):
- Capabilities: scan shows skills/plugins/hooks/MCP/instructions counts; re-scan button works
- Runtime usage and cost: analytics recommendations and tokens per profile
- Prompt versions: per-profile versions; selecting one lists its runs
- Agent readiness: pass / warn / fail rows match
itervox doctor’srepo readiness:section
Auth flow:
- Start the daemon on the default loopback bind and read its URL (Finding the dashboard)
- Without
ITERVOX_API_TOKEN, the daemon auto-generates a token and prints the URL with it on a terminal, or writes it to<logs-dir>/api-tokenwhen stderr is not a terminal (auth is on by default on every bind, including loopback) - Visit
http://127.0.0.1:<port>/?token=<token>— should auto-capture and proceed - Visit
http://127.0.0.1:<port>/without token — should show the login screen - Set
server.allow_unauthenticated: trueand confirm token is no longer required
Terminal panel (itervox watch)
Section titled “Terminal panel (itervox watch)”The terminal UI was removed in v0.3: ./itervox --tui prints one line pointing
to itervox watch and exits with code 2. Watch the daemon from a second shell:
"$ITERVOX" watchPanel:
- Header: running/slots, queued, retrying, waiting for you, waiting for a slot,
blocked by dependencies, paused;
ALL DISPATCH PAUSEDafteritervox pause - One lane per running issue: title, profile · backend, elapsed, turn, tokens,
cost, and a
›activity line (last tool call and step) NEEDS YOUlists input requests (itervox reply …) and guardrail approvals (itervox approve …); run the printed command and the entry disappearsEVENTSshows runs starting and finishing as they happen- Ctrl-C quits; the shell prompt is unaffected (watch never touches the terminal mode)
Other modes:
"$ITERVOX" watch | catprints plain text, the status only when it changes"$ITERVOX" watch --events-only --jsonprints one JSON object per event- Stop watch, note the
--since Nit prints, trigger a run, then"$ITERVOX" watch --events-only --since N --onceshows exactly what was missed "$ITERVOX" watch --issue ITX-1streams that issue’s full log
Hot reload:
- Edit an automation’s instructions in
.itervox/settings.jsonc(orWORKFLOW.md) while the daemon is running - Save the file
- The dashboard reflects the change on the next automation tick without restart
- With an agent running, change Max workers (or any other setting) from the dashboard: the agent keeps running — a settings save does not reload the daemon. A hand edit of WORKFLOW.md or a settings file still does.
Config-invalid backoff:
- Introduce a YAML syntax error in WORKFLOW.md
- Save; daemon continues on the previously-valid config
- The dashboard shows the config-invalid banner
- Fix the YAML; banner clears within ~30s
Tracker integration (Linear / GitHub, optional)
Section titled “Tracker integration (Linear / GitHub, optional)”These require credentials. Use the local tracker first for everything above; come back to these when validating cross-tracker behavior.
Linear:
export LINEAR_API_KEY=lin_api_..."$ITERVOX" init --tracker linear --runner claude"$ITERVOX"- Issues from the project appear in the dashboard
- Comments posted via daemon actions appear in Linear within ~30s
- State moves via
move_stateaction transition the issue in Linear - Blockers come from native Linear issue relations
GitHub:
export GITHUB_TOKEN=ghp_..."$ITERVOX" init --tracker github --runner claude"$ITERVOX"- Issues from the repo appear
- Blockers come from
blocked by #123references in the issue body - The
pr_openedautomation fires when an agent opens a PR for its branch
Common failure modes and diagnostics
Section titled “Common failure modes and diagnostics”| Symptom | Most likely cause | Where to look |
|---|---|---|
go build fails with pattern web/dist: no matching files found | The dashboard was not built | Run make build |
Stale daemon.pid / dashboard_url after a failed start | Code path skipped fatalExit | cmd/itervox/exit.go — ensure os.Exit is not called directly |
| Dashboard 401 on every request | Auth token missing or wrong | Open the URL with ?token=<token>; ITERVOX_API_TOKEN env |
A setting in WORKFLOW.md has no effect | .itervox/settings.jsonc sets the same key and wins | itervox config show names the layer of each value |
| An automation never fires | Filter does not match, or the rule was skipped | The automations: compiled total_configured=N registered=N dropped=N startup line |
| Handoff file not visible to next agent | Looking in the project directory instead of the worktree | <workspace.root>/worktrees/<identifier>/.itervox/handoff/, and the record agent handoff commit |
make verify fails on lint-go | Missing golangci-lint or new lint rule | golangci-lint run directly for the full report |
Race condition reported by go test -race | Probably a real race | Stress with -race -count=10; see orchestrator-invariants skill |
When to update this guide
Section titled “When to update this guide”Add a scenario here when:
- You ship a new feature that has a non-trivial end-to-end behavior
- A regression escapes
make verifyand reaches a release - Onboarding feedback names a behavior that was hard to discover
- A bug report could have been caught earlier by a specific manual check
Update the “Last updated” note at the top when you check the page against a new release.
Remove a scenario when the behavior is fully covered by automated tests AND the manual check no longer provides additional confidence.