scout-cli: qwen3:8b harness default, cap Ollama generations, Windows-safe herd kill

[jcdavis131]

Summary

Three follow-ups from PR #67's review, each its own commit:

1. e97a5cd — harness llm tier default → qwen3:8b
llm_exec.DEFAULT_OLLAMA_MODEL was qwen2.5:7b-instruct, which is not pulled on this box (ollama list: only qwen3:8b and agentos-qwen). Cam's call (2026-09-27): move the default to qwen3:8b. This changes which model produces "scout router probe" labels — a probe run without DOTTIE_LLM_MODEL set now records backend ollama:qwen3:8b instead of ollama:qwen2.5:7b-instruct. Updates docs/ROUTER_LABELS_RUNBOOK.md and docs/ARCHITECTURE.md, which both named the old default.

2. 3989b16 — cap every Ollama generation
None of scout-cli's five direct /api/chat//api/generate callers sent a generation cap (grepped for all of them): chat_with_metrics ignored its own max_tokens on the ollama branch (already honored it for koboldcpp), ollama_chat/_ollama_generate sent no options at all, the fallback direct posts in agent/cli.py/ava/cli.py/write/cli.py had the same gap, and scout ollama run --num-predict sent nothing when omitted. This is the same failure mode apps/dottie's PR #66 fixed: with llama-server's --context-shift, an uncapped call doesn't error, it keeps generating (708s / 81,920 tokens, measured there). Adds bigbang.core.llm.resolve_num_predict() — explicit argument > DOTTIE_OLLAMA_NUM_PREDICT env > 2048 default; 0/negative means no cap — reusing the same env var apps/dottie's OllamaPolicy reads, so one setting caps every local Ollama caller on the box. A malformed env value raises/fails closed rather than silently running uncapped, mirrored from that same PR. bigbang/core/ollama.py's complete() is deliberately untouched — it's documented as pure options passthrough with its own test asserting an omitted options= sends none.

As a side effect, scripts/bench_local_runner.py (which already passes an explicit --max-tokens on both backends) had an asymmetry: koboldcpp respected it, ollama silently didn't — so a prior ollama-vs-kobold bench comparison ran ollama uncapped against a capped kobold run. This fixes that.

3. 2b4e2a4 — Windows-safe herd close --kill
close_session's kill path called os.killpg/signal.SIGKILL unconditionally. Neither exists on Windows at all (os.killpg isn't a module attribute there; signal.SIGKILL isn't defined), so the first line raised AttributeError, caught by nothing. This was latent until PR #67 made _pid_alive correct on Windows — before that, alive() couldn't be trusted enough to route here. Extracts the existing TERM-then-KILL sequence into _kill_session_posix() unchanged, adds _kill_session_win32() using taskkill /PID <pid> /T /F via subprocess with an argument list (never shell=True), dispatches on sys.platform the same way _pid_alive already does. /T matters because the recorded pid is start_session's supervisor, not the real command.

Test plan

• uv sync --all-groups --frozen in a fresh worktree off origin/main
• uv run pytest tests -q from apps/scout-cli: 2683 passed, 1 failed, 2 skipped (302s). The one failure is the pre-existing tests/test_loop_plugin.py::test_evaluate_blocked_on_stale_metrics_exits_2 (fcntl import in packages/dottie-loop, Windows-only, not touched by this PR — matches what was flagged going in).
• Also hit, unrelated to this PR: factory/tests/test_mission.py::test_cancel_terminates_only_recorded_sleeping_process fails on this Windows box too — it probes liveness with os.kill(pid, 0), the exact CTRL_C_EVENT class of bug PR scout-cli: local models only (remove paid backends), 8b-first pickers, win32-safe liveness #67 fixed for herd/store.py's own _pid_alive, just not yet applied to factory/. Zero diff from this PR touches factory/; flagging for awareness, not fixing here (out of scope).
• New tests per commit: a pin test + updated expectations (commit 1); resolve_num_predict precedence/parsing, chat_with_metrics/ollama_chat/_options()/write's _ollama_chat cap coverage including malformed-env fails-closed (commit 2); both kill branches with fakes (POSIX os.killpg/os.kill/signal.SIGKILL patched with raising=False — none exist on this Windows dev box either, same reasoning TestPidAlive already documents), a dispatcher test per platform, and one real end-to-end test that starts a genuine long-running child through the CLI and confirms --kill --force actually terminates it — this one reproduces the exact pre-fix AttributeError on this box.
• uvx ruff@0.15.22 check on every touched file before/after each commit: 0 delta (pre-existing findings in llm.py, agent/cli.py, ava/cli.py, herd/store.py unchanged, all pre-dating this PR)
• scripts/check_documented_counts.py --check: 970 reproduces before and after all three commits — no documented-count update needed
• scripts/gate_audit.py, check_declared_capabilities.py, check_resolver_fallbacks.py, check_shell_true.py, check_cli_path_args.py, store_symmetry_audit.py — all --check clean, no new findings
• scripts/test_gate_audit.py, test_retrieval_eval.py, test_task_eval_slice.py, test_check_declared_capabilities.py, test_store_symmetry_audit.py, test_dag_next.py — all pass
• scripts/dag_next.py --check, factory/tests (1 pre-existing unrelated Windows failure, see above), python -m factory check — pass
• scripts/check_todos_timestamps.py, scripts/check_handoff_fresh.py --check (10-commit drift, within the 20-commit budget) — pass
• bigbang.cli leaks scan . and leaks history --max-commits 0 (1665 commits) — both clean

Notes

• No documented ruff counts changed — nothing here moved the 970 figure.
• Attribution: this session is Sonnet 5, matching the request to use Sonnet sub-agents; commit trailers and this PR credit Sonnet 5 accordingly.
• Never merging this myself, per instructions.

Update: review findings addressed (eec564f)

Three minor findings from this PR's review, all confirmed and fixed:

1. ollama_chat() swallowed a malformed DOTTIE_OLLAMA_NUM_PREDICT into a generic None. cap = resolve_num_predict(num_predict) sat inside its own broad except Exception: return None, so a typo'd env var was indistinguishable from a down server. Moved the resolve call before the client opens, so the actionable ValueError now reaches the caller — matching _ollama_generate/chat_with_metrics.

   Correcting the finding's own evidence: it claimed _ollama_generate/chat_with_metrics already contrasted with ollama_chat by propagating that ValueError into meta['error']. Empirically false — I reproduced it live before touching any code (DOTTIE_OLLAMA_NUM_PREDICT=lots against chat_with_metrics("ollama", ...) returned error: 'backend returned no completion', not the env var's name). _ollama_generate had the identical swallow (its own resolve_num_predict() call was inside the same broad except). Fixed both, and tightened test_ollama_chat_with_metrics_a_garbage_cap_fails_closed_not_uncapped to assert the message now names DOTTIE_OLLAMA_NUM_PREDICT — it failed against pre-fix code, confirming the test exercises something real, not just re-asserting the status quo. Added test_ollama_chat_a_garbage_cap_raises_not_silently_none for ollama_chat() itself.

   Left ava/cli.py's _route_with_ollama untouched: it wraps _core_ollama_chat in its own pre-existing except Exception: raw = None, so scout ava still degrades to keyword routing on a bad env var rather than surfacing the message — a separate, pre-existing resilience design in that plugin, out of this finding's scope.

One more disclosure, found while auditing every real call site of the now-raising ollama_chat/_ollama_generate for an unwrapped crash risk (repo-wide grep, not just within llm.py): bigbang/plugins/harness/executors/llm_exec.py's _ollama() calls llm._ollama_generate(...) directly with no local try/except. It is not agent/ava's _route_with_ollama/_ollama_planner (both already wrap their call in except Exception, confirmed) — this one call site propagates the ValueError up through complete()/run() into bigbang/plugins/harness/runner.py's _dispatch(), whose own except is narrowly (ExecutorUnavailable, NotApplicable) and does not catch it. It does not crash the harness run: _dispatch()'s caller, _attempt(), already wraps every executor call in a generic except Exception as exc: as part of the existing per-node recovery ladder (this is the same mechanism any other executor failure — a network timeout, a malformed response — already goes through). The observable difference is which fallback path engages, and only in auto mode: previously a garbage DOTTIE_OLLAMA_NUM_PREDICT degraded through ExecutorUnavailable straight to the stub executor with a misleading "returned no completion (run ollama pull qwen3:8b)" message; now it's recorded as a TOOL_FAILURE in the recovery ladder with error_class naming the actual problem ("ValueError: DOTTIE_OLLAMA_NUM_PREDICT must be an integer, got 'lots'"), which is more actionable, not less. In real mode this is a non-change: _dispatch lines 272-274 already re-raise ExecutorUnavailable there (if mode == "real" and isinstance(exc, ExecutorUnavailable): raise) into the same generic except Exception in _attempt, so real mode's path was identical before and after this fix. No test currently exercises this exact interaction (test_harness_executors.py never sets this env var; conftest.py pops it suite-wide precisely to keep a developer's local env quirks out of the test run), so flagging it here rather than claiming coverage that doesn't exist. Not changing llm_exec.py to swallow it back to the old behavior — that would be reintroducing the exact "typo degrades silently" failure mode this whole finding exists to close.

2. agent/cli.py's and ava/cli.py's fallback duplicates (reachable only when importing bigbang.core.llm has already failed) had zero coverage, as the finding noted — confirmed _HAS_CORE_LLM/_HAS_LLM are True in this repo's real environment, so the branch never ran, and the commit that added the cap logic there said as much explicitly ("forcing that import failure reliably under pytest is disproportionate to a defensive fallback"). That turned out to be easy and reliable: added one test per plugin using sys.modules["bigbang.core.llm"] = None (the documented sys.modules sentinel that forces ImportError, not a monkeypatch of real internals) + importlib.reload, driving the real fallback function with a fake httpx client, restoring and reloading again in finally.

3. bigbang/core/ollama.py's complete() applies no default cap of its own — deliberate (documented pure options passthrough with its own test), per the finding correctly identified as no action needed beyond a comment. Added one.

Verified: uv run pytest tests -q from apps/scout-cli — 2686 passed, 1 failed, 2 skipped (243s; 2686 = this PR's prior 2683 + 3 new tests). The one failure is the pre-existing tests/test_loop_plugin.py::test_evaluate_blocked_on_stale_metrics_exits_2 (fcntl import in packages/dottie-loop, Windows-only, unrelated). check_documented_counts.py --check: 970 reproduces, unchanged. gate_audit.py, goat_audit.py --min-mean 7.0, check_declared_capabilities.py, check_resolver_fallbacks.py, check_shell_true.py, check_cli_path_args.py, store_symmetry_audit.py, dag_next.py, check_todos_timestamps.py, check_handoff_fresh.py, leaks scan . — all clean, no new findings. State-pollution sweep around the new/changed test files: CLEAN.

🤖 Generated with Claude Code

https://claude.ai/code/session_014LHC4Z5q8HGJxLCSG3h8Db

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
arxiviq	Ready	Preview	Sep 28, 2026 12:07am UTC
dottie-os-console	Ready	Preview	Sep 28, 2026 12:07am UTC

[cursor]

Approved. Cursor Bugbot was not present after the first check poll, so that signal was skipped; remaining policy and review-state checks did not require human review. No reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}