Repository navigation
Commit ea9bb50
authored
fix(*): close the follow-ups from four merged pull requests (#940)
## Summary
Follow-ups to the review threads left on four merged pull requests:
#909, #912, #915 and #923. Each is a defect the review named, reproduced
before any change, and repaired with a test that fails against the old
code. This also addresses the Ollama spelling issue raised in this PR's
review.
**A cache-only window lookup no longer skips a model LiteLLM answers
from its table** (from #909). The gate that keeps metadata lookups off a
model's own server skipped `get_model_info` for any id carrying an
Ollama segment, including ids LiteLLM answers offline: `ollama/Llama3`,
`litellm_proxy/ollama/llama3` and `github/ollama/llama3` were sized on
the 200000-token fallback where main read 8192. The gate now puts
LiteLLM's own precondition to LiteLLM, keyed on the caller's id, and
answers cautiously only for an id with an Ollama segment when the
installed release does not ship that check.
**A window the model's own server reported is reused inside the turn**
(the second item of #909). A caller allowed to fetch resolved a local
model's window by asking its server and threw the answer away; the next
caller, which may not fetch, answered the fallback for the same model on
every iteration, for the life of the process. The answer is now kept
under the id the request goes out under, so `ollama/`, `ollama_chat/`
and the stored `ollama-chat/` spelling share one entry. The table is
consulted before the memo, so a row LiteLLM ships still outranks a
previous ask; only a real figure is stored, so a server that was down is
asked again.
**A measurement taken while the boot verify ran is no longer
overwritten** (from #915). The boot read a row before its handshake and
wrote its verdict from that read, so a Test recorded during the
handshake was replaced by a late timeout, refusal or pass. It reads the
row again immediately before the write and leaves a newer measurement in
place. The comparison is on `measured_at_ms`, so a modes relearn does
not stop the write. Four docstrings that described the replaced rule are
corrected.
**The Windows environment capture's memo is pinned, and its docstring
names the second failure mode** (from #912).
**A key is written under the spelling the loaders read** (from #923).
Below the top level pydantic reads the camelCase alias whenever it is
present, null or not, so preferring a non-null spelling sent a write to
a key the schema ignores, and `load_config` and the live readers then
disagreed. Only the top level drops a null block; below it the first
spelling present is the one read, and every caller now states which case
it is in.
**A write aimed at a wildcard map entry reaches that entry** (the
follow-up #923's review asked for). Below the top level the spelling
helper fell back to another spelling of a key, which is right for a
schema field and wrong for a map entry: with only
`tools.mcpServers.fooBar` present, a write to
`tools.mcpServers.foo_bar.enabled` selected `fooBar`, disabling a server
the caller never named. The catalog knows which segments a path binds,
so a bound segment is now matched as written. `providers` keeps the
tolerant rule, because its runtime reader folds those spellings through
`names_same_provider`.
**Configured Ollama models reach the metadata driver under its accepted
spelling.** The wizard and model picker store Ollama ids as
`ollama-chat/<model>`, while LiteLLM expects `ollama_chat/`. The window
and output-ceiling lookups now translate that endpoint prefix using the
provider registry before consulting the candidate list. The memo and
no-fetch prefilters use the same registry normalization. The `ollama/`
completion prefix is preserved because LiteLLM keys its offline table
rows under it; non-Ollama prefixes are unchanged.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
Latest review fix (Linux, Python 3.12.13, LiteLLM 1.85.0; proxy
variables unset; consumer tests rechecked after rebase onto current
main):
- New and strengthened regression cases ran before the fix: 6 failed /
108 passed, covering the memo spelling, both no-fetch metadata lookups,
the conservative guard fallback, and an id produced by `stored_model_id`
that could not reach the daemon.
- `uv run --no-sync pytest tests/test_provider_rates.py -q -n 0
--tb=short`: 114 passed after the fix.
- `uv run --no-sync pytest -q --tb=short <consumer-test-files>
--cov=raven.providers.rates --cov-branch
--cov-report=json:<coverage-json>`: 1312 passed, 1 skipped. The
selection contains 16 test modules importing `raven.providers.rates`,
plus `test_provider_registry_lookup.py` and
`test_provider_wire_model.py`. The skip is the permission test that
cannot deny a root process with `chmod 000`; existing Pydantic warnings
remain.
- Applying the repository's `parse_changed_lines` and
`calculate_diff_coverage` to the fix commit after rebase yields 100%
(13/13 executable changed lines). This is coverage of this fix, not a
new full-PR coverage or full-suite ratchet result.
- Six independent in-memory mutants were caught: raw memo prefilter, raw
endpoint guard, raw shared predicate, omitted window spelling
conversion, omitted ceiling spelling conversion, and replacing only the
endpoint conversion with `wire_model` (which loses the `ollama/` table
rows). The worktree was not rewritten by the mutation checks.
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`:
passed after rebase; 2268 Python files formatted, 10 import contracts
kept, no dependency issues. Both commit-message validators and the
source-language and large-file gates also passed. `uv run --no-sync ty
check raven/providers/rates.py tests/test_provider_rates.py` also
passed.
- Against a loopback stand-in daemon, the actual stored
`ollama-chat/qwen2.5` id makes no request in cache-only mode before a
fetch. One allowed `/api/show` request learns 8192, after which the
cache-only lookup and `ModelBinding.context_window` both return 8192
without another request, and the output limit is 4096.
- The daemon-path parameter cases clear LiteLLM's own id-keyed cache so
alias cases cannot inherit a previous case's successful response.
Earlier author verification, before this review fix:
Commands recorded at the original PR revision and main baseline, with
proxy variables unset:
```
uv run --frozen --python 3.12 --all-extras pytest -n 4 -q
```
3 failed, 29796 passed, 158 skipped, 2 xfailed. A same-day run of the
original main control gives the same 3 failed with 29771 passed, so the
failure set is identical and the 25 extra passes are the tests added
here. The three are environment-local:
`test_resolve_node_dir_answers_each_case_it_exists_for` (npm on PATH),
`test_a_legacy_workbook_is_converted_and_read` (LibreOffice), and
`test_what_cannot_be_read_names_nothing` (load-dependent).
Focused runs on the changed files:
- `tests/test_provider_rates.py` 110 passed
- `tests/test_config_self_surface.py` `tests/test_rpc_settings.py`
`tests/test_config_update.py` `tests/test_raven_config_tool.py`
`tests/test_rpc_console.py` 805 passed
- `tests/test_subagent_host_env.py` 30 passed
Gates: `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
`deptry`, `scripts/check_commit_messages.py origin/main..HEAD`,
`scripts/check_source_language.py origin/main..HEAD` and
`scripts/check_large_files.py origin/main..HEAD` all pass. `pre-commit
run --all-files` fails two hooks on three files this branch does not
touch; the same three fail the same way at the base, so they are
pre-existing.
Every new test was checked by reverting the fix: six single-construct
mutants over the two repaired rules, each restored and compared after
running. Two survived the first pass and both were findings rather than
tests to tighten. The guard that refused a zeroed window was unreachable
(the one caller already has a window), so the parameter is required
instead of optional. And the memo originally ranked above LiteLLM's own
table, so a previous ask could have overridden a row a caller could have
read offline; the table now comes first.
Reproduced end to end against a stand-in server on a loopback port: one
`/api/show` request answers 8192, `ollama/qwen2.5` read with
`allow_fetch=False` answers 8192 with no second request, and
`ModelBinding.context_window` on that model answers 8192 where it read
the fallback before.
## Risk
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
No stored config changes shape or meaning. The alias-spelling fix
changes where a write lands when a file carries both spellings of one
key, which is the file that did not load correctly before. The wildcard
fix changes where a write lands when two map entries differ only by
spelling; both are left as written, where the old code silently edited
the neighbour.
The window memo is process-wide and holds context-window and optional
output-limit counts per model, cleared by the same
`reset_openrouter_cache` the catalogue uses. Stored `ollama-chat/` ids
can now make a metadata request to the configured daemon when fetch is
allowed; cache-only callers continue to reuse local data. Other
hyphenated provider prefixes are outside this fix. Roll back by
reverting the merged change; the memo is not persisted.
## Related Issues
#909, #912, #915, #9231 parent e3d0135 commit ea9bb50
13 files changed
Lines changed: 817 additions & 62 deletions
File tree
- raven
- agent/subagent
- backends
- config
- providers
- rpc/methods
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
236 | | - | |
237 | | - | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
238 | 239 | | |
239 | 240 | | |
240 | 241 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1176 | 1176 | | |
1177 | 1177 | | |
1178 | 1178 | | |
1179 | | - | |
1180 | | - | |
1181 | | - | |
1182 | | - | |
1183 | | - | |
1184 | | - | |
| 1179 | + | |
| 1180 | + | |
| 1181 | + | |
| 1182 | + | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
1185 | 1186 | | |
1186 | 1187 | | |
1187 | 1188 | | |
| |||
1222 | 1223 | | |
1223 | 1224 | | |
1224 | 1225 | | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
| 1231 | + | |
| 1232 | + | |
| 1233 | + | |
| 1234 | + | |
| 1235 | + | |
1225 | 1236 | | |
1226 | 1237 | | |
1227 | 1238 | | |
| |||
1336 | 1347 | | |
1337 | 1348 | | |
1338 | 1349 | | |
1339 | | - | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
1340 | 1353 | | |
1341 | 1354 | | |
1342 | 1355 | | |
| |||
1380 | 1393 | | |
1381 | 1394 | | |
1382 | 1395 | | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
1383 | 1407 | | |
1384 | 1408 | | |
1385 | 1409 | | |
1386 | 1410 | | |
1387 | | - | |
| 1411 | + | |
1388 | 1412 | | |
1389 | 1413 | | |
1390 | 1414 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | | - | |
13 | | - | |
14 | | - | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
15 | 37 | | |
| 38 | + | |
| 39 | + | |
16 | 40 | | |
17 | | - | |
18 | | - | |
19 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
20 | 45 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| |||
783 | 784 | | |
784 | 785 | | |
785 | 786 | | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
786 | 828 | | |
787 | 829 | | |
788 | 830 | | |
| |||
1424 | 1466 | | |
1425 | 1467 | | |
1426 | 1468 | | |
| 1469 | + | |
1427 | 1470 | | |
1428 | | - | |
| 1471 | + | |
1429 | 1472 | | |
1430 | 1473 | | |
1431 | | - | |
| 1474 | + | |
1432 | 1475 | | |
1433 | 1476 | | |
1434 | 1477 | | |
| |||
1559 | 1602 | | |
1560 | 1603 | | |
1561 | 1604 | | |
| 1605 | + | |
| 1606 | + | |
| 1607 | + | |
| 1608 | + | |
1562 | 1609 | | |
1563 | 1610 | | |
1564 | 1611 | | |
1565 | 1612 | | |
1566 | 1613 | | |
1567 | | - | |
1568 | | - | |
| 1614 | + | |
| 1615 | + | |
1569 | 1616 | | |
1570 | 1617 | | |
1571 | 1618 | | |
| |||
1574 | 1621 | | |
1575 | 1622 | | |
1576 | 1623 | | |
1577 | | - | |
| 1624 | + | |
1578 | 1625 | | |
1579 | 1626 | | |
1580 | 1627 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
243 | 243 | | |
244 | 244 | | |
245 | 245 | | |
246 | | - | |
| 246 | + | |
247 | 247 | | |
248 | 248 | | |
249 | 249 | | |
| |||
0 commit comments