Skip to content

docs: repair canonical and source links #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
roli-lpci merged 1 commit into from
Oct 19, 2026
Merged

docs: repair canonical and source links #89

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ The JSONL reviewer and `hermes-gate-claude-review` command are available in the
```bash
fidelis_gate_home="${XDG_DATA_HOME:-$HOME/.local/share}/fidelis-gate-tools"
fidelis_gate_bin="$fidelis_gate_home/bin"
UV_TOOL_DIR="$fidelis_gate_home/tools" UV_TOOL_BIN_DIR="$fidelis_gate_bin" uv tool install --force 'git+https://github.com/hermes-labs-ai/hermes-gate.git@f20db03fa5f63fc8c5bc11773f4c4b71cb2b61a9'
UV_TOOL_DIR="$fidelis_gate_home/tools" UV_TOOL_BIN_DIR="$fidelis_gate_bin" uv tool install --force 'git+https://github.com/roli-lpci/hermes-gate.git@f20db03fa5f63fc8c5bc11773f4c4b71cb2b61a9'
PATH="$fidelis_gate_bin:$PATH" hermes-gate fast
PATH="$fidelis_gate_bin:$PATH" hermes-gate review
```
Expand Down
2 changes: 1 addition & 1 deletion docs/DISPATCHER_DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ Existing users are never silently migrated. Manual opt-in only.
| **cogito 31-case eval** (bench/eval.py) | Atomic paraphrase recall | direct_recall, semantic_gap, adversarial, cross-reference | Exists |
| **LongMemEval_S** | Session multi-turn retrieval | Temporal ordering, multi-hop, counting | Exists (bench/longmemeval_combined_pipeline_flagship.py) |
| **LOCOMO** (add this) | Conversational memory + question grounding | Multi-session with structured events | Not yet — download from https://github.com/snap-research/LOCOMO |
| **MemGPT MemoryBench** (add this) | Agent task memory (tool-use sessions) | Procedural + declarative in one corpus | Not yet — https://github.com/cpacker/MemGPT |
| **MemGPT MemoryBench** (add this) | Agent task memory (tool-use sessions) | Procedural + declarative in one corpus | Not yet — https://github.com/letta-ai/letta |

**Why LOCOMO:** Structured multi-session with explicit temporal scaffolds — validates Path B on a corpus not used to tune it. First out-of-distribution test.

Expand Down
14 changes: 7 additions & 7 deletions docs/ROADMAP_CODEX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,23 @@ What is missing in the design and would bite:
- Insert-time routing is not wired today. `/store` writes directly to one Chroma collection with payload `{"data": text, "user_id": user_id}` and no `mem_type` metadata ([src/cogito/server.py:195-209](src/cogito/server.py:195)). `mem0_config()` only builds one collection name (`cfg["collection"]`) ([src/cogito/config.py:142-167](src/cogito/config.py:142)). `/add` delegates straight to `memory.add()` with no type hook ([src/cogito/server.py:213-223](src/cogito/server.py:213)).
- The proposed default “query both, merge by score” is underspecified. Both paths expose cosine-like scores after stage 1 (`recall_b` rewrites `score` to cosine at [src/cogito/recall_b.py:280-289](src/cogito/recall_b.py:280); hybrid does the same at [src/cogito/recall_hybrid.py:299-316](src/cogito/recall_hybrid.py:299)), but LLM rerank changes order without recomputing score in either path ([src/cogito/recall.py:61-64](src/cogito/recall.py:61), [src/cogito/recall.py:220-227](src/cogito/recall.py:220), [src/cogito/recall_hybrid.py:419-425](src/cogito/recall_hybrid.py:419), [src/cogito/recall_hybrid.py:507-513](src/cogito/recall_hybrid.py:507)). “Merge by score” would therefore be incorrect after reranking.
- Migration is nontrivial because `seed.py` currently assumes atomized fact writes to `/store` ([src/cogito/seed.py:328-349](src/cogito/seed.py:328)); the design does not define how existing curated facts, `/add`-extracted facts, and future long-form writes are distinguished or backfilled.
- The 400-char split is explicitly unmeasured in the design, and the repo contains no production length-distribution measurement yet ([docs/DISPATCHER_DESIGN.md:56-57](docs/DISPATCHER_DESIGN.md:56)).
- The 400-char split is explicitly unmeasured in the design, and the repo contains no production length-distribution measurement yet ([docs/DISPATCHER_DESIGN.md:56-57](DISPATCHER_DESIGN.md#L56-L57)).

@sourcery-ai sourcery-ai Bot Oct 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Low · Markdown line link misses its target

When a reader follows this link in GitHub's rendered Markdown view, gitHub renders DISPATCHER_DESIGN.md as Markdown, where source line numbers are not anchors, so #L56-L57 does not scroll to the cited text and readers land at the document top.

Link to a rendered heading anchor or use a plain-source URL with the line fragment.

Prompt for AI agents 
In `docs/ROADMAP_CODEX.md` at line 12:

**Markdown line link misses its target**

When a reader follows this link in GitHub's rendered Markdown view, gitHub renders `DISPATCHER_DESIGN.md` as Markdown, where source line numbers are not anchors, so `#L56-L57` does not scroll to the cited text and readers land at the document top.

Link to a rendered heading anchor or use a plain-source URL with the line fragment.


## 2. Is the skeptic right about the 21pt regression?

On the regression itself: yes. `bench/eval.py` runs `/recall_hybrid` directly as mode `E` ([bench/eval.py:358-368](bench/eval.py:358)), and the status/docs consistently report 54% vs 75%.
On the regression itself: yes. `bench/eval.py` runs `/recall_hybrid` directly as mode `E` ([bench/eval.py:358-368](../bench/eval.py#L358-L368)), and the status/docs consistently report 54% vs 75%.

On the architect’s diagnosis: only partly proven from code.

- “BM25 hurts short facts” is plausible and code-consistent. The hybrid stage always fuses BM25 runs into RRF when `bm25s` is present ([src/cogito/recall_hybrid.py:264-303](src/cogito/recall_hybrid.py:264)). That is structurally different from `/recall`, which uses multi-query dense recall plus cosine rerank and no BM25 ([src/cogito/recall.py:48-64](src/cogito/recall.py:48), [src/cogito/recall_b.py:313-343](src/cogito/recall_b.py:313)). On a corpus of short fact strings, extra lexical runs can easily perturb rank-1.
- “150-char filter truncation hurts session content” is true as a code fact: `/recall` shows only 150 chars per candidate to the filter ([src/cogito/recall.py:87-91](src/cogito/recall.py:87)). But it does **not** explain the 21pt regression on cogito’s atomic eval, because that eval compares `/recall` against `/recall_hybrid`, and hybrid mostly keeps Stage 1 order for default-route queries at `tier="filter"` ([src/cogito/recall_hybrid.py:607-613](src/cogito/recall_hybrid.py:607)). The 21pt drop is therefore much more likely a Stage 1 mismatch than a filter-snippet issue.
- For session retrieval, the bigger code-grounded mismatch is elsewhere: the 96.4% benchmark (runP-v35) relies on turn chunking and dedup back to sessions ([bench/longmemeval_combined_pipeline_flagship.py:140-188](bench/longmemeval_combined_pipeline_flagship.py:140)), while the production port explicitly omits turn chunking and session-date scaffolds ([src/cogito/recall_hybrid.py:20-24](src/cogito/recall_hybrid.py:20)). That gap is more important than the 150-char fact.
- For session retrieval, the bigger code-grounded mismatch is elsewhere: the 96.4% benchmark (runP-v35) relies on turn chunking and dedup back to sessions ([bench/longmemeval_combined_pipeline_flagship.py:140-188](../bench/longmemeval_combined_pipeline_flagship.py#L140-L188)), while the production port explicitly omits turn chunking and session-date scaffolds ([src/cogito/recall_hybrid.py:20-24](src/cogito/recall_hybrid.py:20)). That gap is more important than the 150-char fact.

## 3. Fork recommendation

Fast-ship is the right fork.

Reason: the codebase today cleanly supports the narrow claim that `/recall_hybrid` is an opt-in LongMemEval-derived path and leaves `/recall` untouched ([src/cogito/server.py:175-193](src/cogito/server.py:175), [CHANGELOG.md:3-15](CHANGELOG.md:3)). It does **not** yet support the broader dispatcher claim because typed writes, migration, dual-index querying, and valid cross-path merge semantics are missing. Shipping the benchmark result as workload-specific is honest; trying to ship workload-aware dispatch now would be architecture-first, evidence-second.
Reason: the codebase today cleanly supports the narrow claim that `/recall_hybrid` is an opt-in LongMemEval-derived path and leaves `/recall` untouched ([src/cogito/server.py:175-193](src/cogito/server.py:175), [CHANGELOG.md:3-15](../CHANGELOG.md#L3-L15)). It does **not** yet support the broader dispatcher claim because typed writes, migration, dual-index querying, and valid cross-path merge semantics are missing. Shipping the benchmark result as workload-specific is honest; trying to ship workload-aware dispatch now would be architecture-first, evidence-second.

@sourcery-ai sourcery-ai Bot Oct 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Changelog links miss hybrid entry

When readers follow either changelog link, following either changelog link lands at lines 3–15, which describe the 0.3.0rc1 release rather than the opt-in hybrid path. The hybrid entry is at lines 272–284, so readers see unrelated notes instead of the cited evidence.

Update both changelog anchors to the current v0.3.0 entry at lines 272–284.

Also at docs/ROADMAP_CODEX.md:69.

Prompt for AI agents 
In `docs/ROADMAP_CODEX.md` at line 28:

**Changelog links miss hybrid entry**

When readers follow either changelog link, following either changelog link lands at lines 3–15, which describe the `0.3.0rc1` release rather than the opt-in hybrid path. The hybrid entry is at lines 272–284, so readers see unrelated notes instead of the cited evidence.

Update both changelog anchors to the current `v0.3.0` entry at lines 272–284.

Also at `docs/ROADMAP_CODEX.md:69`.


## 4. Concrete engineering roadmap

Expand Down Expand Up @@ -66,13 +66,13 @@ Effort: 6-10h.

## 5. What to commit now vs hold

Commit now: the opt-in `v0.3.0` hybrid path as an explicitly workload-specific feature. The server endpoint is isolated, `/recall` is unchanged, and the changelog already describes it that way ([CHANGELOG.md:5-15](CHANGELOG.md:5)).
Commit now: the opt-in `v0.3.0` hybrid path as an explicitly workload-specific feature. The server endpoint is isolated, `/recall` is unchanged, and the changelog already describes it that way ([CHANGELOG.md:5-15](../CHANGELOG.md#L5-L15)).

Hold: any dispatcher, “generalizable” positioning, or default-routing changes. The current repo does not yet implement the storage split or the merge semantics needed to support that claim.

## 6. What the agents missed

- The benchmark/production mismatch is larger than “150-char truncation.” The benchmark’s win depends on turn chunking and session-level dedup ([bench/longmemeval_combined_pipeline_flagship.py:140-188](bench/longmemeval_combined_pipeline_flagship.py:140)); production hybrid explicitly removed those pieces ([src/cogito/recall_hybrid.py:20-24](src/cogito/recall_hybrid.py:20)).
- The benchmark/production mismatch is larger than “150-char truncation.” The benchmark’s win depends on turn chunking and session-level dedup ([bench/longmemeval_combined_pipeline_flagship.py:140-188](../bench/longmemeval_combined_pipeline_flagship.py#L140-L188)); production hybrid explicitly removed those pieces ([src/cogito/recall_hybrid.py:20-24](src/cogito/recall_hybrid.py:20)).
- The dispatcher design assumes score comparability that the code does not preserve after reranking.
- `bench/results-guard-2026-04-17.json` is only 30 questions, not a full verification run, so it should not be used as evidence of the final claim.
- `bench/eval.py` scores positive cases by keyword hit threshold, not exact answer matching ([bench/eval.py:24-33](bench/eval.py:24), [bench/eval.py:180-201](bench/eval.py:180)). That is fine for regression tracking, but it widens uncertainty when arguing about small deltas or cross-workload transfer.
- `bench/eval.py` scores positive cases by keyword hit threshold, not exact answer matching ([bench/eval.py:24-33](../bench/eval.py#L24-L33), [bench/eval.py:180-201](../bench/eval.py#L180-L201)). That is fine for regression tracking, but it widens uncertainty when arguing about small deltas or cross-workload transfer.
Loading