Skip to content

fix(recall): fold Russian yo spelling in Cyrillic fallback - #1340

Open
mengqing88 wants to merge 2 commits into
mnemosyne-oss:mainfrom
mengqing88:fix/1295-cyrillic-yo-fold
Open

mengqing88 wants to merge 2 commits into
mnemosyne-oss:mainfrom
mengqing88:fix/1295-cyrillic-yo-fold

Conversation

@mengqing88

@mengqing88 mengqing88 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

The Russian fallback can lose a memory before scoring it: a query such as свеклы does not match the four-character LIKE prefix in stored Свёкла, and the scorer also treats е and ё as distinct trigrams.

Apply the same lowercase/ё→е normalization to query prefixes, the SQL candidate predicate and trigram scoring. Stored content and FTS indexes are preserved. Advance enhanced-recall cache generation from 16 to 17 so pre-normalization results are recomputed. The cache regression checks that a generation-16 result is not reused, and that the recomputed result is cached normally.

This addresses the normalization portion of #1295, following the reporter's proposed spelling fold. It intentionally leaves the issue open: query-word mean aggregation, nonzero lexical scores suppressing fallback, lexical gate thresholds and cross-language dense admission retain their existing behavior. The synthetic keyboard query in the discussion still scores (7/9)/6.

Regression coverage includes both spelling directions, uppercase, inflections, working and episodic fallback paths, real linear BeamMemory.recall, preserved stored text, unrelated Cyrillic controls and the existing nonzero-prior behavior. Model downloads and external services are disabled.

Core version, its bundled plugin manifest, README, changelog and generated API documentation are updated as required by CONTRIBUTING. The catalog compatibility floor remains >=4.0.0; the release-contract test now asserts that independent floor instead of incorrectly tying it to every Core iteration.

Validation on Ubuntu/Python 3.12 with TZ=UTC and embeddings disabled:

  • Before the fix: 16 failures / 33 passes in the Cyrillic suite, including empty working/episodic fallback results and zero lexical relevance for the cross-spelling inflections.
  • After the fix: 344 passed / 1 skipped across the Cyrillic, lexical-preparation, BEAM, lexical-gate, enhanced-cache, query-cache, recency and catalog-contract suites.
  • Release metadata, built provider lifecycle, plugin manifests, catalog and CLI version checks: 33 passed.
  • Changed-file Ruff F/RUF022, project-wide fatal lint, generated documentation checks, wheel/sdist builds, twine check and git diff --check pass.

Prepared with assistance from OpenAI Codex.

Summary

Mnemosyne’s Russian lexical fallback now lowercases query and candidate text and folds ё to е for candidate selection and trigram scoring. This helps spelling variants and inflected forms match without changing stored memory text or FTS indexes. The fallback still averages each query word’s best match and uses the existing lexical admission gate.

Enhanced-recall cache generation advances from 16 to 17, so cached results from the previous behavior are not reused. Regression tests cover working and episodic fallback, linear recall, cache invalidation, and preservation of the existing nonzero lexical-score behavior. These tests specify expected behavior; completed test results are not available.

Architecture and compatibility

  • The change affects lexical retrieval in BEAM. It does not change working/episodic memory tiers, consolidation, veracity, sync, or benchmark methodology.
  • Keeping normalization in query and scoring logic, rather than rewriting stored content or FTS indexes, is the right call for data preservation and compatibility.
  • The diff does not add remote processing, telemetry, or new data flows. It does not erode Mnemosyne’s local-first or privacy posture.
  • Release metadata moves to 4.1.0. The Hermes provider manifest version changes, and the catalog compatibility floor remains >=4.0.0. The diff does not change MCP or CLI behavior.
  • README documentation describes the normalization and its limits. Multi-word score dilution and broader admission behavior remain unchanged.

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Russian fallback candidate selection and trigram scoring now fold case and ё to е without changing stored text or lexical admission rules. The enhanced-recall cache generation advances to 17. Package and API documentation versions change to 4.1.0, and the catalog release-contract test uses a fixed Core version floor.

Changes: Russian fallback and 4.1.0 release

Layer / File(s) Summary
Normalize candidate selection and scoring
mnemosyne/core/beam.py
Candidate prefix searches and trigram scoring now apply lowercase and ё-to-е normalization. The enhanced-recall cache generation advances from 16 to 17.
Validate and document fallback behavior
tests/test_cyrillic_fts.py, tests/test_enhanced_recall_cache.py, README.md
Tests cover cross-spelling scores, fallback results, lexical gating, and cache invalidation. The README documents the normalization and states that query-word averaging and the admission gate remain unchanged.
Update release metadata and version floor
tests/test_hermes_core_release_contract.py, mnemosyne/__init__.py, hermes_memory_provider/plugin.yaml, docs/api/configuration.mdx, docs/api/tool-schema.mdx, CHANGELOG.md
Package and API documentation versions change to 4.1.0. The catalog dependency check uses a fixed Core floor of 4.0.0. The changelog records the Russian fallback change.

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix

Possibly related issues

  • mnemosyne-oss/mnemosyne issue 1295 — identifies missing ё-to-е normalization in Cyrillic fallback scoring, which this change adds to candidate selection and scoring.

Possibly related PRs

Merge Risk: Minimal · up to 59a28

No concrete issue remains that should block this release after normal checks.

Pre-merge checks: Passed 4 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Docstring Coverage Inconclusive Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the Russian spelling normalization in Cyrillic fallback.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 4 files. (6 skipped: 5 unsupported, 1 too large.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 45fbebb and 59a28c5.

Files selected for processing: 10

  • CHANGELOG.md
  • README.md
  • docs/api/configuration.mdx
  • docs/api/tool-schema.mdx
  • hermes_memory_provider/plugin.yaml
  • mnemosyne/__init__.py
  • mnemosyne/core/beam.py
  • tests/test_cyrillic_fts.py
  • tests/test_enhanced_recall_cache.py
  • tests/test_hermes_core_release_contract.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

@dplush

dplush commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Reviewed 59a28c5. Candidate selection and trigram scoring consistently lowercase and fold ё to е, including uppercase and reverse-spelling inflections. I verified public linear recall in both working and episodic memory, plus unrelated and other-language controls. The focused offline suite passed 157 tests; the packaging lifecycle test remains locally unverified because pip's isolated build dependency setup could not complete offline.

The implementation keeps the agreed scope: stored content and FTS indexes are preserved, and query-word averaging, lexical thresholds and existing nonzero-score behavior are unchanged. The remaining multi-word dilution work in #1295 stays separate.

Before integration, please refresh against current Main and resolve the cache generation using the actual predecessor plus one. This head uses 17, while Main currently uses 18; retain the graph/embedding changes and update the changelog to the final generation. No additional correctness issue found in the normalization change itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants