diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b46b71c24..5fa89c626 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -219,6 +219,13 @@ jobs: - name: Run core tests run: pytest tests/ -v --tb=short --timeout=900 --timeout-method=thread + - name: Check published minimum Core compatibility (online) + env: + MNEMOSYNE_TEST_PUBLISHED_CORE: "1" + run: >- + pytest tests/test_hermes_core_release_contract.py::test_published_minimum_core_completes_the_provider_lifecycle + -v --tb=short --timeout=900 --timeout-method=thread + - name: Run Hermes integration tests run: pytest integrations/hermes/tests/ -v --tb=short --timeout=900 --timeout-method=thread diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fe3857e4..499a0c9ba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ and this project adheres to [SemVer](https://semver.org/) starting from v3.1.2. - **Advisory offline linear BEAM throughput benchmark (#901).** A fixed synthetic workload on a temporary database reports warm per-call latency, sequential throughput, raw samples and environment/revision metadata as JSON, with compatible same-machine baseline comparisons and a documented retention/interpretation policy. No runtime recall changes or CI timing threshold. ### Fixed +- **Russian fallback matches optional ё/е spelling (#1295, normalization portion).** Candidate prefixes and trigram scoring now fold case and ё to е, so an inflected query such as `свеклы` can retrieve stored `Свёкла`. The bounded candidate pool gives original-prefix and folded-only matches separate slots, then fills any leftover space with original-prefix matches. Stored text, FTS indexes, query-word mean aggregation and lexical gate thresholds are unchanged. Enhanced-recall cache generation advances to 19, invalidating results from Main generation 18. Multi-word score dilution remains separate work. - **Hermes CLI selects the provider's profile database (#1346).** With `HERMES_HOME` set, packaged `stats`, `inspect`, `doctor`, `export`, `import`, `sleep`, and `clear` use the provider's explicit `/mnemosyne/data` root, including its selected profile bank. Doctor's constructed memory and its initial beam now target the same database. A divergent `MNEMOSYNE_DATA_DIR` still controls standalone Core callers, and the CLI keeps the Core resolver when `HERMES_HOME` is absent. No database migration or merge is performed. - **Fleet census bounds SQLite lock waits for read-only candidate banks (#1336).** Each locked file previously consumed SQLite's five-second default busy timeout. Census reads now wait up to 50 milliseconds per candidate; unreadable files remain in the report and keep the cross-bank bound unmeasured. Discovery and bank contents are unchanged. - **Opt-in input caps now cover local embeddings as well as the API (Refs #1153).** Local document and query calls apply the existing character and UTF-8 byte caps to their prefixed text before fastembed runs. Both caps remain disabled by default, and query-cache entries remain keyed by effective cap and prefix. Enhanced recall snapshots both effective input caps for its cache key and embedding calls, with generation 18 invalidating results from the current Main predecessor, generation 17. Changing caps does not rebuild stored vectors; reindex when applying the new policy to existing documents. This limits input size only; it does not establish an ONNX memory ceiling or close the reported arena issue. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e425c5f08..5d8dbc294 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,6 +20,17 @@ MNEMOSYNE_NO_EMBEDDINGS=1 python -m pytest integrations/hermes/tests/ -v The Hermes provider is packaged separately from `mnemosyne-memory`, so its editable install is required before running `integrations/hermes/tests/`. +The published minimum-Core compatibility test is an explicit online check: +it downloads the pinned Core wheel from PyPI, verifies its SHA256, builds the +current Hermes wheel, and exercises the provider lifecycle. It is skipped in +normal test runs and runs in a separate CI step on every supported Python version. +To run it locally: + +```bash +MNEMOSYNE_NO_EMBEDDINGS=1 MNEMOSYNE_TEST_PUBLISHED_CORE=1 python -m pytest \ + tests/test_hermes_core_release_contract.py::test_published_minimum_core_completes_the_provider_lifecycle -v +``` + ## What You Can Do **No contribution is too small.** diff --git a/README.md b/README.md index 735a0bda0..749364ac7 100644 --- a/README.md +++ b/README.md @@ -343,6 +343,12 @@ Full reference: [docs/configuration.md](docs/configuration.md) ### Language Support +The Russian lexical fallback folds case and `ё`/`е` spelling when selecting +candidates and scoring inflections (for example, `свеклы` can match `Свёкла`). +Stored text is preserved. The fallback still averages scores over query words +and uses the normal lexical admission gate; this does not guarantee admission +for multi-word paraphrases or cross-language queries. + Default embeddings are English-optimized (`bge-small-en-v1.5`). For **non-English or multilingual** recall, swap the model: ```bash diff --git a/docs/api/configuration.mdx b/docs/api/configuration.mdx index 101eae6c7..374b04a90 100644 --- a/docs/api/configuration.mdx +++ b/docs/api/configuration.mdx @@ -1,6 +1,6 @@ --- title: "Configuration" -version: 4.0.0 +version: 4.1.0 config_key_count: 117 env_only_count: 52 generated_at: "auto" diff --git a/docs/api/tool-schema.mdx b/docs/api/tool-schema.mdx index 490c64a48..682977ac7 100644 --- a/docs/api/tool-schema.mdx +++ b/docs/api/tool-schema.mdx @@ -1,13 +1,13 @@ --- title: "MCP Tool Schema" -version: 4.0.0 +version: 4.1.0 tool_count: 38 mcp_tool_count: 30 plugin_only_tool_count: 8 generated_at: "auto" --- -# MCP Tool Schema (v4.0.0) +# MCP Tool Schema (v4.1.0) > **Generated file. Do not edit by hand.** > Source: `mnemosyne/tool_schemas.py` and `mnemosyne/mcp_tools.py`. diff --git a/hermes_memory_provider/plugin.yaml b/hermes_memory_provider/plugin.yaml index 909624c98..36523bd32 100644 --- a/hermes_memory_provider/plugin.yaml +++ b/hermes_memory_provider/plugin.yaml @@ -1,5 +1,5 @@ name: mnemosyne -version: 4.0.0 +version: 4.1.0 description: "Native local memory for Hermes — SQLite with vector search, FTS5 hybrid ranking, episodic consolidation, temporal triples, and bidirectional sync with optional client-side encryption. Zero cloud. Zero latency." author: Abdias J provides_tools: diff --git a/mnemosyne/__init__.py b/mnemosyne/__init__.py index 558d21bd1..35a627b70 100644 --- a/mnemosyne/__init__.py +++ b/mnemosyne/__init__.py @@ -10,7 +10,7 @@ >>> results = recall("user preferences") """ -__version__ = "4.0.0" +__version__ = "4.1.0" __author__ = "Abdias J" __license__ = "MIT" diff --git a/mnemosyne/core/beam.py b/mnemosyne/core/beam.py index 6a720ab29..bf102a632 100644 --- a/mnemosyne/core/beam.py +++ b/mnemosyne/core/beam.py @@ -5598,23 +5598,29 @@ def _ngrams(s: str, n: int = 3) -> set: return {s[i:i + n] for i in range(len(s) - n + 1)} +def _normalize_cyrillic_text(text: str) -> str: + """Fold case and optional Russian ё spelling for fallback matching only.""" + return text.lower().replace("ё", "е") + + def _cyrillic_score(query: str, content: str, n: int = 3) -> float: """Trigram Jaccard score in [0, 1] for Russian/Cyrillic recall. For each query word, find the best-matching content word by Jaccard similarity of their character n-gram sets. Return the mean across query words. Words shorter than 3 characters are ignored to avoid - noisy matches on stop words and punctuation. + noisy matches on stop words and punctuation. Case and ё/е spelling + are folded before extracting n-grams; stored content is unchanged. N-gram sets are computed once per unique word and cached in a dict to avoid redundant recomputation when the same content word is compared against multiple query words. """ q_words = [ - w for w in re.findall(r"[а-яёa-z0-9]+", query.lower()) if len(w) >= 3 + w for w in re.findall(r"[а-яёa-z0-9]+", _normalize_cyrillic_text(query)) if len(w) >= 3 ] c_words = [ - w for w in re.findall(r"[а-яёa-z0-9]+", content.lower()) if len(w) >= 3 + w for w in re.findall(r"[а-яёa-z0-9]+", _normalize_cyrillic_text(content)) if len(w) >= 3 ] if not q_words or not c_words: return 0.0 @@ -5652,8 +5658,10 @@ def _cyrillic_like_search( receive comparable scores regardless of which surface form appears in the stored text. - The candidate-generation LIMIT is ``k * 5`` to keep the Python-side - scoring bounded; this matches the CJK fallback's behaviour. + Python scoring is bounded to ``k * 5`` candidates. Original-prefix + matches get half the pool first, and folded-only matches get the + remaining slots. Extra original-prefix matches fill any space left. + Both groups use a stable identifier order. """ if not _has_cyrillic(query): return [] @@ -5664,10 +5672,17 @@ def _cyrillic_like_search( # SQLite's default LOWER() and LIKE are ASCII-only, so they cannot # case-fold Cyrillic letters (Т ≠ т, Ё ≠ ё). Register a Python - # UDF on the connection that performs a real Unicode lower(). It - # is safe to register the same name multiple times — SQLite will - # overwrite the previous function with the same name and arity. - conn.create_function("_py_lower", 1, lambda s: s.lower() if isinstance(s, str) else s) + # UDF for Unicode case-folding and ё/е normalization, matching the + # scorer. Without the spelling fold here, candidates are lost before + # re-ranking. Re-registering the same name and arity is safe. + conn.create_function( + "_py_cyrillic_normalize", 1, + lambda s: _normalize_cyrillic_text(s) if isinstance(s, str) else s, + ) + conn.create_function( + "_py_cyrillic_lower", 1, + lambda s: s.lower() if isinstance(s, str) else s, + ) # Use a 4-character prefix for LIKE matching. Query words and # stored content are inflected forms of the same lemma (тёмная @@ -5675,23 +5690,42 @@ def _cyrillic_like_search( # over-selector that still prunes the candidate set down to a # trigram-scoring-sized slice. q_words = [ - w for w in re.findall(r"[а-яёa-z0-9]+", query.lower()) if len(w) >= 4 + w for w in re.findall(r"[а-яёa-z0-9]+", _normalize_cyrillic_text(query)) if len(w) >= 4 ] if not q_words: return [] - conditions = " OR ".join( - ["_py_lower(content) LIKE ? ESCAPE '\\'"] * len(q_words) + original_words = [ + w for w in re.findall(r"[а-яёa-z0-9]+", query.lower()) if len(w) >= 4 + ] + budget = k * 5 + if budget <= 0: + return [] + original_conditions = " OR ".join( + ["_py_cyrillic_lower(content) LIKE ? ESCAPE '\\'"] * len(original_words) + ) + folded_conditions = " OR ".join( + ["_py_cyrillic_normalize(content) LIKE ? ESCAPE '\\'"] * len(q_words) ) - params = [f"%{w[:4].lower()}%" for w in q_words] + original_params = [f"%{w[:4]}%" for w in original_words] + folded_params = [f"%{w[:4]}%" for w in q_words] try: - all_rows = conn.execute( - f"SELECT {id_col}, content FROM {table} WHERE {conditions} LIMIT ?", - params + [k * 5], + original_rows = conn.execute( + f"SELECT {id_col}, content FROM {table} WHERE {original_conditions}" + f" ORDER BY {id_col} LIMIT ?", + original_params + [budget], + ).fetchall() + original_count = min(len(original_rows), budget // 2) + variant_rows = conn.execute( + f"SELECT {id_col}, content FROM {table} WHERE ({folded_conditions})" + f" AND NOT ({original_conditions}) ORDER BY {id_col} LIMIT ?", + folded_params + original_params + [budget - original_count], ).fetchall() except Exception: return [] + remaining = budget - len(variant_rows) + all_rows = original_rows[:original_count] + variant_rows + original_rows[original_count:remaining] if not all_rows: return [] @@ -11240,8 +11274,9 @@ def _get_query_embedding(): # 16: clamp future-dated recency ages before exponentiation. # 17: strongest eligible graph traversal and scoped active-seed selection. # 18: local embedding input caps and effective cap policy in cache keys. + # 19: Cyrillic ё/е folding in fallback candidates and scores. # Recheck the actual predecessor before each semantic integration. - _ENHANCED_RECALL_CACHE_VERSION = 18 + _ENHANCED_RECALL_CACHE_VERSION = 19 # NOTE: the key carries the env MNEMOSYNE_VEC_TYPE, not the table's # live DDL type — a reindex under a different type without a version # bump serves stale admission/ranking. Reindex flows must bump. diff --git a/tests/test_cyrillic_fts.py b/tests/test_cyrillic_fts.py index 146402a7e..c09ce2b9d 100644 --- a/tests/test_cyrillic_fts.py +++ b/tests/test_cyrillic_fts.py @@ -31,6 +31,7 @@ from mnemosyne.core import beam as beam_module from mnemosyne.core.beam import ( + BeamMemory, init_beam, _has_cyrillic, _ngrams, @@ -125,16 +126,32 @@ def test_unrelated_words_score_low(self): # "тёмная" vs "встреча" share no trigrams at all -> 0 assert _cyrillic_score("тёмная", "встреча") == 0.0 - def test_yo_vs_e_collapses(self): - # We deliberately do NOT collapse ё -> е; the regex preserves - # the user's exact spelling. Trigrams differ, so the score is - # the same as for any unrelated substitution. - s_with_yo = _cyrillic_score("ёжик", "ёжики") - s_with_e = _cyrillic_score("ежик", "ежики") - # Both should be the same since the logic is symmetric - assert s_with_yo == pytest.approx(s_with_e) - # And both should be reasonably high - assert s_with_yo > 0.3 + @pytest.mark.parametrize("query,content,expected", [ + ("ежик", "ёжик", 1.0), + ("ЁЖИК", "ежик", 1.0), + ("свеклы", "свёкла", 0.6), + ("СВЁКЛЫ", "свекла", 0.6), + ("тёмная", "темную", 1 / 3), + ]) + def test_yo_vs_e_collapses(self, query, content, expected): + # Cross-spelling pairs must work, including uppercase and inflection. + assert _cyrillic_score(query, content) == pytest.approx(expected) + + def test_yo_normalization_preserves_query_word_mean(self): + # Folding spelling does not make unrelated query words disappear from + # the denominator or change the aggregation policy discussed in #1295. + assert _cyrillic_score("большая свеклы", "свёкла") == pytest.approx(0.3) + assert _cyrillic_score( + "какая у него любимая клавиатура и буквы на ней", + "он рассказал про механическую клавиатуру с подсветкой вчера", + ) == pytest.approx((7 / 9) / 6) + + @pytest.mark.parametrize("query,content", [ + ("жёлтый", "автобус"), + ("автобус", "ёлочка"), + ]) + def test_yo_normalization_does_not_admit_unrelated_words(self, query, content): + assert _cyrillic_score(query, content) == 0.0 def test_short_query_words_ignored(self): # 1-2 char words are skipped to avoid stop-word noise @@ -222,6 +239,103 @@ def _open(db_path: Path) -> sqlite3.Connection: return conn +@pytest.mark.parametrize("working", [True, False]) +@pytest.mark.parametrize("query,content", [ + ("свеклы", "Свёкла тушёная хранится отдельно"), + ("свёклы", "Свекла тушеная хранится отдельно"), + ("СВЕКЛЫ", "СВЁКЛА ТУШЁНАЯ ХРАНИТСЯ ОТДЕЛЬНО"), + ("СВЁКЛЫ", "СВЕКЛА ТУШЕНАЯ ХРАНИТСЯ ОТДЕЛЬНО"), +]) +def test_yo_spelling_fallback_reaches_both_memory_tiers(temp_db, working, query, content): + """Both fallback entry points admit folded inflections and retain stored text.""" + seed = _seed_working if working else _seed_episodic + seed(temp_db, [ + ("target", content, 0.8), + ("unrelated", "Автобус отправляется утром", 0.8), + ]) + conn = _open(temp_db) + try: + # No exact FTS match: the query and stored word have different endings. + fts_table = "fts_working" if working else "fts_episodes" + assert conn.execute( + f"SELECT count(*) FROM {fts_table} WHERE {fts_table} MATCH ?", (query,) + ).fetchone()[0] == 0 + direct = _cyrillic_like_search(conn, query, k=5, working=working) + routed = (_fts_search_working if working else _fts_search)(conn, query, k=5) + table = "working_memory" if working else "episodic_memory" + assert conn.execute( + f"SELECT content FROM {table} WHERE id = ?", ("target",) + ).fetchone()[0] == content + finally: + conn.close() + result_key, expected_id = ("id", "target") if working else ("rowid", 1) + for results in (direct, routed): + assert [row[result_key] for row in results] == [expected_id] + assert results[0]["rank"] == pytest.approx(-0.6) + + +@pytest.mark.parametrize("query,content", [ + ("свеклы", "Свёкла тушёная хранится отдельно"), + ("свёклы", "Свекла тушеная хранится отдельно"), +]) +def test_yo_spelling_passes_linear_recall_gate(temp_db, monkeypatch, query, content): + """Real linear recall admits either ё/е spelling through the lexical gate.""" + monkeypatch.setenv("MNEMOSYNE_POLYPHONIC_RECALL", "0") + monkeypatch.setenv("MNEMOSYNE_NO_EMBEDDINGS", "1") + monkeypatch.delenv("MNEMOSYNE_LEXICAL_GATE_MIN", raising=False) + memory = BeamMemory(session_id="yo-normalization", db_path=temp_db) + target = memory.remember(content, source="test", importance=0.8) + memory.remember("Автобус отправляется утром", source="test", importance=0.8) + relevance = _lexical_relevance(_recall_tokens(query), content, query) + assert relevance == pytest.approx(0.6) + results = memory.recall(query, top_k=5) + assert [row["id"] for row in results] == [target] + + +@pytest.mark.parametrize("working", [True, False]) +@pytest.mark.parametrize("k", [1, 5]) +@pytest.mark.parametrize("query,content,noise", [ + ("свёклы", "Свёкла тушёная хранится отдельно", "Свекор приехал утром"), + ("свеклы", "Свекла тушеная хранится отдельно", "Свёкор приехал утром"), + ("СВЁКЛЫ", "СВЁКЛА ТУШЁНАЯ ХРАНИТСЯ ОТДЕЛЬНО", "СВЕКОР ПРИЕХАЛ УТРОМ"), + ("свёклы", "Свекла тушеная хранится отдельно", "Свёкор приехал утром"), + ("свеклы", "Свёкла тушёная хранится отдельно", "Свекор приехал утром"), +]) +def test_spelling_fold_keeps_both_candidate_groups_under_pressure( + temp_db, monkeypatch, working, k, query, content, noise, +): + """Either spelling can survive a full candidate pool of the other spelling.""" + seed = _seed_working if working else _seed_episodic + competitors = [(f"noise-{i:03}", f"{noise} {i}", 0.8) for i in range(k * 5 + 6)] + unrelated = "Автобус отправляется вечером" + seed(temp_db, [("aa-unrelated", unrelated, 0.8)] + competitors + [("zz-target", content, 0.8)]) + conn = _open(temp_db) + scorer = beam_module._cyrillic_score + scored = [] + + def track_score(query, content): + """Count candidates handed to the expensive Python scorer.""" + scored.append(content) + return scorer(query, content) + + monkeypatch.setattr(beam_module, "_cyrillic_score", track_score) + expected_id = "zz-target" if working else len(competitors) + 2 + result_key = "id" if working else "rowid" + try: + for search in ( + lambda: _cyrillic_like_search(conn, query, k=k, working=working), + lambda: (_fts_search_working if working else _fts_search)(conn, query, k=k), + ): + scored.clear() + results = search() + assert results[0][result_key] == expected_id + assert results[0]["rank"] == pytest.approx(-0.6) + assert len(scored) == len(set(scored)) == k * 5 + assert unrelated not in scored + finally: + conn.close() + + class TestCyrillicLikeSearchWorking: """Verify _cyrillic_like_search against working_memory.""" @@ -365,6 +479,16 @@ class TestLexicalRelevanceCyrillicGate: results for the very queries Cyrillic users actually type. """ + def test_nonzero_prior_score_still_suppresses_fallback(self, monkeypatch): + def unexpected_fallback(*args): + pytest.fail("a nonzero lexical score must not invoke Cyrillic fallback") + + monkeypatch.setattr(beam_module, "_cyrillic_score", unexpected_fallback) + query = "рецепт блюдо кухня свеклы" + assert _lexical_relevance( + _recall_tokens(query), "кухня свёкла", query + ) == pytest.approx(0.25) + def test_inflected_query_passes_gate(self): # Nominative "тёмная тема" vs accusative "тёмную тему": # exact match returns 0.0; Cyrillic fallback returns trigram diff --git a/tests/test_enhanced_recall_cache.py b/tests/test_enhanced_recall_cache.py index 8da8993ce..23fdbdcca 100644 --- a/tests/test_enhanced_recall_cache.py +++ b/tests/test_enhanced_recall_cache.py @@ -130,11 +130,11 @@ def execute_without_user_version(conn, sql, parameters=(), *args, **kwargs): ) -@pytest.mark.parametrize("previous_version", [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]) +@pytest.mark.parametrize("previous_version", [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]) def test_cache_version_bump_invalidates_prior_candidate_and_cap_entries( enhanced, monkeypatch, tmp_path: Path, previous_version: int ): - """Every prior cache generation misses under the content-cap contract. + """Every prior cache generation misses under the current recall contract. Version 6 predates staged FTS candidate selection, version 7 predates vector admission/ranking, version 8 predates the cap as cache material, @@ -146,6 +146,7 @@ def test_cache_version_bump_invalidates_prior_candidate_and_cap_entries( linear candidate ordering. Version 15 predates the recency-decay age clamp. Version 16 predates strongest-path and scoped active-seed selection. Version 17 predates local input caps and their effective cache policy. + Version 18 predates Cyrillic ё/е normalization. The ``v2:`` key prefix stays fixed for QueryCache's opaque-key path. """ @@ -1097,6 +1098,100 @@ def test_pipeline_flags_and_process_ranking_configuration_change_digest(enhanced assert memory._enhanced_recall_cache_key(**common) != baseline +@pytest.mark.parametrize("restart", [False, True]) +def test_cyrillic_normalization_rejects_persisted_main_generation_18( + enhanced, monkeypatch, restart +): + """Main's prior generation misses after integration, including a Beam restart.""" + memory, calls = enhanced + query = "свеклы" + with monkeypatch.context() as ctx: + ctx.setattr(type(memory), "_ENHANCED_RECALL_CACHE_VERSION", 18) + stale = _call(memory, query) + keys = [row[0] for row in memory._query_cache._conn.execute( + "SELECT normalized FROM query_cache" + )] + assert len(keys) == 1 + target = memory + try: + if restart: + _close_memory(memory) + target = BeamMemory(session_id=memory.session_id, db_path=memory.db_path) + target.recall = memory.recall + target._query_cache = QueryCache(db_path=memory.db_path.parent / "query_cache.db") + assert target._query_cache.get_opaque(keys[0])[0]["id"] == stale[0]["id"] + fresh = _call(target, query) + assert len(calls) == 2 + assert fresh[0]["id"] != stale[0]["id"] + assert _call(target, query) == fresh + assert len(calls) == 2 + finally: + if target is not memory: + _close_memory(target) + + +@pytest.mark.parametrize("restart", [False, True]) +@pytest.mark.parametrize("store", ["working", "episodic"]) +@pytest.mark.parametrize("query,content", [ + ("свеклы", "Свёкла тушёная хранится отдельно"), + ("свёклы", "Свекла тушеная хранится отдельно"), + ("СВЁКЛЫ", "Свекла тушеная хранится отдельно"), +]) +def test_real_cyrillic_recall_replaces_stale_generation_18( + monkeypatch, tmp_path, restart, store, query, content +): + """A persisted pre-fold miss is recomputed into a real match and cached.""" + monkeypatch.setenv("MNEMOSYNE_ENHANCED_RECALL", "1") + monkeypatch.setenv("MNEMOSYNE_POLYPHONIC_RECALL", "0") + monkeypatch.setenv("MNEMOSYNE_NO_EMBEDDINGS", "1") + monkeypatch.delenv("MNEMOSYNE_LEXICAL_GATE_MIN", raising=False) + memory = BeamMemory(session_id="cyrillic-cache", db_path=tmp_path / "memories.db") + target = memory + try: + if store == "working": + wanted = memory.remember(content, source="test", importance=0.8) + unrelated = memory.remember("Автобус отправляется утром", source="test") + else: + wanted = memory.consolidate_to_episodic(content, source_wm_ids=[], source="test") + unrelated = memory.consolidate_to_episodic( + "Автобус отправляется утром", source_wm_ids=[], source="test" + ) + # Reproduce the actual pre-fold result, rather than faking recall. + assert beam_module._cyrillic_score(query, content) == pytest.approx(0.6) + with monkeypatch.context() as ctx: + ctx.setattr(type(memory), "_ENHANCED_RECALL_CACHE_VERSION", 18) + ctx.setattr(beam_module, "_normalize_cyrillic_text", lambda text: text.lower()) + assert beam_module._cyrillic_score(query, content) == pytest.approx(0.0) + stale = _call(memory, query) + assert stale == [] + keys = [row[0] for row in memory._query_cache._conn.execute( + "SELECT normalized FROM query_cache" + )] + assert len(keys) == 1 + if restart: + _close_memory(memory) + target = BeamMemory(session_id=memory.session_id, db_path=memory.db_path) + target._query_cache = QueryCache(db_path=memory.db_path.parent / "query_cache.db") + assert target._query_cache.get_opaque(keys[0]) == [] + misses = target._query_cache.stats()["misses"] + fresh = _call(target, query) + assert [row["id"] for row in fresh] == [wanted] + assert fresh[0]["content"] == content + assert unrelated not in {row["id"] for row in fresh} + assert target._query_cache.stats()["misses"] == misses + 1 + hits = target._query_cache.stats()["hits"] + assert _call(target, query) == fresh + assert target._query_cache.stats()["hits"] == hits + 1 + table = "working_memory" if store == "working" else "episodic_memory" + assert target.conn.execute( + f"SELECT content FROM {table} WHERE id=?", (wanted,) + ).fetchone()[0] == content + finally: + if target is not memory: + _close_memory(target) + _close_memory(memory) + + def test_enhanced_cache_key_version_bump_invalidates_literal_flag_ranking_change( enhanced, monkeypatch ): diff --git a/tests/test_hermes_core_release_contract.py b/tests/test_hermes_core_release_contract.py index 68e009803..d48437d85 100644 --- a/tests/test_hermes_core_release_contract.py +++ b/tests/test_hermes_core_release_contract.py @@ -1,18 +1,21 @@ from __future__ import annotations import ast +import hashlib import importlib import importlib.util import os import subprocess import sys import textwrap +import urllib.request import zipfile from pathlib import Path from packaging.requirements import Requirement from packaging.utils import canonicalize_name from packaging.version import Version +import pytest import yaml try: @@ -25,6 +28,9 @@ STANDALONE_PROJECT = ROOT / "integrations" / "hermes" / "pyproject.toml" CATALOG_PROJECT = ROOT / "integrations" / "hermes-catalog" / "pyproject.toml" STANDALONE_CORE_FLOOR = "4.0.0" +# Catalog promotion chooses a compatibility floor independently of Core's +# current source version; a compatible Core iteration must not raise it. +CATALOG_CORE_FLOOR = "4.0.0" LEGACY_HERMES_MANIFEST = ROOT / "integrations" / "hermes" / "plugin.yaml" CORE_HERMES_MANIFEST = ROOT / "hermes_memory_provider" / "plugin.yaml" REQUIRED_CORE_API = ( @@ -85,7 +91,7 @@ def test_hermes_packages_require_their_supported_core_api_release() -> None: for project, floor, groups in ( (STANDALONE_PROJECT, STANDALONE_CORE_FLOOR, {"base": "embeddings", "llm": "llm", "mcp": "mcp", "all": "all"}), - (CATALOG_PROJECT, core_version, {"base": "embeddings"}), + (CATALOG_PROJECT, CATALOG_CORE_FLOOR, {"base": "embeddings"}), ): requirements = _mnemosyne_memory_specs(project) for group, extra in groups.items(): @@ -163,6 +169,38 @@ def test_built_release_pair_completes_the_provider_lifecycle(tmp_path: Path) -> wheel_dir.mkdir() core_wheel = _build_wheel(ROOT, wheel_dir) hermes_wheel = _build_wheel(ROOT / "integrations" / "hermes", wheel_dir) + _assert_provider_lifecycle(core_wheel, hermes_wheel, tmp_path, _core_version()) + + +@pytest.mark.skipif( + os.environ.get("MNEMOSYNE_TEST_PUBLISHED_CORE") != "1", + reason="Online release check; set MNEMOSYNE_TEST_PUBLISHED_CORE=1 to download the pinned wheel", +) +def test_published_minimum_core_completes_the_provider_lifecycle(tmp_path: Path) -> None: + """The current provider must also run with the declared published Core floor.""" + wheel_dir = tmp_path / "wheels" + wheel_dir.mkdir() + core_wheel = wheel_dir / "mnemosyne_memory-4.0.0-py3-none-any.whl" + # Published PyPI 4.0.0 artifact, not a relabelled build of current source. + url = ( + "https://files.pythonhosted.org/packages/af/d3/" + "ebab3bdf259c8d6f1b572552356751e5f224e2bce4494040611708c3657d/" + "mnemosyne_memory-4.0.0-py3-none-any.whl" + ) + with urllib.request.urlopen(url, timeout=30) as response: + payload = response.read() + assert hashlib.sha256(payload).hexdigest() == ( + "3e80bff63921ab554d3dbc1716900e064c31744b7e60135d9975f73a73026a76" + ), "Published minimum-Core wheel checksum differs" + core_wheel.write_bytes(payload) + hermes_wheel = _build_wheel(ROOT / "integrations" / "hermes", wheel_dir) + _assert_provider_lifecycle(core_wheel, hermes_wheel, tmp_path, CATALOG_CORE_FLOOR) + + +def _assert_provider_lifecycle( + core_wheel: Path, hermes_wheel: Path, tmp_path: Path, expected_core_version: str +) -> None: + """Exercise wheel-only imports, persistence and provider lifecycle transitions.""" site = tmp_path / "site" site.mkdir() _extract_wheel(core_wheel, site) @@ -182,6 +220,7 @@ def test_built_release_pair_completes_the_provider_lifecycle(tmp_path: Path) -> sys.path.insert(1, {str(dependency_site)!r}) import mnemosyne import mnemosyne_hermes + assert mnemosyne.__version__ == {expected_core_version!r}, mnemosyne.__version__ for module in (mnemosyne, mnemosyne_hermes): assert Path(module.__file__).resolve().is_relative_to(site), module.__file__ required_core_api = {REQUIRED_CORE_API!r} @@ -206,6 +245,24 @@ def call(provider, name, args): assert call(provider, "mnemosyne_remember", {{"content": token}})["status"] == "stored" recalled = call(provider, "mnemosyne_recall", {{"query": token, "limit": 5}}) assert any(token in row.get("content", "") for row in recalled["results"]) + recovery_token = "release_contract_persistent_marker" + stored = call(provider, "mnemosyne_remember", {{ + "content": recovery_token, "scope": "global", + }}) + assert stored["status"] == "stored" + private_token = "release_contract_session_marker" + private = call(provider, "mnemosyne_remember", {{ + "content": private_token, "scope": "session", + }}) + assert private["status"] == "stored" + batch_token = "release_contract_batch_marker" + batch = call(provider, "mnemosyne_batch", {{"operations": [{{ + "action": "remember", "content": batch_token, "scope": "global", + }}]}}) + assert batch["status"] == "ok", batch + assert batch["operations_count"] == 1, batch + batch_recalled = call(provider, "mnemosyne_recall", {{"query": batch_token, "limit": 5}}) + assert any(batch_token in row.get("content", "") for row in batch_recalled["results"]) provider.initialize( "release-pair-skip", agent_context="subagent", @@ -223,11 +280,27 @@ def call(provider, name, args): ) assert provider._beam is not None, provider._init_error assert call(provider, "mnemosyne_stats", {{}}).get("status") != "memory_unavailable" + recovered = call(provider, "mnemosyne_recall", {{"query": recovery_token, "limit": 5}}) + assert any(recovery_token in row.get("content", "") for row in recovered["results"]) + recovered_batch = call(provider, "mnemosyne_recall", {{"query": batch_token, "limit": 5}}) + assert any(batch_token in row.get("content", "") for row in recovered_batch["results"]) + isolated = call(provider, "mnemosyne_recall", {{"query": private_token, "limit": 5}}) + assert not any(private_token in row.get("content", "") for row in isolated["results"]) + provider.initialize( + "release-pair-primary", + agent_context="primary", + hermes_home={str(home)!r}, + auto_sleep=False, + ) + assert provider._beam is not None, provider._init_error + original = call(provider, "mnemosyne_recall", {{"query": private_token, "limit": 5}}) + assert any(private_token in row.get("content", "") for row in original["results"]) provider.shutdown() """ ) env = os.environ.copy() env["MNEMOSYNE_NO_EMBEDDINGS"] = "1" + env["MNEMOSYNE_CROSS_SESSION"] = "0" env.pop("PYTHONPATH", None) result = subprocess.run( [sys.executable, "-S", "-c", code],