Skip to content

Commit 9835aad

Browse files
fix(docs): pin public benchmark links for package rendering
1 parent f99fe73 commit 9835aad

3 files changed

Lines changed: 8 additions & 5 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/
4949
The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.
5050

5151
<p align="center">
52-
<img src="docs/images/context-efficiency.svg" alt="Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 84a032d3ea23; see the benchmark guide for the full checksum." width="100%">
52+
<img src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/f99fe73cece0f58ebd26ea7784e70ebb9a3926be/docs/images/context-efficiency.svg" alt="Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix 84a032d3ea23; see the benchmark guide for the full checksum." width="100%">
5353
<br>
5454
<sup>Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.</sup>
5555
</p>
@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run
6060
| Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 |
6161
| Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct |
6262

63-
The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations.
63+
The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/f99fe73cece0f58ebd26ea7784e70ebb9a3926be/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations.
6464

6565
## Optional Jev assistance
6666

‎tests/test_documentation_contracts.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414

1515
ROOT = Path(__file__).resolve().parents[1]
16-
README_BENCHMARK_PIN = "522def372c45a26d46bc7a05829964ae0240f3a9"
16+
README_BENCHMARK_PIN = "f99fe73cece0f58ebd26ea7784e70ebb9a3926be"
1717
README_HOSTED_PLANS_PIN = "522def372c45a26d46bc7a05829964ae0240f3a9"
1818

1919

@@ -121,7 +121,10 @@ def test_core_backend_imports_stay_behind_outer_composition_root() -> None:
121121
readme = _read("README.md")
122122
assert "from engraphis.service import MemoryService" in readme
123123
assert "Configuration reference" in readme
124-
assert "[Benchmark methodology](BENCHMARKS.md)" in readme
124+
assert (
125+
f"[Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/{README_BENCHMARK_PIN}/"
126+
"BENCHMARKS.md)" in readme
127+
)
125128

126129

127130
def test_benchmark_text_alternatives_match_registered_fixture_boundary() -> None:

‎tests/test_setup_plugin_distribution.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
ROOT = Path(__file__).resolve().parents[1]
1010
REPOSITORY = "Coding-Dev-Tools/engraphis"
11-
README_BENCHMARK_PIN = "522def372c45a26d46bc7a05829964ae0240f3a9"
11+
README_BENCHMARK_PIN = "f99fe73cece0f58ebd26ea7784e70ebb9a3926be"
1212
README_LINK_PINS = (
1313
"522def372c45a26d46bc7a05829964ae0240f3a9",
1414
"522def372c45a26d46bc7a05829964ae0240f3a9",

0 commit comments

Comments
 (0)