Skip to content

Commit 1ce23c0

Browse files
fix(review): harden release and maintenance determinism
1 parent 484da71 commit 1ce23c0

18 files changed

Lines changed: 412 additions & 36 deletions

‎BENCHMARKS.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,14 @@ For the locked operator sequence for a public canonical run, see
99
### Public numeric evidence registry
1010

1111
Every exact public aggregate retained below comes from the checked-in, public-safe
12-
[`offline-fixtures-v4.json`](docs/benchmark-evidence/offline-fixtures-v4.json) artifact. Its
12+
[`offline-fixtures-v8.json`](docs/benchmark-evidence/offline-fixtures-v8.json) artifact. Its
1313
SHA-256 is
14-
`530c2620297de6fadd5e8256fe33c7fb15f6d53e3d1353569e56ddb77532e3cf`, also recorded in the
14+
`f48f4aa3b4bde802c1b2f008820ee1fc45faee1901e876e0725a4b77964e44c2`, also recorded in the
1515
adjacent `.sha256` file. The artifact contains no raw questions, answers, prompts, customer data,
1616
or per-record content fingerprints.
1717

1818
The fixture-suite digest is
19-
`06bfa81307dac71ef82ac4c30c782426498ab8ff4699a990d22720d30fc2e563`. The artifact defines
19+
`34312952681809212ada5fd5f1fa608ff8723491d770c8482fd1b0a9e8e87942`. The artifact defines
2020
the digest algorithm and records the SHA-256 of every suite and dataset file. Each evidence ID
2121
also binds its exact command through `sha256(UTF-8 exact command)`:
2222

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,9 +77,9 @@ its counting boundary explicit.
7777
| Packed prompt-context usage in the same 26-question CodeMem sample pass | Hard budget: **1,500** tokens; observed mean: **85.38**; observed maximum: **108** | A hard cap prevents a recall from exceeding its configured context budget | This is usage accounting, not a before/after savings comparison |
7878

7979
These values are evidence IDs `offline-chunking` and `offline-performance` in
80-
[`offline-fixtures-v4.json`](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/benchmark-evidence/offline-fixtures-v4.json),
80+
[`offline-fixtures-v8.json`](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/benchmark-evidence/offline-fixtures-v8.json),
8181
SHA-256
82-
`530c2620297de6fadd5e8256fe33c7fb15f6d53e3d1353569e56ddb77532e3cf`.
82+
`f48f4aa3b4bde802c1b2f008820ee1fc45faee1901e876e0725a4b77964e44c2`.
8383
[`BENCHMARKS.md`](https://github.com/Coding-Dev-Tools/engraphis/blob/main/BENCHMARKS.md#public-numeric-evidence-registry)
8484
records the matching suite digest, exact commands, and per-command config digests. External,
8585
model-dependent, consolidation, productivity, and latency results remain unpublished until the

‎docs/REWORK_EXECUTION.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ scope/time regression coverage. Original user edits were not rewritten.
161161
| Capacity acceptance | Lifecycle RSS includes startup, backlog is sampled and recomputed, and the complete matrix validator enforces prebound hosts, WAL/FULL, all scheduled outcomes, RAM and the required 100k latency limits. | No primary 48-cell matrix was executed. The separate 16 GiB reference host remains necessary. |
162162
| Installed journeys | A packaged stdlib runner performs actual MCP/HTTP writes, restart recall, correction and historical reads. PR CI and release jobs cover Windows, macOS and Linux, with artifact/dependency identities retained. | Cached Windows source semantic startup passed in four fresh processes, taking 20-23 seconds. This is not semantic qualification of all installed platforms. |
163163
| Evidence and publication | Candidate ledger validation checks identities, hashes, dependencies, outcomes and selected evaluation booleans. All four publication/repair writes require [owner qualification](RELEASE_QUALIFICATION.md). | Owner-protected environment/authority setup, final approval and all missing mandatory evidence remain open. |
164-
| Public claims | Fresh [offline fixture evidence](benchmark-evidence/offline-fixtures-v4.json) reproduces retained public aggregates and binds the current engine/eval source. Historical v1 evidence is preserved. | Planner variants still require successful promotion gates; no retrieval default or leadership claim is promoted. |
164+
| Public claims | Fresh [offline fixture evidence](benchmark-evidence/offline-fixtures-v8.json) reproduces retained public aggregates and binds the current engine/eval source. Historical v1 evidence is preserved. | Planner variants still require successful promotion gates; no retrieval default or leadership claim is promoted. |
165165
| Website contract | Active commercial/MCP/install guidance is generated and checked against a selected shipped public contract in the website candidate. | The live portal, authenticated provider journeys and combined deployed identities require attended acceptance. |
166166

167167
Final commits, distributions, dependency inventories, raw execution results and gate

‎docs/benchmark-evidence/offline-fixtures-v8.json‎

Lines changed: 257 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
f48f4aa3b4bde802c1b2f008820ee1fc45faee1901e876e0725a4b77964e44c2 offline-fixtures-v8.json

‎docs/images/context-efficiency.svg‎

Lines changed: 1 addition & 1 deletion
Loading
-317 Bytes
Loading

‎docs/images/evidence-backed-agent-examples.svg‎

Lines changed: 2 additions & 2 deletions
Loading

‎engraphis/core/consolidate.py‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1697,9 +1697,22 @@ def _build_digest_content(
16971697
) -> tuple[str, str, bool, dict[str, Any]]:
16981698
"""Build deterministic content and an optional bounded provider summary."""
16991699
subject = ", ".join(_common_tokens(cluster)) or "recurring episode"
1700+
# Memory ids are time-sortable only to the millisecond and intentionally carry
1701+
# random suffixes. A cluster built from records written at different clock
1702+
# resolutions must therefore not use timestamp or id order to choose its bounded
1703+
# quote set: that would make compaction accounting and the digest body vary across
1704+
# otherwise identical runs.
1705+
quote_cluster = sorted(
1706+
cluster,
1707+
key=lambda memory: (
1708+
memory.content,
1709+
memory.title,
1710+
memory.id,
1711+
),
1712+
)
17001713
quotes = [
17011714
memory.content.strip().replace("\n", " ")[:300]
1702-
for memory in cluster[:DIGEST_QUOTES]
1715+
for memory in quote_cluster[:DIGEST_QUOTES]
17031716
]
17041717
content = (
17051718
f"Recurring pattern ({len(cluster)} occurrences): {subject}.\n"

‎engraphis/core/ids.py‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,16 @@
1010
from __future__ import annotations
1111

1212
import secrets
13+
import threading
1314
import time
1415
from typing import Optional
1516

1617
_CROCKFORD = "0123456789ABCDEFGHJKMNPQRSTVWXYZ" # excludes I, L, O, U
1718
_MAX_TIMESTAMP_MS = 1 << 48
19+
_MAX_RANDOM = (1 << 80) - 1
20+
_ULID_LOCK = threading.Lock()
21+
_LAST_TIMESTAMP_MS = -1
22+
_LAST_RANDOM = -1
1823

1924

2025
# Canonical prefixes for each entity kind.
@@ -45,7 +50,7 @@ def _encode(value: int, length: int) -> str:
4550

4651

4752
def ulid(timestamp_ms: Optional[int] = None) -> str:
48-
"""Return a 26-char, lexicographically sortable ULID."""
53+
"""Return a 26-char ULID that is monotonic within one process and timestamp."""
4954
if timestamp_ms is None:
5055
ts = int(time.time() * 1000)
5156
elif isinstance(timestamp_ms, bool) or not isinstance(timestamp_ms, int):
@@ -54,7 +59,16 @@ def ulid(timestamp_ms: Optional[int] = None) -> str:
5459
ts = timestamp_ms
5560
if not 0 <= ts < _MAX_TIMESTAMP_MS:
5661
raise ValueError("timestamp_ms must be an integer in range [0, 2**48)")
57-
rand = secrets.randbits(80)
62+
global _LAST_RANDOM, _LAST_TIMESTAMP_MS
63+
with _ULID_LOCK:
64+
if ts == _LAST_TIMESTAMP_MS:
65+
if _LAST_RANDOM >= _MAX_RANDOM:
66+
raise RuntimeError("ULID entropy exhausted within one millisecond")
67+
rand = _LAST_RANDOM + 1
68+
else:
69+
rand = secrets.randbits(80)
70+
_LAST_TIMESTAMP_MS = ts
71+
_LAST_RANDOM = rand
5872
return _encode(ts, 10) + _encode(rand, 16)
5973

6074

0 commit comments

Comments
 (0)