Skip to content

Commit a278689

Browse files
feat(spec): add scoped audits and review-safe Spec Studio
1 parent 619f498 commit a278689

40 files changed

Lines changed: 8232 additions & 42 deletions

‎.claude-plugin/skill-assets.sha256‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ c5d0c26f28c9ee14092f9deaf24c98dd8bef49d971fef2b7a537ffb1ab9f2887 .claude-plugin
33
aeee7a94671ceb306fe2d24c5acc9f2d96ad8a8e7410536566799eea6265f080 skills/engraphis-memory/SKILL.md
44
055655db84af07561d002f0c69744313d8413c39f3e873f941f0fa0b1e76dc66 skills/engraphis-memory/references/CONVENTIONS.md
55
9d090a03f5b3f36a34d91f66b72c3844591f6915755ac3a6c6ba5f1b16977de5 skills/engraphis-memory/references/SCOPING.md
6-
4c612f734cc3f50e5566752c6956e43bd432b3c26177e16cde91e196ac9e9b5b skills/engraphis-memory/references/TOOLS.md
6+
cc252abb56bbbcf767ab46dbb6ce04e4f390afd8e8a19c9a1e603f4a88f43b16 skills/engraphis-memory/references/TOOLS.md

‎.github/workflows/ci.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ jobs:
5252
run: python -m eval.grounded
5353
- name: Code-agent arm gate
5454
run: python -m eval.code_arm
55+
- name: Agent specification quality gate
56+
run: python -m eval.spec_crawl
5557
- name: Evidence contract boundary gate
5658
run: python -m eval.evidence_contracts
5759

@@ -129,6 +131,8 @@ jobs:
129131
run: python -m eval.adversarial_memory_security
130132
- name: Evidence contract boundary gate
131133
run: python -m eval.evidence_contracts
134+
- name: Agent specification quality gate (numpy-only)
135+
run: python -m eval.spec_crawl
132136
- name: Build and smoke installed core artifacts
133137
shell: bash
134138
run: |

‎AGENTS.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ python -m eval.reinforcement # bounded re
5959
python -m eval.adversarial_memory_security # prompt/graph boundary
6060
python -m eval.grounded # grounded-abstain decision gate
6161
python -m eval.code_arm # coding-agent arm gate
62+
python -m eval.spec_crawl # agent spec quality & prompt crawl gate
6263
pyright # core + backends typecheck
6364

6465
# ── External benchmarks (real numbers need torch + the dataset; see eval/external.py) ──
@@ -294,6 +295,8 @@ worktree is the delivery boundary:
294295
- **`docs/OBSIDIAN_IMPORT.md`** — the rich Obsidian Markdown adapter (frontmatter, wikilinks,
295296
aliases, attachments) and its compatibility command. It supplements, rather than replaces,
296297
the universal document-import guide.
298+
- **`docs/SPEC_CRAWL.md`** — agent specification quality crawler, 7-axis radar coverage,
299+
concept tentacles, "ask, don't guess" clarification flow, and Dashboard Spec Studio.
297300
- **`AGENTS.md`** (this file) + **`CLAUDE.md`** — how to work in the repo.
298301
- **`skills/engraphis-memory/`** — portable Agent Skill (SKILL.md + `references/`) that teaches any
299302
MCP-capable agent the *memory discipline* (when to remember/recall, scoping, tool selection).

‎BENCHMARKS.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -94,14 +94,14 @@ interpretation and do not count as additional benchmark-quality gains.
9494
### Public numeric evidence registry
9595

9696
Every exact public aggregate retained below comes from the checked-in, public-safe
97-
[`offline-fixtures-pr253-four-gap-stack-20261008.json`](docs/benchmark-evidence/offline-fixtures-pr253-four-gap-stack-20261008.json) artifact. Its
97+
[`offline-fixtures-spec-crawl-final-20261010.json`](docs/benchmark-evidence/offline-fixtures-spec-crawl-final-20261010.json) artifact. Its
9898
SHA-256 is
99-
`e423e99ba683c1928984903aef715ad97440c9717cecb3bd3f6fe4f6746ea7d0`, also recorded in the
99+
`d09acc6a735e0ac1bf2883f3aa4e84de7b445e3bccd7013ec907e696c29bea0f`, also recorded in the
100100
adjacent `.sha256` file. The artifact contains no raw questions, answers, prompts, customer data,
101101
or per-record content fingerprints.
102102

103103
The fixture-suite digest is
104-
`e1b7fcea31cb7bf8f8ec85b56e097a0cbfc7d496aef4aef51de73b849b2d17c6`. The artifact defines
104+
`842dce03439343c97a1cbe1a77488f2deaecae7e269880ad6de2822e5cbf1b90`. The artifact defines
105105
the digest algorithm and records the SHA-256 of every suite and dataset file. Each evidence ID
106106
also binds its exact command through `sha256(UTF-8 exact command)`:
107107

@@ -126,13 +126,13 @@ grounded checks in separate panels; provider billing and MCP transport are not m
126126
the SVG and matching PNG with:
127127

128128
```bash
129-
python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-pr253-four-gap-stack-20261008.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png
129+
python scripts/render_benchmark_report.py --report docs/benchmark-evidence/offline-fixtures-spec-crawl-final-20261010.json --output docs/images/context-efficiency.svg --png-output docs/images/context-efficiency.png
130130
```
131131

132132
The companion examples are also generated from that artifact with:
133133

134134
```bash
135-
python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-pr253-four-gap-stack-20261008.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png
135+
python -m scripts.render_benchmark_examples --report docs/benchmark-evidence/offline-fixtures-spec-crawl-final-20261010.json --output docs/images/evidence-backed-agent-examples.svg --png-output docs/images/evidence-backed-agent-examples.png
136136
```
137137
The historical-to-executable mapping is in
138138
[`docs/BENCHMARK_CHANGE_COVERAGE.md`](docs/BENCHMARK_CHANGE_COVERAGE.md).

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Connect a coding agent over MCP with the [agent setup guide](https://github.com/
4949
The current registered artifact contains three deterministic offline fixture runs. It separates context size, retrieval quality, and grounded decision checks; these small fixtures do not establish general task performance.
5050

5151
<p align="center">
52-
<img src="https://raw.githubusercontent.com/Coding-Dev-Tools/engraphis/522def372c45a26d46bc7a05829964ae0240f3a9/docs/images/context-efficiency.svg" alt="Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix e423e99ba683; see the benchmark guide for the full checksum." width="100%">
52+
<img src="docs/images/context-efficiency.svg" alt="Three registered offline fixtures: structure-aware chunking reduced mean retrieved context from 740.3 to 214.3 tokens per question (71.1%, 18 questions); the serialized JSON-shape proxy fell from 24,590 to 11,138 tokens across 26 payload samples and 260 recalls. Candidate and packed retrieval quality (Recall@5, Hit@5, and answer-token recall) are shown separately (each 1.000). Grounded checks show 5/5 answerable queries grounded and 6/6 abstention queries rejected, including a 1/1 quarantined-evidence probe; 11/11 decisions were correct. MCP transport and provider billing were not measured. Artifact SHA-256 prefix d09acc6a735e; see the benchmark guide for the full checksum." width="100%">
5353
<br>
5454
<sup>Three offline fixtures separate context reduction, candidate and packed retrieval quality, and grounded behavior. The chart shows the artifact checksum prefix; see the benchmark guide for the full checksum and reproduction steps.</sup>
5555
</p>
@@ -60,7 +60,7 @@ The current registered artifact contains three deterministic offline fixture run
6060
| Recall payload proxy | JSON-shape proxy: 24,590 → 11,138 tokens (54.71% lower, 26 samples; 260 timed recalls) | Candidate and packed Recall@5, Hit@5, and answer-token recall are each 1.000 |
6161
| Grounded decisions | 5/5 answerable queries grounded; 6/6 abstention queries rejected, including 1/1 quarantined-evidence check | 11/11 decisions correct |
6262

63-
The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](https://github.com/Coding-Dev-Tools/engraphis/blob/522def372c45a26d46bc7a05829964ae0240f3a9/BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations.
63+
The payload figure is a serialized JSON-shape estimate, not an MCP transport measurement or provider billing total. See the [Benchmark methodology](BENCHMARKS.md) for artifact identity, counting methods, reproduction commands, external-evaluation boundaries, and limitations.
6464

6565
## Optional Jev assistance
6666

‎docs/ARCHITECTURE_V3.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ current schema version is 16).
88
flowchart LR
99
Agent["Agent / host LLM"] --> Intent["remember · link · recall_context (compact) · recall"]
1010
CLI["engraphis-graph CLI"] --> Service["MemoryService"]
11-
MCP["Smart MCP (9 tools) / Classic MCP (39 tools)"] --> Service
11+
MCP["Smart MCP (9 tools) / Classic MCP (43 tools)"] --> Service
1212
HTTP["Dashboard + read-only graph HTTP"] --> Service
1313
Import["Local resources / PostgreSQL catalog"] --> Extractors["Optional local extractors"]
1414
Extractors --> Service

‎docs/KILO_CODE_INTEGRATION.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -226,7 +226,7 @@ class, and the appropriate executor revalidates all of it before running.
226226
preserves the named-tool compatibility surface below; new Kilo Code installations should keep the zero-config
227227
Smart command shown above.
228228

229-
### Classic 39-tool inventory
229+
### Classic 43-tool inventory
230230

231231
| Category | Tool | What it does |
232232
|---|---|---|
@@ -269,6 +269,10 @@ Smart command shown above.
269269
| Ops | `engraphis_check_update` | Check the release source and refresh the persistent update cache. |
270270
| Maintenance | `engraphis_consolidate` | Pure dry-run or live sweep; structured calls may process a large cluster across retries. |
271271
| Decision | `engraphis_decide` | Advisory command, contradiction, support, and completion checks. Remote processing requires backend selection and explicit permission for each call. |
272+
| Quality | `engraphis_spec_crawl` | Multi-dimensional prompt & spec crawler; radar scores, concept tentacles, clarify questions, simulated traces. |
273+
| Quality | `engraphis_spec_crawl_answer` | Answer a clarification question with user-provided answers, recalculating scores & simulation trace. |
274+
| Quality | `engraphis_spec_crawl_memories` | Analyze a cluster of memories for contradiction, divergence, and convergence. |
275+
| Quality | `engraphis_spec_crawl_resolve` | Reconcile divergent or contradicted memories with resolved state and supersede chain. |
272276

273277
---
274278

0 commit comments

Comments
 (0)