Skip to content

feat(*): add an experimental Curator that cultivates harness-of-harnesses workers - #802

Merged
0xKT merged 4 commits into
EverMind-AI:mainfrom
tzteyang:feat/experimental_curator_si
Sep 26, 2026
Merged

0xKT merged 4 commits into
EverMind-AI:mainfrom
tzteyang:feat/experimental_curator_si

Conversation

@tzteyang

Copy link
Copy Markdown
Contributor

Summary

Adds experimental/, a first version of worker RSI: a Curator that shapes a
Raven worker's harness from the task, the materials handed to it and feedback,
the way a new employee is trained.

The worker is a harness-of-harnesses. A root Raven harness faces the user and
calls child harnesses, each a full Raven agent hosted over ACP, through the
nodes of its playbooks. Every harness has the same four strategy facets
(memory, planning, capability, action). The Curator reads the actual harnesses
and generates code and config on those facets at both levels: the root
candidate sets the playbooks and writes per-node requirements, each child is
curated against the requirements of its nodes, and the set is checked on
copies and activated together.

Layout:

  • curator/: staged generation (understand, select, design, implement,
    repair), two-level composition, harness contracts, and raven_adapter/,
    the only layer that touches a worker's Raven runtime
  • analyst/: turns an evaluator's words and execution records into
    observable behavior requirements
  • iteration/: the generic loop (onboarding curation, then trial, review and
    curation each round) and a CLI for a human evaluator
  • simulation/: a simulated travel agency that runs the loop, plus one
    showcase case (simulation/cases/s0925c/: case notes, the exported
    transcript split by phase, and the round 1 and round 3 decks)
  • docs/design.md (structure, curation flow, composition sequence, the Raven
    connection) and docs/rsi-iteration.md (the loop and the Analyst)

Key decisions:

  • No change under raven/, and raven/ does not import experimental/.
    Generated artifacts land on Raven's existing extension points. The few Raven
    internals the adapter reads are listed in docs/design.md section 5.4.
  • The Curator explores a read-only source snapshot; the caller names the
    evaluation side to withhold from it, so the generic layers name no scenario.
  • The single-layer owner analysis is kept only to reproduce the showcase case
    and is marked for removal; the two-layer Analyst is the default.
  • The two decks are .pptx files of about 230 KB each. They pass
    check-large-files; flagging them in case they count as report assets
    under AGENTS.md section 7, in which case they can move out of git behind a
    link.
  • One follow-up for Raven 0.2.2: playbooks in prompt and rounds modes ship no
    fixed nodes, so node requirements are read from dag playbooks only.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

On the branch rebased onto the latest main:

  • uv run pytest tests/test_harness_curator_*.py tests/test_analyst_*.py tests/test_iteration_*.py tests/test_simulation_*.py -q: 368 passed

  • uv run pytest -m integration tests/integration/test_harness_curator_*.py --deselect tests/integration/test_harness_curator_real_llm.py -q
    (offline, stub providers): 57 passed

  • uv run pytest -q (whole suite, with the host proxy variables unset and
    PYTHONTZPATH set): 26692 passed, 2 failed. One was an order-dependent test
    of this PR and is fixed here; the other,
    tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing,
    fails only because the machine runs as root and is unrelated.

  • uv run ruff check and uv run ruff format --check on the touched files:
    clean

  • scripts/check_large_files.py origin/main..HEAD,
    scripts/check_source_language.py origin/main..HEAD,
    scripts/check_commit_messages.py origin/main..HEAD and
    npx commitlint --from origin/main --to HEAD: all exit 0

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

Risk

No user-visible change for Raven: experimental/ is not imported by
raven/ and is not packaged in the wheel (packages = ["raven"]).
CONTEXT.md gains one glossary entry. The Curator's exploration offers a
shell only under an OS sandbox executor. Rollback is a revert of the squash
commit; nothing else depends on experimental/.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

N/A

Yuqi Yang and others added 4 commits September 26, 2026 05:36
…sses workers

Add experimental/, a first version of worker RSI.

The worker is a harness-of-harnesses: a root Raven harness faces the user
and calls child harnesses, each a full Raven agent hosted over ACP, through
the nodes of its playbooks. Every harness, root or child, has the same four
strategy facets (memory, planning, capability, action).

A Curator reads the actual harnesses and, from the task, the materials
handed to it and feedback, generates code and config on those facets at
both levels. The root candidate sets the playbooks and writes per-node
requirements; each child is then curated against the requirements of its
nodes; the whole set is checked on copies and activated together.

Layout:
- curator/: staged generation, two-level composition, harness contracts,
  and raven_adapter, the only layer that touches a worker's Raven runtime
- analyst/: turns an evaluator's words and execution records into
  observable behavior requirements
- iteration/: the generic loop (onboarding curation, then trial, review
  and curation each round) and a CLI for a human evaluator
- simulation/: a simulated travel agency that runs the loop, and one
  showcase case (s0925c: case notes, exported transcript, round 1 and
  round 3 decks)
- docs/design.md, docs/rsi-iteration.md, README.md

No change under raven/. Generated artifacts land on Raven's existing
extension points; the few Raven internals the adapter reads are listed in
docs/design.md. The single-layer owner analysis is kept only to reproduce
the showcase case and is marked for removal.

Tests: unit tests in tests/test_{harness_curator,analyst,iteration,
simulation}_*.py; offline integration tests in
tests/integration/test_harness_curator_*.py, deselected by default.

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
Raven now also loads prompt and rounds playbooks, which ship no fixed
nodes (spec.nodes is None). The Curator's node inventory and its check on
node requirements iterated every loaded playbook's nodes and failed on
them; both now read nodes from dag playbooks only, the one mode whose
nodes can hold requirements.

The per-session planning test now binds a turn with no conversation first,
so a permission turn another test left in the same worker process cannot
add a session to it.

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
The s0925c transcript is now transcript/, an index plus one file per
phase: inputs, onboarding, rounds 1 to 3 and the rule ledger.

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
…ions

Both options had one value, their default, and were kept only so older
recorded command lines would parse; the reproduce command no longer
emits them.

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
@0xKT
0xKT merged commit ad2f054 into EverMind-AI:main Sep 26, 2026
19 of 20 checks passed
0xKT pushed a commit that referenced this pull request Sep 26, 2026
… claims (#803)

## Summary

The self-evolution section described a capability with no author in it.
The Curator landed on main in #802, so this names it: it is what
rewrites the four strategy modules, it reaches past the prompt into the
tools, skills and judgement an agent runs on, and it works in rounds
that end when the reader is satisfied, when it has nothing left worth
changing, or when the round budget runs out. Quality is held by
verification before installation and by withholding the reference
answer.

The heading drops `(Experimental)`.

Four claims were wrong or loose against the code, and are corrected:

- **"The next turn of the same conversation already runs on it"** is
true of a charter bound for one turn, not of a Curator, which installs
at an execution boundary
(`experimental/curator/raven_adapter/deployment.py`, `activate`). The
section now says the agent runs on the new implementation once it is
installed.
- **Feedback described as requirements "rather than a score"** was
wrong: a `Signal` carries per-item verdicts and metrics
(`experimental/iteration/protocols.py`). What is withheld is the
reference answer -- `answerless` exists precisely because a reference
that reaches the Curator gets copied into the harness and scores the
same cases again without the agent improving. That is what the section
now says.
- **Capability** is "which tool definitions one iteration exposes" per
`raven/contracts/harness.py`, not "what it can reach".
- **Action** decides what to do next as well as judging it before it
runs; the previous line carried only the second half.

The Chinese keeps the register set in #787 rather than reverting it to
the earlier wording.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Documentation only: one section rewritten in each README, no code
touched.

```
npx commitlint --from origin/main --to HEAD --config commitlint.config.cjs
python3 scripts/check_commit_messages.py origin/main..HEAD
python3 scripts/check_source_language.py origin/main..HEAD
python3 scripts/check_large_files.py origin/main..HEAD
```

All four pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

No behaviour change.

One thing a reviewer should weigh: the third paragraph describes a
Persona, whose implementation is in #709 and is not on main. That was
raised on #780 and is not resolved by this change; it is called out here
so the decision is deliberate rather than inherited.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: yao pengfei <yaopengfei@shanda.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
@0xKT 0xKT mentioned this pull request Sep 27, 2026
12 tasks
0xKT added a commit that referenced this pull request Sep 27, 2026
## Summary

Bump the package version from 0.2.2 to 0.2.3 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.2.2, main merged 13 PRs: three features (#802 an experimental
Curator that cultivates harness-of-harnesses workers, #805 one header
per desk pane, #806 upgrade from the page with progress), four fixes
(#792 and #799 name grok and copilot connection failures, #807 records
the questions an unattended turn could not ask, #808 keeps the report
shape on interrupted research turns) and six docs PRs. None carries a
BREAKING CHANGE footer, so this is a patch bump, the way v0.2.1 and
v0.2.2 were for their features. The tag and release notes follow once
this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` on a clean worktree cut from origin/main: Resolved 251
packages, Updated raven v0.2.2 -> v0.2.3
- `git diff --stat origin/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- `uv version` in the worktree: raven 0.2.3
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.3 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants