Skip to content

Commit 3d7bc20

Browse files
authored
Add rlm module: config-driven RLM runtime with pluggable interpreters (#31)
* Add rlm module: sandboxed RLM runtime with pluggable interpreters A driver model writes code cells executed in a sandboxed interpreter whose only host surface is capability calls back into the CapabilityRegistry (sub-LLM queries, tools, sub-agent delegation). Ships the embedded Rhai backend (hermetic) and an external-process backend (Python/Node via a line-delimited JSON wire protocol), an RlmSession cell API, a template-driven RlmRunner loop, and serde-serializable RlmConfig for config-driven use by external harnesses. Gated behind the new `rlm` Cargo feature. Claude-Session: https://claude.ai/code/session_01FFRLM18Fw1mzBhpWMe3B3k * Add rlm e2e tests: rhai subagents, external python/node, config-driven runner Claude-Session: https://claude.ai/code/session_01FFRLM18Fw1mzBhpWMe3B3k * Harden rlm runner loop and prompts from live runs - Nudge once on fence-less driver replies before accepting prose as the answer (models sometimes emit raw unfenced code). - Stop leaking registry model names into provider requests. - Teach the Rhai usage guide the syntax gotchas live models trip on. - Validate tool arguments against the tool schema at the host boundary. - Add rlm_rhai / rlm_python runnable examples (verified live against OpenAI). Claude-Session: https://claude.ai/code/session_01FFRLM18Fw1mzBhpWMe3B3k * Add rlm docs, live tests, and crate-doc feature note Claude-Session: https://claude.ai/code/session_01FFRLM18Fw1mzBhpWMe3B3k
1 parent b23e213 commit 3d7bc20

18 files changed

Lines changed: 3708 additions & 3 deletions

File tree

‎Cargo.lock‎

Lines changed: 21 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,12 @@ default = []
6262
sqlite = ["dep:rusqlite"]
6363
# Embedded Rhai-backed `.ragsh` REPL session runtime (`repl::session`).
6464
repl = ["dep:rhai"]
65+
# Recursive-language-model runtime (`rlm`): a driver model writes code cells
66+
# executed in a sandboxed interpreter (embedded Rhai, or an external Python /
67+
# JavaScript process) whose only host surface is capability calls back into
68+
# the registry (`llm`, `tool`, `agent`). `tokio/process` + `tokio/io-util`
69+
# drive the external interpreter subprocesses.
70+
rlm = ["dep:rhai", "tokio/process", "tokio/io-util"]
6571
# Builtin generic tools (`harness::tools`) kept out of the default dependency
6672
# graph so host applications can choose whether they want these implementations.
6773
tools = ["dep:chrono", "dep:chrono-tz"]
@@ -78,3 +84,12 @@ futures = "0.3"
7884
# The OpenAI provider is always compiled, so these build with a plain
7985
# `cargo build --examples`; they only need a network call + `OPENAI_API_KEY`
8086
# at run time.
87+
88+
# --- RLM examples (need the `rlm` feature) ---
89+
[[example]]
90+
name = "rlm_rhai"
91+
required-features = ["rlm"]
92+
93+
[[example]]
94+
name = "rlm_python"
95+
required-features = ["rlm"]

‎docs/modules/rlm/README.md‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# RLM — the recursive-language-model runtime (`src/rlm/`)
2+
3+
The `rlm` module (Cargo feature `rlm`) turns a code sandbox into a
4+
recursive-language-model harness: a **driver model** writes code cells, the
5+
cells execute in a **sandboxed interpreter**, and the only host surface the
6+
scripts see is a set of **capability calls back into the registry** — sub-LLM
7+
queries, tools, and sub-agent delegation. Because scripts can call models
8+
(and agents that call models), the loop is recursive by construction, which
9+
is the execution pattern of the RLM literature this crate is architected
10+
around (see the crate-level docs in `src/lib.rs`).
11+
12+
```text
13+
┌──────────────────────────────────────────────┐
14+
task ───► │ RlmRunner: driver model ⇄ code cells ⇄ obs. │ ───► answer
15+
└──────────────┬───────────────────────────────┘
16+
│ eval(code)
17+
┌──────────────▼───────────────┐
18+
│ RlmSession │
19+
│ ┌──────────┐ HostCall │
20+
│ │ RlmInter-│ ─────────────► │ RlmHost ──► CapabilityRegistry
21+
│ │ preter │ ◄───────────── │ (policy, counters, depth)
22+
│ └──────────┘ Value/error │ ├─ llm → ChatModel::invoke
23+
└──────────────────────────────┘ ├─ tool → Tool::call
24+
└─ agent → HarnessAgent::run
25+
```
26+
27+
## The three layers
28+
29+
| Layer | Type | What it is |
30+
|---|---|---|
31+
| Interpreter | `RlmInterpreter` (trait) | Pluggable cell execution: `eval_cell`, `set_variable`, `usage_guide`, `shutdown`. State persists across cells like a notebook. |
32+
| Session | `RlmSession` | One interpreter bound to one `RlmHost`; the programmatic "interpreter as an API" surface. Enforces per-cell policy fail-closed. |
33+
| Runner | `RlmRunner` | The model-driven loop: template → system prompt, fenced code cells in, observations out, stop on `final_answer(...)`. |
34+
35+
Every layer is usable on its own: an embedder that wants its own loop (or a
36+
notebook UI) drives `RlmSession::eval` directly and never touches the runner.
37+
38+
## Interpreter backends
39+
40+
- **`InterpreterSpec::Rhai`** (default) — the embedded Rhai engine. This is
41+
the only *hermetic* sandbox: no filesystem, network, or process access
42+
exists inside the engine; the registered capability closures are its whole
43+
world. Bounded by `max_operations`, the cell deadline (`on_progress`
44+
hook), and the blocking bridge around every capability call (the same
45+
fail-closed adapter as `repl::session`).
46+
- **`InterpreterSpec::Python { binary, args }`** — an external CPython child
47+
process (`python3` by default). A bootstrap prelude is injected via `-c`;
48+
cells run in a persistent exec namespace.
49+
- **`InterpreterSpec::Javascript { binary, args }`** — a Node.js child
50+
(`node` by default, prelude via `-e`); cells run in a persistent `vm`
51+
context.
52+
- **`InterpreterSpec::Command { binary, args }`** — any command that speaks
53+
the wire protocol itself (a containerized runner, a jailed interpreter, a
54+
different language).
55+
56+
### The wire protocol (external backends)
57+
58+
Line-delimited JSON on the child's stdin/stdout; calls are strictly
59+
sequential so no correlation ids are needed. Child → host: `ready`,
60+
`call {call: HostCall}`, `result {stdout, value, error}`, `var_set`.
61+
Host → child: `eval {code}`, `set_var {name, value}`,
62+
`call_result {ok, value|error}`, `shutdown`. `HostCall` is the shared,
63+
serde-stable call shape (`{"capability": "llm"|"tool"|"agent"|"final_answer", ...}`).
64+
Any runtime that speaks this protocol is a valid backend — that is the
65+
extension point for other harnesses (e.g. openhuman) to bring their own
66+
interpreter.
67+
68+
### Sandboxing honesty
69+
70+
The host enforces every policy limit fail-closed for **all** backends (a
71+
child that exceeds its deadline or trips a bound is killed, not asked). But
72+
an external child process has whatever OS access the environment grants it.
73+
For untrusted driver models, either stay on the embedded Rhai backend or run
74+
the external interpreter inside real isolation (container/jail/seccomp) via
75+
`InterpreterSpec::Command`.
76+
77+
## The capability surface inside scripts
78+
79+
Identical semantics across languages (spelling varies per usage guide):
80+
81+
- `llm(prompt)` / `llm({model, prompt, system})` → string — a sub-LLM call;
82+
the unnamed form uses the session's default sub-model.
83+
- `tool(name, args)` → tool result (raw JSON when the tool provides it).
84+
Arguments are validated against the tool's schema at the host boundary.
85+
- `agent(name, input)` → string — delegates to a registered
86+
`HarnessAgent`; a full nested agent run with event fan-out and the shared
87+
recursion-depth guard (`RunConfig::checked_child_depth`).
88+
- `final_answer(text)` — ends the run with this answer.
89+
- `print(...)` / `console.log(...)` — captured and echoed back to the
90+
driver next turn, bounded by `max_output_bytes` (explicit truncation
91+
marker).
92+
93+
### Error contract
94+
95+
Script-visible failures (unknown tool, schema mismatch, tool error, provider
96+
error) surface *inside* the script as catchable exceptions (`RlmError` in
97+
Python/JS, `try`/`catch` in Rhai) so the driving model can adapt — that
98+
feedback loop is the point of an RLM. Policy violations (`LimitExceeded`,
99+
`Timeout`, `Cancelled`, `SubAgentDepth`) are **fatal**: the cell aborts (and
100+
an external child is killed) so scripts can never observe and route around
101+
their own resource limits. `rlm::is_fatal` is the classifier.
102+
103+
## Config-driven runs
104+
105+
Everything a run needs is one serde document — the integration surface for
106+
external harnesses:
107+
108+
```json
109+
{
110+
"interpreter": {"kind": "python", "binary": "/opt/venv/bin/python3"},
111+
"driver_model": "openai",
112+
"sub_model": "openai",
113+
"template": "context-explorer",
114+
"policy": {
115+
"max_cells": 8, "max_llm_calls": 16, "max_tool_calls": 64,
116+
"max_agent_calls": 8, "max_depth": 8,
117+
"max_script_bytes": 65536, "max_output_bytes": 262144,
118+
"cell_timeout": 90000, "max_operations": 5000000
119+
}
120+
}
121+
```
122+
123+
`RlmConfig::from_json` → `RlmRunner::from_config(config, registry, state)` →
124+
`runner.set_context(json)` (optional) → `runner.run(task)`.
125+
126+
### Templates
127+
128+
`TemplateSpec::Named` selects a built-in; `TemplateSpec::Inline` carries a
129+
custom `RlmTemplate` in the config document. Placeholders `{{language}}`,
130+
`{{usage}}`, `{{capabilities}}`, `{{limits}}` are substituted at run time
131+
from the live session (so the prompt always reflects the actual registry and
132+
policy). Built-ins:
133+
134+
- `general` — solve the task with code; sub-LLMs for fuzzy subproblems.
135+
- `context-explorer` — the RLM long-context pattern: material injected as
136+
the `context` variable, probed programmatically, never printed whole.
137+
- `orchestrator` — decompose and delegate to registered sub-agents.
138+
139+
## Policy (all fail-closed)
140+
141+
`RlmPolicy`: `max_cells`, `max_script_bytes`, `max_output_bytes`,
142+
`max_llm_calls`, `max_tool_calls`, `max_agent_calls`, `max_depth`,
143+
`cell_timeout` (ms in JSON), `max_operations` (Rhai only). Counters are
144+
session-cumulative. `RlmCancelFlag` provides sticky external cancellation,
145+
observed mid-script (Rhai `on_progress`), mid-call (blocking bridge), and
146+
before each cell.
147+
148+
## Runner loop details
149+
150+
- The driver must reply with exactly one fenced code block; the first fence
151+
is extracted regardless of its info string (models mislabel languages).
152+
- A fence-less reply earns one nudge (it is often raw unfenced code); a
153+
second consecutive fence-less reply is accepted as a prose answer
154+
(`RlmStopReason::ModelAnswered`).
155+
- Observations echo captured stdout, the cell value, and any script error.
156+
- Stop reasons: `Answered` (a cell called `final_answer`), `ModelAnswered`,
157+
`CellBudgetExhausted`. The full per-cell trajectory is returned in
158+
`RlmOutcome::steps`.
159+
160+
## Tests and examples
161+
162+
- Unit: `src/rlm/test.rs` (config round-trips, templates, extraction, the
163+
embedded backend, runner loop against `ScriptedModel`).
164+
- E2E: `tests/e2e_rlm.rs` — sub-agent delegation from scripts, real
165+
`python3`/`node` protocol round-trips (skipped when the binary is
166+
missing), fail-closed timeout kill, config-driven runner.
167+
- Live: `tests/live_rlm.rs` — network-gated on `OPENAI_API_KEY`.
168+
- Examples: `examples/rlm_rhai.rs` (context-explorer over expense records),
169+
`examples/rlm_python.rs` (tools + a real sub-agent, external Python).
170+
Run with `cargo run --features rlm --example rlm_rhai`.
171+
172+
## Relationship to `repl::session`
173+
174+
`repl::session` is the `.ragsh` interactive orchestration surface: Rhai
175+
only, richer built-ins (graph authoring, batching), REPL-first. `rlm` is the
176+
model-*driven* counterpart: interpreter-pluggable, config-first, and shaped
177+
for embedding in other harnesses. They share design DNA deliberately — the
178+
fail-closed blocking bridge, reserved capability boundary, and policy
179+
posture are the same pattern.

‎examples/rlm_python.rs‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
//! A live RLM run over an **external Python interpreter**, with a real tool
2+
//! and a real sub-agent registered.
3+
//!
4+
//! The driver model writes Python cells; the child `python3` process executes
5+
//! them, and every capability call (`llm`, `tool`, `agent`, `final_answer`)
6+
//! travels back over the wire protocol to the host, which enforces the policy
7+
//! and lowers to the harness runtime. The `agent("summarizer", ...)` call
8+
//! drives a *complete nested agent run* on its own OpenAI-backed harness —
9+
//! scripts calling agents calling models.
10+
//!
11+
//! Run with (needs `OPENAI_API_KEY` and `python3` on PATH):
12+
//!
13+
//! ```text
14+
//! cargo run --features rlm --example rlm_python
15+
//! ```
16+
17+
use std::sync::Arc;
18+
19+
use async_trait::async_trait;
20+
use serde_json::json;
21+
22+
use tinyagents::harness::providers::openai::OpenAiModel;
23+
use tinyagents::harness::runtime::AgentHarness;
24+
use tinyagents::harness::tool::{Tool, ToolCall, ToolResult, ToolSchema};
25+
use tinyagents::registry::CapabilityRegistry;
26+
use tinyagents::rlm::{RlmConfig, RlmRunner};
27+
use tinyagents::{HarnessSubAgent, Result, SubAgent};
28+
29+
/// A deterministic metrics tool the script can query.
30+
struct MetricsTool;
31+
32+
#[async_trait]
33+
impl Tool<()> for MetricsTool {
34+
fn name(&self) -> &str {
35+
"service_metrics"
36+
}
37+
38+
fn description(&self) -> &str {
39+
"Returns latency and error-rate metrics for a named service."
40+
}
41+
42+
fn schema(&self) -> ToolSchema {
43+
ToolSchema::new(
44+
"service_metrics",
45+
"Returns latency and error-rate metrics for a named service.",
46+
json!({
47+
"type": "object",
48+
"properties": { "service": { "type": "string" } },
49+
"required": ["service"],
50+
"additionalProperties": false
51+
}),
52+
)
53+
}
54+
55+
async fn call(&self, _state: &(), call: ToolCall) -> Result<ToolResult> {
56+
let service = call
57+
.arguments
58+
.get("service")
59+
.and_then(|v| v.as_str())
60+
.unwrap_or("unknown");
61+
let metrics = match service {
62+
"checkout" => json!({"p99_ms": 2140, "error_rate": 0.031, "deploys_today": 3}),
63+
"search" => json!({"p99_ms": 180, "error_rate": 0.002, "deploys_today": 0}),
64+
"auth" => json!({"p99_ms": 95, "error_rate": 0.001, "deploys_today": 1}),
65+
other => json!({"error": format!("unknown service `{other}`")}),
66+
};
67+
let mut result = ToolResult::text(call.id, call.name, metrics.to_string());
68+
result.raw = Some(metrics);
69+
Ok(result)
70+
}
71+
}
72+
73+
#[tokio::main]
74+
async fn main() -> Result<()> {
75+
dotenvy::dotenv().ok();
76+
77+
let mut registry: CapabilityRegistry<()> = CapabilityRegistry::new();
78+
registry.register_model("openai", Arc::new(OpenAiModel::from_env()?))?;
79+
registry.register_tool(Arc::new(MetricsTool))?;
80+
81+
// A real sub-agent on its own harness: scripts delegate to it by name.
82+
let mut child: AgentHarness<()> = AgentHarness::new();
83+
child
84+
.register_model("openai", Arc::new(OpenAiModel::from_env()?))
85+
.set_default_model("openai");
86+
let summarizer = Arc::new(
87+
SubAgent::new(
88+
"summarizer",
89+
"Writes a crisp two-sentence incident summary from raw findings.",
90+
Arc::new(child),
91+
)
92+
.with_system_prompt(
93+
"You summarize incident findings for executives: two sentences, plain language, \
94+
lead with impact.",
95+
),
96+
);
97+
registry.register_agent(Arc::new(HarnessSubAgent::new(summarizer)))?;
98+
99+
let config = RlmConfig::from_json(
100+
r#"{
101+
"interpreter": {"kind": "python"},
102+
"driver_model": "openai",
103+
"template": "general",
104+
"policy": {"max_cells": 8, "cell_timeout": 90000}
105+
}"#,
106+
)?;
107+
108+
let mut runner = RlmRunner::from_config(config, Arc::new(registry), Arc::new(()))?;
109+
let outcome = runner
110+
.run(
111+
"Check the service_metrics tool for the services checkout, search, and auth. \
112+
Identify which service looks unhealthy and why. Then delegate to the `summarizer` \
113+
agent to produce an executive summary of your findings, and return that summary \
114+
as the final answer.",
115+
)
116+
.await?;
117+
118+
for (i, step) in outcome.steps.iter().enumerate() {
119+
println!("── cell {} ──\n{}\n", i + 1, step.code);
120+
if !step.outcome.stdout.is_empty() {
121+
println!("stdout:\n{}", step.outcome.stdout);
122+
}
123+
if let Some(error) = &step.outcome.error {
124+
println!("error: {error}");
125+
}
126+
}
127+
println!("── answer ({:?}) ──", outcome.stop_reason);
128+
println!("{}", outcome.answer.as_deref().unwrap_or("(none)"));
129+
println!(
130+
"\ndriver calls: {}, sub-llm calls: {}, tool calls: {}, agent calls: {}",
131+
outcome.driver_calls, outcome.sub_llm_calls, outcome.tool_calls, outcome.agent_calls
132+
);
133+
runner.shutdown().await?;
134+
Ok(())
135+
}

0 commit comments

Comments
 (0)