diff --git a/.github/ci-paths-filter.yml b/.github/ci-paths-filter.yml index d6ebe4db2be..3a2c5ea41d1 100644 --- a/.github/ci-paths-filter.yml +++ b/.github/ci-paths-filter.yml @@ -69,6 +69,7 @@ rust-core: - 'scripts/ci/assert-coverage-presence.sh' - 'scripts/ci/coverage-presence-allowlist.txt' - 'scripts/ci/check-openhuman-rust-layout.mjs' + - 'scripts/ci/check-crate-chain.mjs' - 'scripts/ci/check-agent-runtime-boundary.mjs' - 'scripts/ci/agent-runtime-boundary-baseline.json' - 'scripts/ci/check-saas-ambient.mjs' diff --git a/AGENTS.md b/AGENTS.md index 4ed855ae700..a8b76b8a587 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,15 +13,15 @@ Architecture: [overview](gitbooks/developing/architecture.md), | Path | Purpose | | --- | --- | | `app/src/` | Vite and React frontend | -| `crates/openhuman-app/` | Thin desktop host; excluded from the root workspace, build with `--manifest-path crates/openhuman-app/Cargo.toml` | +| `crates/openhuman-app/` | Thin desktop host; excluded from the root workspace, build with `--manifest-path crates/openhuman-app/Cargo.toml`. Depends on `openhuman-rpc` only and boots its in-process core with `openhuman_rpc::host::desktop` | | `crates/openhuman-core/` | Package `openhuman`: business domains under `src//`, the controller contract, dispatch and auth under `src/core/` | | `crates/openhuman-core/src//` | Flat business-domain modules (agent, memory, tools, security, channels, ...) | | `crates/openhuman-core/src/core/` | CLI, controller contract (`Outcome`, schemas) and in-process dispatch, controller registry, event bus, runtime composition; no business logic and no JSON-RPC server | -| `crates/openhuman-cli/` | The `openhuman-core` binary (`src/main.rs`), the developer bins (`src/bin/`), and every root `tests/*.rs` / `examples/*.rs` target; depends on `openhuman-tinyhumans` for the backend transport the core does not carry | -| `crates/openhuman-embed/` | Typed library facade for embedding the core in another product | -| `crates/openhuman-rpc/` | JSON-RPC 2.0 over the core: envelopes, HTTP client, and the server (router, Socket.IO, listener, `run_server*`) used by app, CLI and TUI; plus `session_store` (`session-store` feature), the on-disk session store (`session_raw/`, `session_db/`, `tinyagents_store/`, turn states) behind TinyAgents' session store port, which the app, CLI and TUI install. Core and embed reach session state through the port (`agent::session_store`); a few legacy paths still fall back to workspace files when no store is installed. With a storage URL (`OPENHUMAN_STORAGE_URL` / `[storage] url`), `install_for_host` installs TinyAgents' `DriverSessionStores` over that backend instead (core `storage` domain) | -| `crates/openhuman-tinyhumans/` | The TinyHumans layer above embed: SDK-backed backend transport, a `RuntimeBuilder` that boots connected, and the host-side login/session owner (login-token exchange, `/auth/me`, current-user cache, credential handoff) used by app and TUI | -| `crates/openhuman-tui/` | Standalone terminal frontend | +| `crates/openhuman-cli/` | The `openhuman-core` binary (`src/main.rs`, `openhuman_rpc::host::cli`), the ops bins (`src/bin/`: `openhuman-fleet`, `test-mcp-stub`), and every root `tests/*.rs` / `examples/*.rs` target. Normal dependency: `openhuman-rpc` only; the tests reach core, embed and tinyhumans through `[dev-dependencies]`. The benchmark bins live in the openhuman-benchmarks repository | +| `crates/openhuman-embed/` | Library facade over the core (depends on core only): `Runtime`/`RuntimeBuilder` with host presets, `embed::process` (tokio runtime, logging, dotenv, master key, Sentry options), and the curated facades hosts use (`config`, `artifacts`, `chat_surface`, `modules`, `identity`). Its doc-hidden `__host` list is for tinyhumans and rpc only | +| `crates/openhuman-rpc/` | Top of the library chain (depends on tinyhumans only). JSON-RPC 2.0 over the core: envelopes, HTTP client, and the server (router, Socket.IO, listener, `run_server*`); `host::{cli, desktop, tui}`, the shared host boot; re-exports `embed` and `tinyhumans` as the hosts' curated facade; plus `session_store` (`session-store` feature), the on-disk session store (`session_raw/`, `session_db/`, `tinyagents_store/`, turn states) behind TinyAgents' session store port, which the app, CLI and TUI install. Core and embed reach session state through the port (`agent::session_store`); a few legacy paths still fall back to workspace files when no store is installed. With a storage URL (`OPENHUMAN_STORAGE_URL` / `[storage] url`), `install_for_host` installs TinyAgents' `DriverSessionStores` over that backend instead (core `storage` domain) | +| `crates/openhuman-tinyhumans/` | The TinyHumans layer above embed (depends on embed only): SDK-backed backend transport, a `RuntimeBuilder` that boots connected, and the host-side login/session owner (login-token exchange, `/auth/me`, current-user cache, credential handoff) used by app and TUI | +| `crates/openhuman-tui/` | Standalone terminal frontend; depends on `openhuman-rpc` only and boots with `openhuman_rpc::host::tui` | | `tests/` | Rust integration and JSON-RPC tests | | `gitbooks/` | Public product and contributor documentation | | `docs/` | Internal maintainer documentation | @@ -30,6 +30,20 @@ Architecture: [overview](gitbooks/developing/architecture.md), Run commands from the repository root. The root package is a private pnpm workspace. +The Rust crates form a strict chain; each one's normal dependencies name only +the layer directly below it: + +```text +openhuman-core -> openhuman-embed -> openhuman-tinyhumans -> openhuman-rpc -> { app, cli, tui } +``` + +Hosts (app, CLI, TUI) depend on `openhuman-rpc` alone and reach the core +through its curated facade (`openhuman_rpc::host`, `openhuman_rpc::embed`, +`openhuman_rpc::tinyhumans`), never through `__host` / `core_host` or an +`openhuman_core::` path. `node scripts/ci/check-crate-chain.mjs` (part of +`pnpm rust:layout`) enforces both. Dev-dependencies are exempt, which is how +the root tests keep reaching into the core. + ## Product boundaries - The shipped Tauri product targets Windows, macOS, and Linux. @@ -40,8 +54,9 @@ workspace. - The frontend and Tauri shell present or orchestrate core behavior. Do not duplicate core policy in TypeScript or shell code. - The desktop core runs as a tokio task managed by - `crates/openhuman-app/src/core_process.rs`. Frontend RPC uses the per-launch bearer - returned through the `core_rpc_token` command. + `crates/openhuman-app/src/core_process.rs` (`openhuman_rpc::host::desktop`). + Frontend RPC uses the per-launch bearer returned through the + `core_rpc_token` command. - `OPENHUMAN_CORE_REUSE_EXISTING=1` connects the shell to an external core for debugging. @@ -180,9 +195,10 @@ coverage must be at least 80 percent. need no entry. Run them as `cargo test -p openhuman-cli --test `. - A suite that boots the core **in-process** and reaches the backend (mock) must call `tinyhumans_boot::boot()` from `tests/support/tinyhumans_boot.rs` - first; the core has no backend transport of its own, and without it every - backend call answers `BACKEND_UNAVAILABLE:`. Suites that spawn the - `openhuman-core` binary get it from `main.rs`. + first (it runs `openhuman_tinyhumans::install`, a dev-dependency of + `openhuman-cli`); the core has no backend transport of its own, and without + it every backend call answers `BACKEND_UNAVAILABLE:`. Suites that spawn the + `openhuman-core` binary get it from `main.rs` (`openhuman_rpc::host::cli`). Shared mock backend: @@ -369,14 +385,23 @@ Additional rules: (`http-client` feature), and the whole server (`server` feature): the axum router and handlers, auth middleware, Socket.IO, `/dev/connect`, the listener bind (`openhuman_rpc::server::serve`) and the `run_server*` entry - points. A host that runs `openhuman-core run`/`serve` calls - `openhuman_rpc::server::install_cli_server()` before `run_core_from_args`. + points. `openhuman_rpc::host::cli` gives the core this crate's server as + the `run`/`serve` launcher (the older `install_cli_server()` + + `run_core_from_args` pair does the same for embedders that predate it). Domain-owned HTTP handlers the router mounts (`inference::http`, the dictation WebSocket) stay in their domains behind core's `http-server` feature. The `http_host` static-directory file server lives here too (`openhuman_rpc::http_host`); `install_cli_server()` and `build_core_http_router()` register its `http_host.*` controllers as a core extension, so a host without this crate has no `http_host` surface. +- The hosts boot through `openhuman_rpc::host`: `host::cli(args)` is the + `openhuman-core` binary (and the app's `core` / `mcp` subcommands); + `host::desktop(DesktopOptions, shutdown, ready_tx)` is the desktop shell's + embedded server (in-memory bearer, preferred port with stale-listener + takeover, ready signal); `host::tui()` builds the TUI's runtime. Each + connects the TinyHumans backend itself. One embed runtime exists per + process, so a host that restarts its server must let the old task (and the + runtime it owns) drop before it spawns the next. ## Tool, harness, and runtime boundaries @@ -476,11 +501,12 @@ narrow capabilities. Cargo default features define the contributor build; `scripts/ci/product-features.txt` defines the shipped product. The Tauri shell -disables default features, so product gates must be forwarded explicitly in -`crates/openhuman-app/Cargo.toml` and checked by -`scripts/ci/check-feature-forwarding.mjs`. The same gate checks the library -chain: a core gate must be forwarded by `openhuman-embed`, then -`openhuman-tinyhumans`, then `openhuman-cli`, or be listed in +disables default features, so product gates must be forwarded explicitly on +its `openhuman-rpc` dependency in `crates/openhuman-app/Cargo.toml` and +checked by `scripts/ci/check-feature-forwarding.mjs`. The same gate checks the +library chain: a core gate must be forwarded by `openhuman-embed`, then +`openhuman-tinyhumans`, then `openhuman-rpc`, then the `openhuman-cli` and +`openhuman-tui` hosts (each to `openhuman-rpc/`), or be listed in `CHAIN_GATES_NOT_FORWARDED` / `CHAIN_LOCAL_GATES` with a reason. Test both enabled and disabled builds after changing a gate. Use `scripts/assert-shed.sh` or `scripts/dep-sim.py` before claiming a dependency reduction. @@ -622,9 +648,13 @@ to them. The `cortexdb` engine likewise calls CortexDB directly with the user's key. Never add `tinyhumans-sdk` back to the core; the only crate allowed to depend on it is `openhuman-tinyhumans` (`cargo tree -p openhuman -i tinyhumans-sdk` must stay empty). Every host that boots a core -(`crates/openhuman-app/src/main.rs` and `lib.rs::run`, +(`crates/openhuman-app/src/core_process.rs` and `lib.rs::run_core_from_args`, `crates/openhuman-tui/src/runner.rs`, `crates/openhuman-cli/src/main.rs`) -calls `openhuman_tinyhumans::install` first; it also registers the hosted RPC +does it through an `openhuman_rpc::host` entry, which connects the TinyHumans +layer (`openhuman_tinyhumans::RuntimeBuilder::connect`: the transport, as the +process global and bound to the runtime; library hosts call +`openhuman_tinyhumans::install` or the `RuntimeBuilder` directly). Connecting +also registers the hosted RPC proxies (`billing`, `team`, `referral`, `announcements`, `webhooks`, `channel_link`, `oauth` — `crates/openhuman-tinyhumans/src/hosted/`) into the core's controller registry through `core::all::register_controller_extension` diff --git a/Cargo.lock b/Cargo.lock index d3cf7a4af42..7c3d33c0df0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4667,7 +4667,6 @@ dependencies = [ "base64 0.22.1", "chrono", "clap", - "dotenvy", "env_logger", "flate2", "futures", @@ -4677,6 +4676,7 @@ dependencies = [ "libc", "log", "openhuman", + "openhuman-embed", "openhuman-rpc", "openhuman-tinyhumans", "parking_lot", @@ -4816,13 +4816,9 @@ dependencies = [ "base64 0.22.1", "chrono", "crossterm", - "dotenvy", "log", - "openhuman", "openhuman-rpc", - "openhuman-tinyhumans", "ratatui", - "sentry", "serde_json", "tempfile", "tokio", diff --git a/Cargo.toml b/Cargo.toml index 23d063e4490..0818ed4f238 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -35,7 +35,10 @@ license = "GPL-3.0-only" repository = "https://github.com/tinyhumansai/openhuman" [workspace.dependencies] -openhuman-core = { path = "crates/openhuman-core", package = "openhuman" } +# The library chain: core -> embed -> tinyhumans -> rpc -> app/cli/tui. Hosts +# name `openhuman-rpc` only (`scripts/ci/check-crate-chain.mjs`). There is no +# `openhuman-core` entry on purpose: one spelled here carried the core's whole +# `default` feature set into whichever member named it. openhuman-rpc = { path = "crates/openhuman-rpc", default-features = false } openhuman-embed = { path = "crates/openhuman-embed", default-features = false } openhuman-tinyhumans = { path = "crates/openhuman-tinyhumans", default-features = false } diff --git a/crates/README.md b/crates/README.md index 19c2b443a38..17bf1524509 100644 --- a/crates/README.md +++ b/crates/README.md @@ -12,72 +12,74 @@ with `--manifest-path crates/openhuman-app/Cargo.toml`. | Crate | Package / lib | What it is | | --- | --- | --- | | [`openhuman-core`](openhuman-core/README.md) | package `openhuman`, lib `openhuman_core` | The core library: every business domain under `src//`, the controller contract and registry, in-process dispatch, the event bus and the CLI dispatcher. No JSON-RPC server, no backend client, no binary targets. | -| [`openhuman-embed`](openhuman-embed/README.md) | `openhuman-embed` | The typed library facade for running the core in-process in another product (`Runtime` then `Agent`). | -| [`openhuman-rpc`](openhuman-rpc/README.md) | `openhuman-rpc` | JSON-RPC 2.0 over the core: envelopes, the HTTP client (`http-client`), the server with Socket.IO and the `run_server*` entry points (`server`), and the on-disk session store (`session-store`). | +| [`openhuman-embed`](openhuman-embed/README.md) | `openhuman-embed` | The typed library facade for running the core in-process in another product (`Runtime` then `Agent`), plus the host presets, `embed::process` lifecycle helpers and the curated facades the hosts use. | +| [`openhuman-rpc`](openhuman-rpc/README.md) | `openhuman-rpc` | JSON-RPC 2.0 over the core: envelopes, the HTTP client (`http-client`), the server with Socket.IO and the `run_server*` entry points (`server`), the on-disk session store (`session-store`), and `host::{cli, desktop, tui}`, the shared host boot. Re-exports `embed` and `tinyhumans`: the one dependency a host needs. | | [`openhuman-tinyhumans`](openhuman-tinyhumans/README.md) | `openhuman-tinyhumans` | The hosted-backend layer: the SDK-backed `BackendTransport`, `install()`, a `RuntimeBuilder` that boots connected, the hosted RPC proxies, the login and session owner, and the Jev ranker. The only crate allowed to depend on `tinyhumans-sdk`. | -| [`openhuman-cli`](openhuman-cli/README.md) | `openhuman-cli` | The `openhuman-core` binary, the developer and benchmark binaries, and every root `tests/*.rs` and `examples/*.rs` target. | +| [`openhuman-cli`](openhuman-cli/README.md) | `openhuman-cli` | The `openhuman-core` binary, the `openhuman-fleet` and `test-mcp-stub` ops binaries, and every root `tests/*.rs` and `examples/*.rs` target. | | [`openhuman-tui`](openhuman-tui/README.md) | `openhuman-tui` | The standalone terminal client, embedding the core in-process. | | [`openhuman-app`](openhuman-app/README.md) | `openhuman-app` (lib `openhuman`) | The thin Tauri v2 desktop host. Runs the core and its JSON-RPC server as a tokio task. Outside the root workspace. | ## How they layer Arrows point from a crate to what it depends on (normal `[dependencies]`, -taken from each [`Cargo.toml`](../Cargo.toml)). +taken from each [`Cargo.toml`](../Cargo.toml)). It is a strict chain: each crate +names only the layer directly below it. ```text - openhuman-app openhuman-tui openhuman-cli (executables) + openhuman-app openhuman-tui openhuman-cli (hosts) | | | +------------------+------------------+ - | each of the three depends on all three crates below - | - +-----------------------+-------------------------+ - | | | - v v | - openhuman-tinyhumans openhuman-rpc | - | | (server, client, | - | | session store) | - | v | | - | vendor/tinyhumans-sdk | | - v | | - openhuman-embed | | - | | | - v v v - +--------------------------------------------------------------+ - | openhuman-core | - | (domains, controller registry, BackendTransport port) | - +--------------------------------------------------------------+ + | openhuman-rpc only + v + openhuman-rpc host::{cli, desktop, tui}, server, + | client, session store + v + openhuman-tinyhumans SDK transport, hosted proxies, + | session owner, Jev ranker + | (+ vendor/tinyhumans-sdk) + v + openhuman-embed Runtime/RuntimeBuilder, presets, + | process helpers, facades + v + openhuman-core domains, controller registry, + BackendTransport port ``` The same edges as a list: -| Crate | Depends on (first-party) | +| Crate | Depends on (first-party, normal) | | --- | --- | | `openhuman-core` | none | | `openhuman-embed` | `openhuman-core` | -| `openhuman-rpc` | `openhuman-core` | -| `openhuman-tinyhumans` | `openhuman-embed`, `openhuman-core` (plus [`vendor/tinyhumans-sdk`](../vendor/tinyhumans-sdk/)) | -| `openhuman-cli` | `openhuman-core`, `openhuman-tinyhumans`, `openhuman-rpc` (`server`) | -| `openhuman-tui` | `openhuman-core`, `openhuman-rpc` (`session-store`), `openhuman-tinyhumans` | -| `openhuman-app` | `openhuman-core`, `openhuman-rpc` (`http-client`, `server`), `openhuman-tinyhumans` (`jev`) | - -Every executable depends on the core directly as well as through the layers, -because each names `openhuman_core::` paths. `openhuman-tinyhumans` also -depends on the core directly for surfaces embed does not re-export -(`backend::transport`, `core::all`). +| `openhuman-tinyhumans` | `openhuman-embed` (plus [`vendor/tinyhumans-sdk`](../vendor/tinyhumans-sdk/)) | +| `openhuman-rpc` | `openhuman-tinyhumans` | +| `openhuman-cli` | `openhuman-rpc` (`server`) | +| `openhuman-tui` | `openhuman-rpc` (`session-store`) | +| `openhuman-app` | `openhuman-rpc` (`http-client`, `server`, `jev`, the product gates) | + +[`scripts/ci/check-crate-chain.mjs`](../scripts/ci/check-crate-chain.mjs) (run by `pnpm rust:layout`) fails on any +other edge, and on a host `src/` that names `__host`, `core_host` or +`openhuman_core::`. The layers above embed reach core internals through +embed's doc-hidden `__host` list; the hosts use the curated facade re-exported +by rpc (`openhuman_rpc::embed`, `openhuman_rpc::tinyhumans`). Dev-dependencies +are exempt: `openhuman-cli`'s root tests and examples take the core, embed and +tinyhumans as dev-dependencies, which never reach the shipped binary. ## Why it is split this way The core runs agents, memory, tools and controllers without any hosted backend. It reaches the backend only through the `BackendTransport` port and knows nothing of JSON-RPC, so it can be embedded with neither. The layers -above add those pieces, and each host installs what it needs at startup: +above add those pieces, and each host boots through one `openhuman_rpc::host` +entry that connects them: ```text - host main() - openhuman_tinyhumans::install(..) backend transport, hosted - proxies, Jev ranker - openhuman_rpc::server::install_cli_server() (CLI: run/serve) - boot the core (run_core_from_args, CoreBuilder, embedded server) + openhuman-core binary openhuman_rpc::host::cli(args) + desktop app (GUI) openhuman_rpc::host::desktop(options, shutdown, ready_tx) + desktop app core/mcp openhuman_rpc::host::cli(args) + terminal UI openhuman_rpc::host::tui() + each: tinyhumans RuntimeBuilder::connect (transport, hosted proxies, + Jev ranker) -> embed Runtime (one per process) -> serve / dispatch ``` A core with no transport installed answers backend calls with @@ -89,8 +91,10 @@ empty, and the core must not depend on `openhuman-rpc`. Cargo default features define the contributor build; [`scripts/ci/product-features.txt`](../scripts/ci/product-features.txt) defines the shipped product. A core gate is forwarded along the library chain (`openhuman-embed`, then -`openhuman-tinyhumans`, then `openhuman-cli`), and the desktop app, which -builds with `default-features = false`, forwards product gates explicitly. +`openhuman-tinyhumans`, then `openhuman-rpc`, then the `openhuman-cli` and +`openhuman-tui` hosts), and the desktop app, which builds with +`default-features = false`, forwards product gates explicitly on its +`openhuman-rpc` dependency. [`scripts/ci/check-feature-forwarding.mjs`](../scripts/ci/check-feature-forwarding.mjs) checks both. ## Build and test diff --git a/crates/openhuman-app/Cargo.lock b/crates/openhuman-app/Cargo.lock index f6d936c37d9..49c2cf89cee 100644 --- a/crates/openhuman-app/Cargo.lock +++ b/crates/openhuman-app/Cargo.lock @@ -4590,9 +4590,7 @@ dependencies = [ "objc2-foundation 0.3.2", "objc2-user-notifications", "objc2-web-kit", - "openhuman", "openhuman-rpc", - "openhuman-tinyhumans", "parking_lot", "rand 0.9.5", "reqwest 0.12.28", @@ -4630,6 +4628,7 @@ dependencies = [ "async-trait", "log", "openhuman", + "sentry", "serde", "serde_json", "tempfile", diff --git a/crates/openhuman-app/Cargo.toml b/crates/openhuman-app/Cargo.toml index 323e2458de1..57147468366 100644 --- a/crates/openhuman-app/Cargo.toml +++ b/crates/openhuman-app/Cargo.toml @@ -34,16 +34,61 @@ tauri-build = { version = "2", features = [] } serde_json = "1" [dependencies] -openhuman-rpc = { path = "../openhuman-rpc", default-features = false, features = ["http-client", "server"] } -# The desktop session owner (login-token exchange, /auth/me, current-user -# cache) and the SDK-backed backend transport the core needs installed at -# boot. `default-features = false`: the shell controls every product gate -# through the `openhuman_core` line above, and this crate must not re-enable -# defaults behind `check-feature-forwarding`'s back. -# `jev` is that crate's own gate (not a core gate, so not in -# `scripts/ci/product-features.txt`): the Jev-backed `tool_search` ranker the -# product ships. -openhuman-tinyhumans = { path = "../openhuman-tinyhumans", default-features = false, features = ["jev"] } +# The ONLY openhuman crate this host names (chain: core -> embed -> tinyhumans +# -> rpc -> app/cli/tui; enforced by `scripts/ci/check-crate-chain.mjs`). The +# embedded core, the TinyHumans session owner and transport, and the embed +# facades (`openhuman_rpc::embed`, `openhuman_rpc::tinyhumans`) all come +# through it. +# +# `default-features = false` (set in #1061, before the compile-time domain +# gates existed) means the embedded core does NOT inherit the contributor +# default gate set, so every product gate is forwarded explicitly here, and +# `openhuman-rpc` forwards it down the chain to the core. +# +# This list is NOT optional polish — a gate missing here vanishes from the +# shipped app silently, with no build error and no test failure: +# +# - `voice` — without it the `#[cfg(feature = "voice")]` controllers in +# `crates/openhuman-core/src/core/all.rs` are never registered, so the whole +# `openhuman.voice_*` namespace answers "unknown method" at runtime. This +# shipped broken from v0.58.19 to v0.61.x (#4901); the `VOICE_COMPILED_IN` +# const assert at the top of `src/lib.rs` now fails the build if it is +# dropped again. +# - `http-server` — the shell reaches the in-process core only over +# http://127.0.0.1:/rpc (#5048). Enforced by the +# HTTP_SERVER_COMPILED_IN compile assert in lib.rs. +# - `scheduler-gate` — without it `require_ac_power` / `battery_floor` are +# silently unenforced. +# - `file-logging` — the packaged app's only durable log. +# - `hosting` — registration stays credential-gated in tools/ops.rs. +# +# `scripts/ci/check-feature-forwarding.mjs` fails CI when the product gates +# here drift from `scripts/ci/product-features.txt` (#4919) — do not +# hand-maintain it from memory. The three non-product entries are +# `openhuman-rpc`'s own: `http-client` (the relay client), `server` (the +# embedded JSON-RPC server and `host::desktop`) and `jev` (the Jev-backed +# `tool_search` ranker the product ships). +openhuman-rpc = { path = "../openhuman-rpc", default-features = false, features = [ + "http-client", + "server", + "jev", + "channels", + "media", + "inference", + "voice", + "web3", + "documents", + "modules", + "flows", + "skills", + "mcp", + "crash-reporting", + "http-server", + "scheduler-gate", + "file-logging", + "runtime-node", + "hosting", +] } # Tauri core and plugins. Use upstream Tauri's native WebView runtime (Wry), # rather than the removed Chromium/CEF fork. tauri = { version = "2.11", default-features = false, features = [ @@ -171,66 +216,6 @@ async-trait = "0.1" # in total. If mascot SVG rasterisation ever comes back, re-add them here # rather than reaching for a heavier image stack. -# Core domain logic, embedded in-process so the core's HTTP/JSON-RPC server -# runs as a tokio task inside the Tauri host. Avoids the orphan-sidecar class -# of bugs (PR #1061: Cmd+Q leaving `openhuman-core` and CEF helpers behind) -# by tying the core's lifetime to the GUI process. The existing port-7788 -# probe in `core_process::ensure_running` still attaches to a running -# `openhuman-core run` harness when one is already listening. -# -# `default-features = false` (set in #1061, before the compile-time domain -# gates existed) means the embedded core does NOT inherit the root crate's -# default gate set, so each default-ON gate must be forwarded explicitly to -# keep the shipped desktop build byte-identical (AGENTS.md "Compile-time -# domain gates"). -# -# This list is NOT optional polish — a gate missing here vanishes from the -# shipped app silently, with no build error and no test failure: -# -# - `voice` — without it the `#[cfg(feature = "voice")]` controllers in -# `crates/openhuman-core/src/core/all.rs` are never registered, so the whole `openhuman.voice_*` -# namespace answers "unknown method" at runtime. This shipped broken from -# v0.58.19 to v0.61.x (#4901); the `VOICE_COMPILED_IN` const assert at the -# top of `src/lib.rs` now fails the build if it is dropped again. -# - `media` — re-registers the `media_generate_*` agent tools that #4804 moved -# behind `#[cfg(feature = "media")]`; it sheds no deps, so this only restores -# the pre-gate desktop tool surface. -# - `web3` — keeps the wallet/web3/x402 domains and their agent tools in the -# desktop build while allowing slim builds to omit the crypto-only deps. -# `scripts/ci/check-feature-forwarding.mjs` fails CI when this list drifts from -# the core's `[features] default` (#4919) — do not hand-maintain it from memory. -openhuman_core = { path = "../openhuman-core", package = "openhuman", default-features = false, features = [ - "channels", - "media", - "inference", - "voice", - "web3", - "documents", - "modules", - "flows", - "skills", - "mcp", - "crash-reporting", - # The desktop shell reaches the in-process core only over - # http://127.0.0.1:/rpc, so it REQUIRES the HTTP + Socket.IO transport - # (#5048). Enforced by the HTTP_SERVER_COMPILED_IN compile assert in lib.rs. - "http-server", - # Without this the desktop app has no battery/AC probe, so a user who sets - # `require_ac_power` gets no enforcement and `battery_floor` throttling never - # fires — silently, since the off-state is a valid "on AC" reading. - "scheduler-gate", - # The packaged app's only durable log. Without this, a support request - # comes back with nothing to attach — and the absence is silent. - "file-logging", - "runtime-node", - # Declared "Default-OFF, product-ON" by its own gate comment in the root - # Cargo.toml, but it reached neither the product set nor this list, so the - # family was compiled in no configuration at all. Registration stays - # credential-gated in tools/ops.rs, so a host with no hosting credential - # sees no new tools. - "hosting", -] } - [target.'cfg(unix)'.dependencies] nix = { version = "0.29", default-features = false, features = ["hostname", "signal", "user"] } @@ -288,9 +273,9 @@ default = ["gateways"] # the tinybox crates that provision and reach those boxes. # # This is a **shell-local** gate, unrelated to the feature-forwarding rules in -# AGENTS.md: those govern which `openhuman_core` gates the shell forwards, and -# `scripts/ci/check-feature-forwarding.mjs` reads the core dependency's feature -# list, not this table. Nothing here belongs in `scripts/ci/product-features.txt`. +# AGENTS.md: those govern which product gates the shell forwards on its +# `openhuman-rpc` dependency, and `scripts/ci/check-feature-forwarding.mjs` +# reads that dependency's feature list, not this table. Nothing here belongs in `scripts/ci/product-features.txt`. gateways = [ "dep:tinybox-core", "dep:tinybox-docker", @@ -302,10 +287,10 @@ gateways = [ # turns this on automatically for release; do not put it in `default` or # every `pnpm dev:app` will silently load the production bundle. DO NOT REMOVE!! custom-protocol = ["tauri/custom-protocol"] -# Forwarded to the core crate to expose `openhuman.test_reset`. Off by -# default; the E2E build flips it on via `cargo tauri build --features -# e2e-test-support`. See app/scripts/e2e-build.sh. -e2e-test-support = ["openhuman_core/e2e-test-support"] +# Forwarded down the chain (rpc -> tinyhumans -> embed -> core) to expose +# `openhuman.test_reset`. Off by default; the E2E build flips it on via +# `cargo tauri build --features e2e-test-support`. See app/scripts/e2e-build.sh. +e2e-test-support = ["openhuman-rpc/e2e-test-support"] [patch."https://github.com/tinyhumansai/tinytools"] # tinymcp's `tools` feature names tinytools by git rev so it can be built on its diff --git a/crates/openhuman-app/README.md b/crates/openhuman-app/README.md index c062a83c155..ba31d956f27 100644 --- a/crates/openhuman-app/README.md +++ b/crates/openhuman-app/README.md @@ -34,7 +34,7 @@ For the user-facing tour of windows, tray and data flow, see | v +-------------+--------------+ | | +---------------------------+ | | | | embedded core server |<------------------------+ | -| | openhuman_rpc::server | shell-side callers (session link, | +| | openhuman_rpc::host | shell-side callers (session link, | | | (tokio task, 127.0.0.1) | iMessage scanner) use the same HTTP | | | openhuman_core domains | | | +---------------------------+ | @@ -42,21 +42,22 @@ For the user-facing tour of windows, tray and data flow, see ``` There is no sidecar binary. `core_process::CoreProcessHandle` owns a tokio task -running `openhuman_rpc::server::run_server_embedded_with_ready`, so the core -lives and dies with the window. The handle generates a 256-bit hex bearer per +running `openhuman_rpc::host::desktop`, so the core lives and dies with the +window. That task owns the process's one embed runtime; a restart waits for +the old task to drop it before spawning the next. The handle generates a 256-bit hex bearer per launch (`generate_rpc_token`) and hands it to the embedded server in memory, not through the environment. The frontend gets it back with the `core_rpc_endpoint` command. ### Boot sequence -`main.rs` runs first. It calls `openhuman_tinyhumans::install` so the core has -a backend transport (the core carries none of its own), then looks at -`argv[1]`: +`main.rs` runs first and looks at `argv[1]`. It installs nothing itself: +every path boots through an `openhuman_rpc::host` entry, which connects the +TinyHumans backend transport (the core carries none of its own). -- `OpenHuman core ` goes to `run_core_from_args`, which installs the - JSON-RPC server (`openhuman_rpc::server::install_cli_server`) and dispatches - into the core CLI. On Windows the process reattaches to the parent console +- `OpenHuman core ` goes to `run_core_from_args`, which calls + `openhuman_rpc::host::cli` (the same entry as the `openhuman-core` binary: + connected, with the JSON-RPC server behind `run` / `serve`). On Windows the process reattaches to the parent console first so output appears in the shell. - `OpenHuman mcp` and `OpenHuman mcp-server` do the same, which makes the app binary a stdio MCP server for clients such as the Claude Code CLI. @@ -65,17 +66,17 @@ a backend transport (the core carries none of its own), then looks at `run()` in `lib.rs` then does, in order: 1. Builds the Tauri context (on Windows it drops native decorations for the - custom titlebar), neutralizes a broken parent stderr - (`stderr_panic_hook`), and installs the backend transport again in case - the library is entered without `main.rs`. -2. Replaces Tauri's async runtime with a multi-thread tokio runtime that uses - the core's `AGENT_WORKER_STACK_BYTES` stack size and - `MAX_BLOCKING_THREADS`, so agent turns started from commands do not - overflow the default stack. -3. Initializes Sentry (DSN from `OPENHUMAN_TAURI_SENTRY_DSN`), with a - `before_send` filter that drops dev-server fetch noise and the core's - known transient classes (`openhuman_core::core::observability::is_*`) and - tags the signed-in user id from `session::peek_user_id`. Then the stderr + custom titlebar) and neutralizes a broken parent stderr + (`stderr_panic_hook`). +2. Replaces Tauri's async runtime with `openhuman_rpc::embed::process::tokio_runtime()`, + a multi-thread runtime with the core's `AGENT_WORKER_STACK_BYTES` stack + size and `MAX_BLOCKING_THREADS`, so agent turns started from commands do + not overflow the default stack. +3. Initializes Sentry (DSN from `OPENHUMAN_TAURI_SENTRY_DSN`) with + `embed::process::sentry::client_options`: the shared `before_send` chain + (dev-server fetch noise, the core's known transient classes, hostname + stripping, secret scrubbing) with the signed-in user id from + `session::peek_user_id` as the fallback. Then the stderr panic hook, file logging (`file_logging::init`), and the Linux display and WSL checks. 4. Single-instance guards that must run before any window exists. On Windows @@ -198,7 +199,7 @@ Updates, reset and diagnostics: | [`src/app_update.rs`](src/app_update.rs) | Bounded retry policy for the updater download. The update commands themselves are in `lib.rs`. | | [`src/local_data_reset.rs`](src/local_data_reset.rs) | `reset_local_data`: asks the core which paths to remove, shuts the core down so its file handles close, removes the active user's local data, and starts the core again. | | [`src/reset_reboot_schedule.rs`](src/reset_reboot_schedule.rs) | Windows: schedules deletion at next reboot when files are locked during a reset. | -| [`src/file_logging.rs`](src/file_logging.rs) | Resolves the data dir and calls `openhuman_core::core::logging::init_for_embedded`; `reveal_logs_folder`, `logs_folder_path`. | +| [`src/file_logging.rs`](src/file_logging.rs) | Resolves the data dir and calls `openhuman_rpc::embed::process::init_for_embedded`; `reveal_logs_folder`, `logs_folder_path`. | | [`src/stderr_panic_hook.rs`](src/stderr_panic_hook.rs) | Stops a closed parent stderr pipe from turning log writes into panics. | Other commands: @@ -216,7 +217,7 @@ Configuration and packaging: | [`tauri.conf.json`](tauri.conf.json) | Windows, bundle resources (agent prompts and `bundled-modules`), updater and installer settings. | | [`capabilities/`](capabilities/) | The capability granted to the `main` and `overlay` windows. See [`capabilities/`](capabilities/README.md). | | [`permissions/`](permissions/) | App permission sets referenced by the capability. See [`permissions/`](permissions/README.md). | -| [`bundled-modules/`](bundled-modules/) | Installer resource directory for native module releases. Empty in git (only `.gitkeep`; everything else is ignored). Release builds fill it with [`scripts/release/stage-modules.mjs`](../../scripts/release/stage-modules.mjs), laid out as `///`, and `setup()` hands it to `openhuman_core::modules::ops::set_bundled_releases_dir`. Its contents still pass the core's digest and TinyBus admission checks. On macOS, [`scripts/release/macos-bundled-modules.sh`](../../scripts/release/macos-bundled-modules.sh) signs and checks it. | +| [`bundled-modules/`](bundled-modules/) | Installer resource directory for native module releases. Empty in git (only `.gitkeep`; everything else is ignored). Release builds fill it with [`scripts/release/stage-modules.mjs`](../../scripts/release/stage-modules.mjs), laid out as `///`, and `setup()` hands it to `openhuman_rpc::embed::modules::set_bundled_releases_dir`. Its contents still pass the core's digest and TinyBus admission checks. On macOS, [`scripts/release/macos-bundled-modules.sh`](../../scripts/release/macos-bundled-modules.sh) signs and checks it. | | [`build.rs`](build.rs) | Runs `tauri_build`, and empties `bundle.resources` for non-release builds. | | [`profiling/`](profiling/) | Standalone CPU and RAM profiler for a running app. See [`profiling/`](profiling/README.md). | | `Info.plist`, `entitlements.sidecar.plist`, `nsis-hooks.nsh`, `main.desktop`, `postinst`, `postrm` | Platform packaging files for macOS, the Windows installer, and Linux packages. | @@ -273,9 +274,11 @@ pnpm dev:app # Vite dev server + this crate pnpm build # production bundle ``` -The core dependency is `openhuman_core = { path = "../openhuman-core", -package = "openhuman", default-features = false, features = [...] }`. Because -default features are off, every product gate must be listed by hand: +The only OpenHuman dependency is `openhuman-rpc = { path = "../openhuman-rpc", +default-features = false, features = [...] }` +(`scripts/ci/check-crate-chain.mjs` enforces it); it forwards each gate down +the chain to the core. Because default features are off, every product gate +must be listed by hand: `channels`, `media`, `inference`, `voice`, `web3`, `documents`, `modules`, `flows`, `skills`, `mcp`, `crash-reporting`, `http-server`, `scheduler-gate`, `file-logging`, `runtime-node`, `hosting`. A gate missing here disappears from @@ -285,10 +288,11 @@ compares the list with [`scripts/ci/product-features.txt`](../../scripts/ci/prod `HTTP_SERVER_COMPILED_IN`) that fail the build if `voice` or `http-server` is dropped. -Other dependencies: `openhuman-rpc` with `http-client` and `server` (the -embedded server and the relay), `openhuman-tinyhumans` with `jev` (backend -transport and the session owner), and the `tinybox-*` crates behind -`gateways`. +The same line turns on rpc's own `http-client` and `server` (the relay and the +embedded server) and `jev` (the Jev ranker). The session owner, the embed +facades and the backend transport come through it as +`openhuman_rpc::tinyhumans` and `openhuman_rpc::embed`. The `tinybox-*` crates +sit behind `gateways`. The `[patch]` tables mirror the root [`Cargo.toml`](Cargo.toml) for `tinytools`, the `tinyinference-*` crates, `tinyflows` and `tinychannels`. Keep them in sync: @@ -306,7 +310,7 @@ These are shell-local and unrelated to the core feature forwarding above. | --- | --- | | `gateways` (default) | Compiles in [`src/gateway/`](src/gateway/) and the `tinybox-*` crates. Off, the gateway commands are absent and `active_rpc_endpoint` always answers with the embedded core. | | `custom-protocol` | Serves the bundled `frontendDist` from `tauri://localhost` instead of the Vite `devUrl`. `cargo tauri build` turns it on; never add it to `default`. | -| `e2e-test-support` | Forwards `openhuman_core/e2e-test-support` to expose `openhuman.test_reset`. The E2E build ([`app/scripts/e2e-build.sh`](../../app/scripts/e2e-build.sh)) enables it. | +| `e2e-test-support` | Forwards `openhuman-rpc/e2e-test-support` (down the chain to the core) to expose `openhuman.test_reset`. The E2E build ([`app/scripts/e2e-build.sh`](../../app/scripts/e2e-build.sh)) enables it. | ## Boundaries diff --git a/crates/openhuman-app/profiling/README.md b/crates/openhuman-app/profiling/README.md index c14b90708b0..5c070805345 100644 --- a/crates/openhuman-app/profiling/README.md +++ b/crates/openhuman-app/profiling/README.md @@ -83,7 +83,8 @@ pnpm profile:tauri --pid --duration 15 For the smaller embedded-core-only Linux RSS and PSS benchmark, use the `rss-bench` binary in the `profile/` crate of -[openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks), from a checkout of that repository: +[openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) (#6944); it no longer builds from +`crates/openhuman-cli`. From a checkout of that repository: ```bash ./profile/scripts/rss-bench.sh diff --git a/crates/openhuman-app/src/artifact_commands.rs b/crates/openhuman-app/src/artifact_commands.rs index d81d7da545b..80ba28622f0 100644 --- a/crates/openhuman-app/src/artifact_commands.rs +++ b/crates/openhuman-app/src/artifact_commands.rs @@ -31,14 +31,14 @@ use std::path::{Path, PathBuf}; /// in the given workspace. Isolated from config loading for unit testing. async fn resolve_source( workspace_dir: &Path, - roots: &openhuman_core::agent::artifacts::FileRoots, + roots: &openhuman_rpc::embed::artifacts::FileRoots, artifact_id: &str, ) -> Result { let artifact_id = artifact_id.trim(); if artifact_id.is_empty() { return Err("artifact_id must not be empty".to_string()); } - openhuman_core::agent::artifacts::resolve_ready_file(workspace_dir, roots, artifact_id).await + openhuman_rpc::embed::artifacts::resolve_ready_file(workspace_dir, roots, artifact_id).await } /// Copy `source` to `dest`, returning the byte count. Isolated so it is @@ -62,8 +62,8 @@ pub async fn download_artifact_to_downloads( if artifact_id.trim().is_empty() { return Err("artifact_id must not be empty".to_string()); } - let config = openhuman_core::config::rpc::load_config_with_timeout().await?; - let roots = openhuman_core::agent::artifacts::FileRoots::from_config(&config); + let config = openhuman_rpc::embed::config::load_config_with_timeout().await?; + let roots = openhuman_rpc::embed::artifacts::FileRoots::from_config(&config); let source = resolve_source(&config.workspace_dir, &roots, &artifact_id).await?; if filename.trim().is_empty() { return Err("filename must not be empty".to_string()); diff --git a/crates/openhuman-app/src/artifact_commands_tests.rs b/crates/openhuman-app/src/artifact_commands_tests.rs index 0da3a9aec26..b5f7ac46042 100644 --- a/crates/openhuman-app/src/artifact_commands_tests.rs +++ b/crates/openhuman-app/src/artifact_commands_tests.rs @@ -1,5 +1,5 @@ use super::*; -use openhuman_core::agent::artifacts::FileRoots; +use openhuman_rpc::embed::artifacts::FileRoots; #[test] fn sanitize_rejects_path_separators() { @@ -22,7 +22,7 @@ fn sanitize_accepts_plain_names() { } async fn ready_artifact(workspace: &Path, files_dir: &Path) -> String { - use openhuman_core::agent::artifacts::{create_artifact, finalize_artifact, ArtifactKind}; + use openhuman_rpc::embed::artifacts::{create_artifact, finalize_artifact, ArtifactKind}; let (meta, path) = create_artifact( workspace, files_dir, @@ -75,7 +75,7 @@ async fn resolve_source_rejects_unknown_ids_and_paths() { #[tokio::test] async fn resolve_source_refuses_a_file_the_store_does_not_vouch_for() { - use openhuman_core::agent::artifacts::{ArtifactKind, ArtifactMeta, ArtifactStatus}; + use openhuman_rpc::embed::artifacts::{ArtifactKind, ArtifactMeta, ArtifactStatus}; let temp = tempfile::tempdir().unwrap(); let secret = temp.path().join("secret.txt"); std::fs::write(&secret, b"private").unwrap(); @@ -110,7 +110,7 @@ async fn resolve_source_refuses_a_file_the_store_does_not_vouch_for() { /// refused because that root is not one of the vouched-for files folders. #[tokio::test] async fn resolve_source_refuses_a_record_that_claims_its_own_root() { - use openhuman_core::agent::artifacts::{ArtifactKind, ArtifactMeta, ArtifactStatus}; + use openhuman_rpc::embed::artifacts::{ArtifactKind, ArtifactMeta, ArtifactStatus}; let temp = tempfile::tempdir().unwrap(); let home = temp.path().join("home"); std::fs::create_dir_all(&home).unwrap(); diff --git a/crates/openhuman-app/src/core_process.rs b/crates/openhuman-app/src/core_process.rs index 0f8f1fad930..b253fef4ad2 100644 --- a/crates/openhuman-app/src/core_process.rs +++ b/crates/openhuman-app/src/core_process.rs @@ -41,10 +41,14 @@ const CORE_READY_POLL_MS: u64 = 100; // aborts under contention. const CORE_READY_ATTEMPTS: usize = 600; const CORE_READY_TIMEOUT_MS: u64 = CORE_READY_POLL_MS * CORE_READY_ATTEMPTS as u64; +/// How long `abort_task` waits for an aborted server task to drop its +/// runtime. An abort lands at the task's next poll, so this is normally +/// instant; the bound only matters if the task is stuck in blocking code. +const ABORT_DRAIN_SECS: u64 = 2; /// Generate a 256-bit cryptographically-random bearer token as a hex string. /// -/// Uses the same encoding as `openhuman_core::core::auth::generate_token` +/// Uses the same encoding as the core's `core::auth::generate_token` /// (`hex::encode`) so the token format never silently diverges between the /// Tauri-side generator and the core-side validator. pub fn generate_rpc_token() -> String { @@ -76,8 +80,8 @@ pub struct CoreProcessHandle { last_port_fallback: Arc>>, /// Bearer token the embedded server validates on every inbound request. /// - /// Handed to the embedded server **in-memory** (via the `rpc_token` - /// argument of [`openhuman_rpc::server::run_server_embedded_with_ready`]) + /// Handed to the embedded server **in-memory** (via + /// [`openhuman_rpc::host::DesktopOptions::rpc_token`]) /// rather than through `OPENHUMAN_CORE_TOKEN` on the process environment. /// Avoiding the env crossing keeps the bearer off `/proc//environ` /// (Linux) and out of `sysctl KERN_PROCARGS2` / `ps eww -p ` (macOS) @@ -91,8 +95,8 @@ impl CoreProcessHandle { pub fn new(port: u16) -> Self { // CURRENT_RPC_TOKEN is intentionally NOT set here. It is published by // ensure_running() only after the embedded server has been spawned - // with this token handed over via the in-memory `rpc_token` arg of - // `run_server_embedded_with_ready`. Setting it here would advertise + // with this token handed over in memory (`DesktopOptions::rpc_token` + // of `openhuman_rpc::host::desktop`). Setting it here would advertise // a token that an existing process listening on the port (the // harness-attach fast-path) has never seen, causing 401s on every // authenticated call. @@ -224,7 +228,7 @@ impl CoreProcessHandle { let mut retry_after_takeover = false; let shutdown_token = self.fresh_shutdown_token().await; let (ready_tx, mut ready_rx) = - tokio::sync::oneshot::channel::(); + tokio::sync::oneshot::channel::(); let mut received_ready = false; { @@ -232,8 +236,7 @@ impl CoreProcessHandle { if guard.is_none() { let port = self.preferred_port; // RPC bearer is handed to the embedded server in-memory - // via the `rpc_token` argument of - // run_server_embedded_with_ready (see below) — never + // via `DesktopOptions::rpc_token` (see below) — never // through OPENHUMAN_CORE_TOKEN on the process env. // Sidecar-era env-var transport was a leftover from the // PR #1061 cleanup; with the core in-process there is no @@ -290,21 +293,31 @@ impl CoreProcessHandle { log::info!( "[core] spawning embedded in-process core server on preferred port {port}" ); + // The shared desktop host boot: the `desktop` embed preset, + // connected to the TinyHumans backend (transport, hosted + // proxies, Jev ranker), the on-disk session store, every + // background service, then the listener — taking over a + // stale listener of our own on the preferred port, else + // falling back — and `ready_tx` once bound. + // + // One embed runtime per process: the runtime this task + // builds is dropped when the task ends, which is what + // frees the slot for the next spawn. Every path that + // replaces this task (`shutdown`, `abort_task`) therefore + // waits for the old future to be dropped first. + let options = openhuman_rpc::host::DesktopOptions { + host: None, + port: Some(port), + socketio: true, + // In-memory bearer handoff: the embedded server seeds + // its auth subsystem from this value, so the token + // never crosses OPENHUMAN_CORE_TOKEN on the process + // env. + rpc_token: Some(token_for_core), + }; + log::debug!("[core] host::desktop options={options:?}"); let task = tokio::spawn(async move { - openhuman_rpc::server::run_server_embedded_with_ready( - None, - Some(port), - true, - shutdown_token, - ready_tx, - // In-memory bearer handoff: the embedded server - // seeds its auth subsystem from this value via - // `auth::init_rpc_token_with_value`, so the token - // never crosses OPENHUMAN_CORE_TOKEN on the - // process env. - Some(token_for_core), - ) - .await + openhuman_rpc::host::desktop(options, shutdown_token, ready_tx).await }); *guard = Some(task); // Publish only after the embedded server has been spawned @@ -369,8 +382,13 @@ impl CoreProcessHandle { .to_string()) } Ok(Err(err)) => { - if let Some(openhuman_core::platform::connectivity::rpc::PickListenPortError::WouldTakeOver { preferred, .. }) = err - .downcast_ref::() + if let Some( + openhuman_rpc::embed::PickListenPortError::WouldTakeOver { + preferred, + .. + }, + ) = + err.downcast_ref::() { if startup_attempt == 0 { log::warn!( @@ -451,7 +469,7 @@ impl CoreProcessHandle { aborting embedded startup task before retry" ); self.cancel_shutdown_token(" after startup timeout").await; - self.abort_task(" after startup timeout").await; + self.abort_task(" after startup timeout", true).await; format!( "core process did not become ready within {CORE_READY_TIMEOUT_MS}ms \ (port={port}, ready_signal={received_ready}, port_open={port_open}, \ @@ -461,7 +479,7 @@ impl CoreProcessHandle { pub(crate) fn apply_embedded_ready_signal( &self, - ready: openhuman_rpc::server::EmbeddedReadySignal, + ready: openhuman_rpc::host::EmbeddedReadySignal, ) { *self.active_port.write() = ready.port; std::env::set_var("OPENHUMAN_CORE_RPC_URL", self.rpc_url()); @@ -617,11 +635,32 @@ impl CoreProcessHandle { /// Lock the task slot, take its handle if any, and abort it. Shared by /// `shutdown` (cleanup-on-drop semantics) and `send_terminate_signal` /// (cooperative early teardown from `RunEvent::ExitRequested`). - async fn abort_task(&self, log_context: &str) { - let mut task_guard = self.task.lock().await; - if let Some(task) = task_guard.take() { - log::info!("[core] aborting embedded core server task{log_context}"); - task.abort(); + /// + /// With `wait_release`, waits (bounded) for the aborted task to actually + /// finish: aborting only marks it, and its future — which owns the embed + /// runtime — is dropped at the task's next poll. The process holds one + /// embed runtime at a time, so a respawn that raced that drop would fail + /// to build its own. App shutdown passes `false`: nothing respawns, and + /// the UI thread should not wait. + async fn abort_task(&self, log_context: &str, wait_release: bool) { + let task = { + let mut task_guard = self.task.lock().await; + task_guard.take() + }; + let Some(task) = task else { + return; + }; + log::info!("[core] aborting embedded core server task{log_context}"); + task.abort(); + if !wait_release { + return; + } + match timeout(Duration::from_secs(ABORT_DRAIN_SECS), task).await { + Ok(_) => log::debug!("[core] aborted embedded core server task released{log_context}"), + Err(_) => log::warn!( + "[core] aborted embedded core server task did not release within \ + {ABORT_DRAIN_SECS}s{log_context}; a respawn may find the runtime slot taken" + ), } } @@ -694,7 +733,7 @@ impl CoreProcessHandle { pub async fn send_terminate_signal(&self) { self.cancel_shutdown_token(" on app shutdown").await; self.drain_task_briefly().await; - self.abort_task(" on app shutdown").await; + self.abort_task(" on app shutdown", false).await; } /// Wait a bounded moment for the server task to finish on its own after diff --git a/crates/openhuman-app/src/core_process_tests.rs b/crates/openhuman-app/src/core_process_tests.rs index 0d26d842382..94264434b47 100644 --- a/crates/openhuman-app/src/core_process_tests.rs +++ b/crates/openhuman-app/src/core_process_tests.rs @@ -28,8 +28,8 @@ fn env_lock() -> MutexGuard<'static, ()> { fn core_test_runtime() -> tokio::runtime::Runtime { tokio::runtime::Builder::new_multi_thread() .enable_all() - .thread_stack_size(openhuman_core::core::runtime::AGENT_WORKER_STACK_BYTES) - .max_blocking_threads(openhuman_core::core::runtime::MAX_BLOCKING_THREADS) + .thread_stack_size(openhuman_rpc::embed::process::AGENT_WORKER_STACK_BYTES) + .max_blocking_threads(openhuman_rpc::embed::process::MAX_BLOCKING_THREADS) .build() .expect("build core test runtime") } @@ -82,7 +82,7 @@ fn core_process_handle_new_creates_instance() { #[test] fn ready_signal_updates_runtime_port_and_fallback_notice() { let handle = CoreProcessHandle::new(7788); - handle.apply_embedded_ready_signal(openhuman_rpc::server::EmbeddedReadySignal { + handle.apply_embedded_ready_signal(openhuman_rpc::host::EmbeddedReadySignal { port: 7789, fallback_from: Some(7788), }); @@ -102,8 +102,8 @@ fn ready_signal_updates_runtime_port_and_fallback_notice() { /// Regression: `ensure_running` must NOT publish the per-launch RPC bearer /// to the `OPENHUMAN_CORE_TOKEN` environment variable. /// -/// The bearer is now handed to the in-process core in-memory via the -/// `rpc_token` argument of `run_server_embedded_with_ready`; setting it on +/// The bearer is now handed to the in-process core in-memory via +/// `DesktopOptions::rpc_token` of `openhuman_rpc::host::desktop`; setting it on /// the process env would put it within reach of any same-UID process /// reading `/proc//environ` (Linux) or `sysctl KERN_PROCARGS2` / /// `ps eww -p ` (macOS). diff --git a/crates/openhuman-app/src/file_logging.rs b/crates/openhuman-app/src/file_logging.rs index 1e1c77d6adb..638fb9a4905 100644 --- a/crates/openhuman-app/src/file_logging.rs +++ b/crates/openhuman-app/src/file_logging.rs @@ -2,7 +2,7 @@ //! //! Resolves the OpenHuman data directory the same way the core does //! (`~/.openhuman` or `OPENHUMAN_WORKSPACE` override) and hands it to -//! [`openhuman_core::core::logging::init_for_embedded`], which installs a +//! [`openhuman_rpc::embed::process::init_for_embedded`], which installs a //! daily-rotated file appender so packaged GUI builds — where stderr is //! invisible — still produce a log users can share for support. //! @@ -11,7 +11,7 @@ use std::path::PathBuf; -use openhuman_core::core::logging::{self, log_directory}; +use openhuman_rpc::embed::process::{self as logging, log_directory}; /// Initialize logging for the Tauri shell + embedded core. Idempotent and /// safe to call from any startup position; the underlying `Once` guard means @@ -41,7 +41,7 @@ pub(crate) fn resolve_data_dir() -> PathBuf { return PathBuf::from(workspace); } } - openhuman_core::config::default_root_openhuman_dir().unwrap_or_else(|err| { + openhuman_rpc::embed::config::default_root_openhuman_dir().unwrap_or_else(|err| { eprintln!( "[file_logging] default_root_openhuman_dir failed ({err}); falling back to temp dir" ); diff --git a/crates/openhuman-app/src/file_logging_tests.rs b/crates/openhuman-app/src/file_logging_tests.rs index 9b832e9a4d4..63fbba70999 100644 --- a/crates/openhuman-app/src/file_logging_tests.rs +++ b/crates/openhuman-app/src/file_logging_tests.rs @@ -50,7 +50,7 @@ fn reveal_logs_folder_errors_when_uninitialized() { // If logging hasn't been initialized, the command must surface a // typed error so the UI can show it instead of silently launching // an `open` against an empty path. - if openhuman_core::core::logging::log_directory().is_none() { + if openhuman_rpc::embed::process::log_directory().is_none() { let err = reveal_logs_folder().expect_err("must error pre-init"); assert!(err.contains("not initialized"), "unexpected error: {err}"); } diff --git a/crates/openhuman-app/src/lib.rs b/crates/openhuman-app/src/lib.rs index db415cb4a07..3c00f868a06 100644 --- a/crates/openhuman-app/src/lib.rs +++ b/crates/openhuman-app/src/lib.rs @@ -1,8 +1,11 @@ //! Desktop host for OpenHuman: Tauri v2 + Wry, targeting Windows, macOS, and //! Linux. //! -//! `openhuman_core` is linked in-process; its JSON-RPC server runs as a -//! tokio task (`core_process`) instead of a spawned sidecar. The renderer +//! The core is linked in-process; its JSON-RPC server runs as a tokio task +//! (`core_process`, booted through `openhuman_rpc::host::desktop`) instead of +//! a spawned sidecar. `openhuman-rpc` is this crate's only openhuman +//! dependency: the embed facades and the TinyHumans session owner are reached +//! as `openhuman_rpc::embed` and `openhuman_rpc::tinyhumans`. The renderer //! reaches it over `http://127.0.0.1:/rpc`, using the per-launch //! bearer returned by the `core_rpc_token` command. //! @@ -12,7 +15,7 @@ //! //! The Cargo features `gateways`, `custom-protocol`, `e2e-test-support`, and //! `sandbox-bubblewrap` are shell-local and not part of the product feature -//! list; the `openhuman_core` product gates are forwarded explicitly in +//! list; the product gates are forwarded explicitly on `openhuman-rpc` in //! `Cargo.toml` and guarded by the `VOICE_COMPILED_IN` / //! `HTTP_SERVER_COMPILED_IN` compile-time asserts below. //! @@ -30,36 +33,37 @@ compile_error!("src-tauri host supports desktop (Windows/macOS/Linux) only. Mobi // The shipped desktop app must always embed the real voice domain. Cargo // features are per-crate, so `#[cfg(feature = "voice")]` here would test THIS // crate's features, not the core's — a voice-less core is only observable via -// the core's own always-compiled facade. Without this assert the failure is -// silent and runtime-only: every `openhuman.voice_*` RPC answers "unknown -// method" and the UI blames a stale sidecar (#4901). Keep `voice` in the -// `openhuman_core` feature list in Cargo.toml to satisfy this. +// the core's own always-compiled facade (re-exported by embed). Without this +// assert the failure is silent and runtime-only: every `openhuman.voice_*` RPC +// answers "unknown method" and the UI blames a stale sidecar (#4901). Keep +// `voice` in the `openhuman-rpc` feature list in Cargo.toml to satisfy this. const _: () = assert!( - openhuman_core::voice::VOICE_COMPILED_IN, - "openhuman_core must be built with the `voice` feature: the desktop app ships voice, \ + openhuman_rpc::embed::VOICE_COMPILED_IN, + "the core must be built with the `voice` feature: the desktop app ships voice, \ and without it every openhuman.voice_* controller is unregistered (#4901). \ - Add \"voice\" to the openhuman_core `features` list in crates/openhuman-app/Cargo.toml." + Add \"voice\" to the openhuman-rpc `features` list in crates/openhuman-app/Cargo.toml." ); // The shell talks to the in-process core only over http://127.0.0.1:/rpc, // so the core MUST embed the HTTP + Socket.IO transport (#5048). Same failure // class as #4901: with `http-server` dropped the core never binds a listener // and every RPC is unreachable — silent and runtime-only. The marker lives in -// the core's always-compiled facade (`core::http_server_status`) precisely so -// this assert can observe the core's feature state (a dependent's own -// `#[cfg(feature = ...)]` would test THIS crate's features, not the core's). +// the core's always-compiled facade (`core::http_server_status`, re-exported by +// embed) precisely so this assert can observe the core's feature state (a +// dependent's own `#[cfg(feature = ...)]` would test THIS crate's features, not +// the core's). const _: () = assert!( - openhuman_core::core::http_server_status::HTTP_SERVER_COMPILED_IN, - "openhuman_core must be built with the `http-server` feature: the desktop app reaches \ + openhuman_rpc::embed::HTTP_SERVER_COMPILED_IN, + "the core must be built with the `http-server` feature: the desktop app reaches \ the core only over http://127.0.0.1:/rpc, and without it the core binds no \ listener so every RPC is unreachable (#5048). \ - Add \"http-server\" to the openhuman_core `features` list in crates/openhuman-app/Cargo.toml." + Add \"http-server\" to the openhuman-rpc `features` list in crates/openhuman-app/Cargo.toml." ); // The desktop shell runs the same in-process core as the CLI. Keep its module // loader and TinyComputer browser adapter compiled in; the verified release module // is resolved by that core at first use. -const _: &str = openhuman_core::modules::browser::MODULE_ID; +const _: &str = openhuman_rpc::embed::modules::browser::MODULE_ID; mod app_update; // Artifact export command (#2779) — cross-platform Downloads copy. The `rfd` @@ -507,9 +511,9 @@ async fn restart_app(app: tauri::AppHandle) -> Result<(), String> { /// `OPENHUMAN_WORKSPACE` overrides used in test harnesses. (#900) #[tauri::command] fn get_active_user_id() -> Result, String> { - let root = openhuman_core::config::default_root_openhuman_dir() + let root = openhuman_rpc::embed::config::default_root_openhuman_dir() .map_err(|err| format!("resolve active-user state directory: {err}"))?; - Ok(openhuman_core::config::read_active_user_id(&root)) + Ok(openhuman_rpc::embed::config::read_active_user_id(&root)) } /// Information about an available shell-app update returned to the frontend. @@ -2372,14 +2376,10 @@ pub fn run() { // stderr is a console or file. See `stderr_panic_hook`. stderr_panic_hook::neutralize_broken_parent_stderr(); - // The in-process core reaches the hosted backend only through the - // transport `openhuman-tinyhumans` installs. `main.rs` installs it before - // dispatching here; this call is idempotent and covers embedders of - // `run()` that skip `main.rs`. - if let Err(err) = openhuman_tinyhumans::install(openhuman_tinyhumans::InstallOptions::default()) - { - log::error!("[boot] TinyHumans backend transport unavailable: {err}"); - } + // No `openhuman_tinyhumans::install` here any more: the embedded core is + // booted through `openhuman_rpc::host::desktop` (see `core_process`), + // which connects the TinyHumans backend transport, the hosted RPC + // proxies and the Jev ranker as part of building its runtime. // Must run before any GTK/CEF code that could trigger X calls — otherwise // Xlib's default handler calls exit(1) on the first BadWindow and we never @@ -2391,7 +2391,7 @@ pub fn run() { // Tauri's default async runtime uses tokio multi-thread workers with // a ~2 MB stack. The in-process core (spawned by // `core_process::CoreProcessHandle::ensure_running` via - // `tokio::spawn(run_server_embedded(..))`) runs *on* that runtime, so + // `tokio::spawn(openhuman_rpc::host::desktop(..))`) runs *on* that runtime, so // every JSON-RPC handler — including the deep tower // `web channel chat → orchestrator turn → integration action tool // → composio execute → load_config_with_timeout` (and, at the time, the @@ -2410,18 +2410,15 @@ pub fn run() { // stack overflow` once an orchestrator delegated. PR #3155 raised the // standalone server to 16 MiB; the desktop Tauri host is the *same* // tower running on a *different* runtime and needs the same headroom. - // Share the constant with the rest of `crates/openhuman-core/src/core/*` via - // [`openhuman_core::core::runtime::AGENT_WORKER_STACK_BYTES`] so all - // multi-thread runtimes that may host an agent turn stay in sync. + // `embed::process::tokio_runtime` sizes the workers with the core's + // `AGENT_WORKER_STACK_BYTES` (and caps blocking threads at + // `MAX_BLOCKING_THREADS`) so all multi-thread runtimes that may host an + // agent turn stay in sync. // // Must happen before any `tauri::async_runtime::*` call, otherwise // `set(...)` panics with "runtime already initialized". { - let custom_runtime = tokio::runtime::Builder::new_multi_thread() - .enable_all() - .thread_stack_size(openhuman_core::core::runtime::AGENT_WORKER_STACK_BYTES) - .max_blocking_threads(openhuman_core::core::runtime::MAX_BLOCKING_THREADS) - .build() + let custom_runtime = openhuman_rpc::embed::process::tokio_runtime() .expect("build custom tokio runtime for tauri async surface"); let handle = custom_runtime.handle().clone(); // Tauri docs: "you cannot drop the underlying TokioRuntime." @@ -2441,193 +2438,29 @@ pub fn run() { // `tauri::cef_entry_point`) and the `OpenHuman core …` in-process core // path do NOT spin up a second client — those have their own reporting // surfaces. - let _sentry_guard = sentry::init(sentry::ClientOptions { - dsn: std::env::var("OPENHUMAN_TAURI_SENTRY_DSN") - .ok() - .filter(|s| !s.is_empty()) - .or_else(|| option_env!("OPENHUMAN_TAURI_SENTRY_DSN").map(|s| s.to_string())) - .filter(|s| !s.is_empty()) - .and_then(|s| s.parse().ok()), - release: Some(std::borrow::Cow::Owned(build_sentry_release_tag())), - environment: Some(std::borrow::Cow::Owned(resolve_sentry_environment())), - send_default_pii: false, - before_send: Some(std::sync::Arc::new(|mut event| { - // Drop "dev-server fetch failed" noise: the vendored - // `tauri-runtime-cef` dev proxy - // (vendor/tauri-cef/crates/tauri/src/protocol/tauri.rs) calls - // `log::error!("Failed to request {url}: {err}")` whenever the - // CEF webview asks for an asset on `http://localhost:1420` (the - // Vite dev URL baked into `tauri.conf.json`). That `log::error!` - // is bridged into `tracing` and picked up by the sentry-tracing - // layer as an Event — see `crates/openhuman-core/src/core/logging.rs::sentry_tracing_layer`. - // In packaged staging/production builds Vite isn't running, so - // the request correctly fails — but the failure is noise we - // don't want in Sentry (issue OPENHUMAN-TAURI-V, 66+ events). - // See [sentry-localhost-filter] log line below for diagnostics. - if event_is_localhost_dev_fetch_noise(&event) { - log::debug!( - "[sentry-localhost-filter] dropping dev-server fetch noise event: {:?}", - event.message.as_deref().unwrap_or("") - ); - return None; - } - if openhuman_core::core::observability::is_budget_event(&event) { - // Log only structured tag metadata — `event.message` can carry - // upstream provider error text including tokens / pasted-through - // secrets, and per `CLAUDE.md` "never log secrets or full PII". - // The (domain, status) pair is sufficient diagnostic since - // those are the tags `is_budget_event` gates on. - log::debug!( - "[sentry-budget-filter] dropping budget-exhausted event (domain={:?}, status={:?})", - event.tags.get("domain"), - event.tags.get("status") - ); - return None; - } - // Defense-in-depth: drop max-tool-iterations cap events that - // slipped past the call-site filters in the core (see - // `openhuman_core::core::observability::is_max_iterations_event` - // for the rationale). The shell links the core in-process so - // any captured event for this deterministic agent-state - // outcome is filtered here too (OPENHUMAN-TAURI-99 / -98). - if openhuman_core::core::observability::is_max_iterations_event(&event) { - log::debug!( - "[sentry-max-iter-filter] dropping max-iteration cap noise event: {:?}", - event.message.as_deref().unwrap_or("") - ); - return None; - } - if openhuman_core::core::observability::is_transient_backend_api_failure(&event) - || openhuman_core::core::observability::is_transient_integrations_failure(&event) - || openhuman_core::core::observability::is_updater_transient_event(&event) - || openhuman_core::core::observability::is_skill_install_user_fetch_failure(&event) - { - return None; - } - // Defense-in-depth: drop managed-backend `errorCode` events (#870) - // the backend owns (F2/F4). The shell links the core in-process, - // so a managed inference error captured here must be filtered - // identically to the core binary's main.rs chain. The malformed - // `BAD_REQUEST` carve-out (F8) is excluded by the underlying - // decision, so a client-built bad payload still pages. - if openhuman_core::core::observability::is_backend_error_code_event(&event) { - log::debug!( - "[sentry-error-code-filter] dropping backend-owned errorCode event_id={:?}", - event.event_id - ); - return None; - } - // Defense-in-depth: drop transient streaming transport blips - // (domain=llm_provider, failure=transport) — flaky-network - // timeouts/resets recovered by retry/fallback (F7). Mirrors the - // core binary's main.rs filter. - if openhuman_core::core::observability::is_transient_provider_transport_failure(&event) - { - log::debug!( - "[sentry-transport-filter] dropping transient provider transport event_id={:?}", - event.event_id - ); - return None; - } - // Drop 401 "Session expired. Please log in again." bodies and - // pre-flight "no session token stored" guards — mirrors the - // core binary's before_send chain. Since #1061 the Tauri shell - // links the core in-process, so any session-expired event - // captured by either surface lands in the same Sentry client - // here and must be filtered identically. Keeps - // OPENHUMAN-TAURI-25 / -1Q / -27 / -1G off Sentry. - if openhuman_core::core::observability::is_session_expired_event(&event) { - // Metadata-only log shape — `event.message` carries the raw - // backend response body which CLAUDE.md forbids from local - // logs. Mirror the core binary's main.rs filter. - log::debug!( - "[sentry-session-expired-filter] dropping session-expired event_id={:?}", - event.event_id - ); - return None; - } - // Drop provider insufficient-credits 402s — the user's own BYO - // account (e.g. OpenRouter) is out of balance, a billing state - // OpenHuman has no lever over once the request already caps - // max_tokens. The core binary's main.rs before_send already - // filters these; since #1061 the core runs in-process inside this - // shell, so the cron `agent_job` retries-exhausted report (and any - // other compatible-provider path) lands in THIS Sentry client and - // must be filtered identically. Closes the #3617 drift that wired - // the filter only into the standalone-CLI chain (TAURI-RUST-514 / - // -C62). - if openhuman_core::core::observability::is_insufficient_credits_event(&event) { - // Metadata-only log shape — `event.message` carries the raw - // provider 402 body which CLAUDE.md forbids from local logs. - log::debug!( - "[sentry-insufficient-credits-filter] dropping insufficient-credits 402 event_id={:?}", - event.event_id - ); - return None; - } - // Drop provider monthly-quota exhausted events — the user's - // third-party plan has spent its allotment (e.g. Kiro - // `MONTHLY_REQUEST_COUNT`, sometimes wrapped in a 500 envelope so - // the 402-gated credits filter above misses it). No local lever; - // mirrors the core binary's main.rs before_send chain - // (TAURI-RUST-C9A: 9k events from a single quota-capped user). - if openhuman_core::core::observability::is_quota_exhausted_event(&event) { - // Metadata-only log shape — `event.message` carries the raw - // provider body which CLAUDE.md forbids from local logs. - log::debug!( - "[sentry-quota-exhausted-filter] dropping monthly-quota event_id={:?}", - event.event_id - ); - return None; - } - // Defense-in-depth: drop Windows `ERROR_FILE_SYSTEM_LIMITATION` - // (os error 665) — a persistent host-filesystem condition with - // zero local lever and no Sentry remediation path. The Tauri - // shell is a separate crate from the core, so the core's emit-site - // classifier (`expected_error_kind`) can only catch events that - // originate inside the core binary. Any filesystem-error event - // that starts in the shell (e.g. file_logging, window_state, - // CEF profile I/O) bypasses the core classifier and lands here; - // this filter is the only net for those events (TAURI-RUST-QT0: - // 6,050 events / 1 user). - if openhuman_core::core::observability::is_windows_file_system_limitation_event(&event) - { - log::debug!( - "[sentry-fs-limitation-filter] dropping Windows file-system-limitation event (os error 665) event_id={:?}", - event.event_id - ); - return None; - } - // Strip server_name (hostname) to avoid leaking machine identity. - event.server_name = None; - // Attach the cached account uid so Sentry can count unique users - // affected by an issue. We only carry `id` — never email, name, - // or IP — so this stays consistent with `send_default_pii: false`. - // Since #1061 the core runs in-process inside this shell, so this - // is the surface that tags ~all desktop events. - // - // Issue #3135: the primary source for `event.user` is now the - // Sentry scope, bound proactively at session boundaries - // (credentials::set_credential / clear_credential) and at server - // boot (run_server_inner). The shell's session owner mirrors the - // signed-in user id for this fallback, consulted only when the - // scope hasn't already bound a user — otherwise we'd silently - // clobber the scope binding when the slot is empty (the original - // userCount=0 root cause). - if event.user.is_none() { - event.user = session::peek_user_id().map(|id| sentry::User { - id: Some(id), - ..Default::default() - }); - } - Some(event) - })), - sample_rate: 1.0, - transport: Some(std::sync::Arc::new( - openhuman_core::core::sentry_transport::factory, - )), - ..sentry::ClientOptions::default() - }); + // + // The `before_send` chain is embed's shared one + // (`openhuman_rpc::embed::process::sentry`): the union of the shell's and + // the CLI's old filters (including this shell's dev-server + // "Failed to request http://localhost:…" noise, OPENHUMAN-TAURI-V), then + // hostname stripping, the user-id fallback and secret scrubbing. Since + // #1061 the core runs in-process here, so this is the surface that tags + // ~all desktop events. The fallback user id is the shell's session owner + // (`session::peek_user_id`), consulted only when the Sentry scope has not + // bound a user (#3135). + let _sentry_guard = { + use openhuman_rpc::embed::process::sentry as oh_sentry; + let mut config = oh_sentry::SentryConfig::new( + oh_sentry::first_non_blank([ + std::env::var("OPENHUMAN_TAURI_SENTRY_DSN").ok(), + option_env!("OPENHUMAN_TAURI_SENTRY_DSN").map(str::to_owned), + ]), + build_sentry_release_tag(), + resolve_sentry_environment(), + ); + config.user_id = session::peek_user_id; + sentry::init(oh_sentry::client_options(config)) + }; // Tag every Sentry event with CPU architecture and OS so Intel-specific // crashes (issue #1012 — SIGABRT in CrBrowserMain on x86_64 macOS) are // clearly identified without needing a separate build identifier. @@ -3065,7 +2898,7 @@ pub fn run() { if let Ok(resource_dir) = app.path().resource_dir() { let bundled = resource_dir.join("bundled-modules"); if bundled.is_dir() { - if openhuman_core::modules::ops::set_bundled_releases_dir(bundled) + if openhuman_rpc::embed::modules::set_bundled_releases_dir(bundled) .is_err() { log::warn!("[modules] bundled release directory was already set"); @@ -3634,85 +3467,31 @@ fn should_center_main_window(restored: bool, windows: bool, maximized: bool) -> } pub fn run_core_from_args(args: &[String]) -> Result<(), String> { - // Core lives in-process: dispatch directly through the linked `openhuman_core` - // library instead of shelling out to a separate binary. The Tauri main() - // routes `OpenHuman core ` here so users can still drive the core CLI - // from the bundled app. `run` / `serve` start the JSON-RPC server from - // `openhuman-rpc`, which the core cannot depend on, so install it first. - openhuman_rpc::server::install_cli_server(); - openhuman_core::run_core_from_args(args).map_err(|e| format!("{e:#}")) + // Core lives in-process: dispatch through the shared CLI host entry + // instead of shelling out to a separate binary. The Tauri main() routes + // `OpenHuman core ` (and `mcp`) here so users can still drive the + // core CLI from the bundled app. `host::cli` connects the TinyHumans + // backend and puts the JSON-RPC server behind `run` / `serve`, exactly as + // the standalone `openhuman-core` binary does. + log::debug!( + "[core-cli] dispatch command={}", + args.first().map(String::as_str).unwrap_or("") + ); + openhuman_rpc::host::cli(args).map_err(|e| format!("{e:#}")) } // --------------------------------------------------------------------------- // Sentry release / environment resolution (Tauri shell — desktop only) // --------------------------------------------------------------------------- -/// Canonical release tag: `openhuman@[+]`. -/// -/// Mirrors `build_release_tag` in `crates/openhuman-core/src/main.rs` and the -/// `SENTRY_RELEASE` value computed in `app/vite.config.ts` so events from -/// every surface (React frontend, standalone `openhuman-core` binary, Tauri -/// shell) group under the same release in Sentry and benefit from the same -/// source-map / debug-info upload. -/// Return `true` when the Sentry event is a "Failed to request -/// http://localhost:…" message originating from the vendored -/// `tauri-runtime-cef` dev-server proxy. -/// -/// The proxy logs this message via `log::error!` (see -/// `crates/openhuman-app/vendor/tauri-cef/crates/tauri/src/protocol/tauri.rs`) -/// every time the CEF webview asks for an asset on the Vite dev URL -/// (`http://localhost:1420` per `tauri.conf.json`). In packaged -/// staging/production builds Vite isn't running, so the request fails — -/// but the failure is benign and shouldn't be reported. -/// -/// The match is conservative: it checks the exact `Failed to request ` + -/// `http://localhost` / `http://127.0.0.1` prefix that only the dev-proxy -/// emits. Production HTTP errors from elsewhere in the shell or core use -/// different message shapes and won't be filtered. -fn event_is_localhost_dev_fetch_noise(event: &sentry::protocol::Event<'_>) -> bool { - // sentry-tracing 0.47 (with default `attach_stacktrace=false`) stores the - // log message in `event.message`. Check there first; fall back to the - // last exception's `value` for the (currently unused) stacktrace-enabled - // path so the filter stays correct if attach_stacktrace ever flips. - let direct = event.message.as_deref(); - let from_exception = event.exception.last().and_then(|e| e.value.as_deref()); - [direct, from_exception] - .into_iter() - .flatten() - .any(message_is_localhost_dev_fetch_noise) -} - -/// Pure prefix check, separated from `event_is_localhost_dev_fetch_noise` -/// so the matching rule can be unit-tested without constructing a full -/// Sentry `Event`. -fn message_is_localhost_dev_fetch_noise(message: &str) -> bool { - // The tauri-cef dev proxy formats the message as: - // `Failed to request {url}: {err}` - // so anchoring on `Failed to request http://localhost` / `127.0.0.1` is - // sufficient and avoids matching unrelated "Failed to request …" errors - // elsewhere in the codebase that target real hosts. - // - // Note: no `[::1]` (IPv6 loopback) entry — the vendored tauri-cef dev - // proxy resolves `localhost` to IPv4 via reqwest's default resolver, so - // dev-server fetches always surface as `http://localhost:` or - // `http://127.0.0.1:`. Add an `[::1]` prefix if that ever changes - // (per graycyrus note on PR #1545). - const PREFIXES: &[&str] = &[ - "Failed to request http://localhost:", - "Failed to request http://127.0.0.1:", - ]; - PREFIXES.iter().any(|p| message.starts_with(p)) -} - +/// Canonical release tag: `openhuman@[+]`, built by +/// embed's shared `release_tag` so the shell, the `openhuman-core` binary, +/// the TUI and the frontend's `SENTRY_RELEASE` all group under one release. fn build_sentry_release_tag() -> String { - let version = env!("CARGO_PKG_VERSION"); - let sha = option_env!("OPENHUMAN_BUILD_SHA").unwrap_or("").trim(); - let sha_short: String = sha.chars().take(12).collect(); - if sha_short.is_empty() { - format!("openhuman@{version}") - } else { - format!("openhuman@{version}+{sha_short}") - } + openhuman_rpc::embed::process::sentry::release_tag( + env!("CARGO_PKG_VERSION"), + option_env!("OPENHUMAN_BUILD_SHA"), + ) } /// Resolve the Sentry environment tag from `OPENHUMAN_APP_ENV` (runtime) or diff --git a/crates/openhuman-app/src/lib_tests.rs b/crates/openhuman-app/src/lib_tests.rs index 1522fd56a19..9bb5002a895 100644 --- a/crates/openhuman-app/src/lib_tests.rs +++ b/crates/openhuman-app/src/lib_tests.rs @@ -738,6 +738,16 @@ fn sentry_environment_defaults_to_production_when_unset() { // builds (issue OPENHUMAN-TAURI-V). Tests target the pure // `message_is_localhost_dev_fetch_noise` helper so the rule can be // asserted without standing up a Sentry client. +// +// The filter now lives in embed's shared `before_send` chain, which this +// shell installs; these tests pin that the chain still carries the +// shell's rule, under its `localhost-dev-fetch` name. + +use openhuman_rpc::embed::process::sentry::message_is_localhost_dev_fetch_noise; + +fn event_is_localhost_dev_fetch_noise(event: &sentry::protocol::Event<'static>) -> bool { + openhuman_rpc::embed::process::sentry::known_noise(event) == Some("localhost-dev-fetch") +} #[test] fn localhost_dev_fetch_noise_drops_vite_dev_url_1420() { diff --git a/crates/openhuman-app/src/local_data_reset.rs b/crates/openhuman-app/src/local_data_reset.rs index 7abbbaab8e4..b675573689f 100644 --- a/crates/openhuman-app/src/local_data_reset.rs +++ b/crates/openhuman-app/src/local_data_reset.rs @@ -89,7 +89,7 @@ pub async fn reset_local_data( // below to fail with `ERROR_SHARING_VIOLATION` (os error 32). Drop // the writer guard now so the background flushing thread exits and // the file handle is closed before the removal walks the tree. - let log_guard_dropped = openhuman_core::core::logging::shutdown_file_guard(); + let log_guard_dropped = openhuman_rpc::embed::process::shutdown_file_guard(); log::info!("[core] reset_local_data: shutdown_file_guard dropped guard = {log_guard_dropped}"); // ── 4. Remove the paths ───────────────────────────────────────────── diff --git a/crates/openhuman-app/src/main.rs b/crates/openhuman-app/src/main.rs index 0a72934fe96..a04931d3827 100644 --- a/crates/openhuman-app/src/main.rs +++ b/crates/openhuman-app/src/main.rs @@ -7,14 +7,10 @@ fn main() { // Every path below boots a core that must reach the hosted backend - // (billing, integrations, channel relay, login): give it the SDK-backed - // transport before the first dispatch. The core itself carries none. - if let Err(err) = openhuman_tinyhumans::install(openhuman_tinyhumans::InstallOptions::default()) - { - eprintln!("failed to install the TinyHumans backend transport: {err}"); - std::process::exit(1); - } - + // (billing, integrations, channel relay, login). Each boots through an + // `openhuman_rpc::host` entry (`cli` for `core`/`mcp`, `desktop` for the + // GUI's embedded server), which connects the TinyHumans backend transport + // itself, so there is no separate install step here. let args: Vec = std::env::args().collect(); let sub = args.get(1).map(String::as_str); if sub == Some("core") { diff --git a/crates/openhuman-app/src/session/commands.rs b/crates/openhuman-app/src/session/commands.rs index 71226a0e87e..da91b81653d 100644 --- a/crates/openhuman-app/src/session/commands.rs +++ b/crates/openhuman-app/src/session/commands.rs @@ -1,13 +1,13 @@ //! Tauri commands the renderer uses to drive login, logout and the current //! user. Each is a one-line delegate to the managed `SessionHost`; the -//! `Err(String)` carries `openhuman_tinyhumans::SessionError`'s stable +//! `Err(String)` carries `openhuman_rpc::tinyhumans::SessionError`'s stable //! `PREFIX:` so the frontend can classify without parsing prose. -use openhuman_tinyhumans::{CachedUser, SessionState}; +use openhuman_rpc::tinyhumans::{CachedUser, SessionState}; use super::SessionHost; -fn err(error: openhuman_tinyhumans::SessionError) -> String { +fn err(error: openhuman_rpc::tinyhumans::SessionError) -> String { error.to_string() } diff --git a/crates/openhuman-app/src/session/link.rs b/crates/openhuman-app/src/session/link.rs index af98df65d82..0cc6c718dd7 100644 --- a/crates/openhuman-app/src/session/link.rs +++ b/crates/openhuman-app/src/session/link.rs @@ -5,7 +5,7 @@ //! loopback. use async_trait::async_trait; -use openhuman_tinyhumans::CoreLink; +use openhuman_rpc::tinyhumans::CoreLink; use serde_json::Value; use crate::core_process::CoreProcessHandle; diff --git a/crates/openhuman-app/src/session/mod.rs b/crates/openhuman-app/src/session/mod.rs index 0934d1cc5c9..0e6e404e6d6 100644 --- a/crates/openhuman-app/src/session/mod.rs +++ b/crates/openhuman-app/src/session/mod.rs @@ -13,7 +13,7 @@ mod link; use std::sync::Arc; -use openhuman_tinyhumans::{ClientHeaders, SessionEvent, SessionManager}; +use openhuman_rpc::tinyhumans::{ClientHeaders, SessionEvent, SessionManager}; use tauri::{AppHandle, Emitter, Manager}; use crate::core_process::CoreProcessHandle; @@ -22,7 +22,7 @@ use crate::AppRuntime; pub(crate) use link::HttpCoreLink; /// Tauri event emitted whenever the credential or the current user changes. -/// Payload: `openhuman_tinyhumans::SessionState`. +/// Payload: `openhuman_rpc::tinyhumans::SessionState`. pub const AUTH_CHANGED_EVENT: &str = "auth://changed"; /// Tauri event emitted when the backend rejected the stored credential and it /// has been cleared. Payload: `{ source }`. @@ -37,7 +37,7 @@ impl SessionHost { pub fn new(desktop: CoreProcessHandle) -> Self { // The shell and the core ship as one release, so one version answers // for both `x-core-version` and `x-tauri-version`. - let headers = ClientHeaders::new(openhuman_tinyhumans::product_identity().as_str()) + let headers = ClientHeaders::new(openhuman_rpc::tinyhumans::product_identity().as_str()) .with_core_version(env!("CARGO_PKG_VERSION")) .with_tauri_version(env!("CARGO_PKG_VERSION")); let link = Arc::new(HttpCoreLink::new(desktop)); @@ -89,5 +89,5 @@ pub fn install(app: &AppHandle, desktop: CoreProcessHandle) { /// The signed-in user id, for synchronous callers (Sentry `before_send`). pub fn peek_user_id() -> Option { - openhuman_tinyhumans::identity::peek_user_id() + openhuman_rpc::tinyhumans::identity::peek_user_id() } diff --git a/crates/openhuman-app/src/workspace_paths.rs b/crates/openhuman-app/src/workspace_paths.rs index 0060752aebb..0d8b17df7a7 100644 --- a/crates/openhuman-app/src/workspace_paths.rs +++ b/crates/openhuman-app/src/workspace_paths.rs @@ -52,7 +52,7 @@ pub async fn preview_workspace_text(path: String) -> Result Result { - let config = openhuman_core::config::Config::load_or_init() + let config = openhuman_rpc::embed::config::load_or_init() .await .map_err(|err| workspace_path_error(format!("failed to load OpenHuman config: {err}")))?; fs::create_dir_all(&config.workspace_dir).map_err(|err| { diff --git a/crates/openhuman-cli/Cargo.toml b/crates/openhuman-cli/Cargo.toml index 37ffda3cf9c..4ea65a26f73 100644 --- a/crates/openhuman-cli/Cargo.toml +++ b/crates/openhuman-cli/Cargo.toml @@ -1,9 +1,13 @@ -# The `openhuman-core` binary, the developer/benchmark bins, and every root -# `tests/*.rs` / `examples/*.rs` target. They used to live in the core package; -# they moved here because a core that carries no backend client cannot host the -# binary that must talk to the hosted backend — `main.rs` installs the -# `openhuman-tinyhumans` transport before dispatching, and the mock-backend -# integration suites boot the same way through `tests/support/tinyhumans_boot.rs`. +# The `openhuman-core` binary, the ops bins (`openhuman-fleet`, the MCP test +# stub), and every root `tests/*.rs` / `examples/*.rs` target. +# +# The binary is a host like the desktop app and the TUI: its normal +# dependencies name `openhuman-rpc` and no other openhuman crate (chain: core +# -> embed -> tinyhumans -> rpc -> app/cli/tui; enforced by +# `scripts/ci/check-crate-chain.mjs`). `main.rs` is `openhuman_rpc::host::cli`. +# The integration tests still reach into the core directly, so the core, embed +# and tinyhumans are DEV-dependencies here: they build into the test and +# example targets and never into the shipped binary. # # Auto-discovery is off because Cargo only scans beside this manifest; the # repository layout gate (`scripts/ci/check-openhuman-rust-layout.mjs`) @@ -230,36 +234,39 @@ name = "embed_kernel" path = "../../examples/embed_kernel.rs" [dependencies] -# Direct core access: every test and bin names `openhuman_core::` paths. -# `default-features = false` so the gates below are the only ones on. -openhuman-core = { path = "../openhuman-core", package = "openhuman", default-features = false } -openhuman-tinyhumans = { path = "../openhuman-tinyhumans", default-features = false } +# The only openhuman crate the binary depends on. Every gate below forwards to +# it, and it forwards down the chain to the core. openhuman-rpc = { workspace = true, features = ["server"] } anyhow = "1.0" -async-trait = "0.1" axum = { version = "0.8", default-features = false, features = ["http1", "json", "tokio", "query", "ws", "macros"] } -chrono = { version = "0.4", features = ["serde"] } clap = { version = "4.5", features = ["derive"], optional = true } -dotenvy = "0.15" env_logger = "0.11" -futures = "0.3" libc = "0.2" log = "0.4" reqwest = { version = "0.12", default-features = false, features = ["json", "blocking", "rustls-tls", "stream", "http2", "multipart", "socks"] } -sentry = { version = "0.47.0", default-features = false, optional = true, features = ["backtrace", "contexts", "panic", "tracing", "debug-images", "httpdate"] } -serde = { version = "1", features = ["derive"] } serde_json = "1" -tempfile = "3" -tinyinference-llm = { path = "../../vendor/tinyagents/vendor/tinyinference/crates/tinyinference-llm" } -tinytools = { path = "../../vendor/tinyagents/vendor/tinytools/crates/tinytools" } -tinytools-agent = { path = "../../vendor/tinyagents/vendor/tinytools/crates/tinytools-agent" } tokio = { version = "1", features = ["full"] } -toml = "1.0" -tracing = "0.1" uuid = { version = "1", features = ["v4"] } [dev-dependencies] +# Root tests and examples reach into the core, embed and tinyhumans directly. +# Dev-only, so none of them is an edge of the shipped binary. Same spelling as +# the library crates' own manifests (`default-features = false`; the gates come +# from this crate's features through `openhuman-rpc`), so the dev graphs unify +# and `cargo test -p` reuses one core build. +openhuman-core = { path = "../openhuman-core", package = "openhuman", default-features = false } +openhuman-embed = { workspace = true } +openhuman-tinyhumans = { workspace = true } aes-gcm = "0.10" +async-trait = "0.1" +chrono = { version = "0.4", features = ["serde"] } +futures = "0.3" +serde = { version = "1", features = ["derive"] } +tempfile = "3" +tinyinference-llm = { path = "../../vendor/tinyagents/vendor/tinyinference/crates/tinyinference-llm" } +tinytools = { path = "../../vendor/tinyagents/vendor/tinytools/crates/tinytools" } +toml = "1.0" +tracing = "0.1" # The TinyComputer contract, for the live computer-use e2e (`computer_bali_live_e2e`). tinycomputer-bus = { path = "../../vendor/tinycomputer/crates/tinycomputer-bus" } # sentry's TestTransport for the observability before_send smoke tests; @@ -295,35 +302,50 @@ urlencoding = "2.1" wiremock = "0.6" [features] -# Same contributor set as the core's `default`, forwarded. The product lanes -# pass `--features "$(scripts/ci/product-features.sh)"`; every product gate -# name resolves here so `cargo test --workspace --features ...` keeps working. -default = ["openhuman-core/default", "openhuman-tinyhumans/default", "jev"] -# The Jev-backed `tool_search` ranker; `openhuman-tinyhumans`'s own gate. -jev = ["openhuman-tinyhumans/jev"] -http-server = ["openhuman-core/http-server", "openhuman-tinyhumans/http-server"] -inference = ["openhuman-core/inference", "openhuman-tinyhumans/inference"] -documents = ["openhuman-core/documents", "openhuman-tinyhumans/documents"] -hosting = ["openhuman-core/hosting", "openhuman-tinyhumans/hosting"] -modules = ["openhuman-core/modules", "openhuman-tinyhumans/modules"] -voice = ["openhuman-core/voice", "openhuman-tinyhumans/voice"] -web3 = ["openhuman-core/web3", "openhuman-tinyhumans/web3"] +# The contributor set: the core's `[features] default` gates (listed, because +# `openhuman-rpc`'s own default carries only its server and client) plus the +# Jev ranker. The product lanes pass `--features +# "$(scripts/ci/product-features.sh)"`; every product gate name resolves here. +default = [ + "media", + "skills", + "flows", + "mcp", + "channels", + "http-server", + "scheduler-gate", + "file-logging", + "modules", + "jev", +] +# Every gate forwards to `openhuman-rpc` alone (chain: rpc -> tinyhumans -> +# embed -> core); `scripts/ci/check-feature-forwarding.mjs` checks the link. +# The Jev-backed `tool_search` ranker. +jev = ["openhuman-rpc/jev"] +http-server = ["openhuman-rpc/http-server"] +inference = ["openhuman-rpc/inference"] +documents = ["openhuman-rpc/documents"] +hosting = ["openhuman-rpc/hosting"] +modules = ["openhuman-rpc/modules"] +voice = ["openhuman-rpc/voice"] +web3 = ["openhuman-rpc/web3"] # Storage drivers for `[storage] url` (see the core `storage` domain). -storage-sqlite = ["openhuman-core/storage-sqlite", "openhuman-tinyhumans/storage-sqlite"] -storage-mongodb = ["openhuman-core/storage-mongodb", "openhuman-tinyhumans/storage-mongodb"] -storage-file = ["openhuman-core/storage-file", "openhuman-tinyhumans/storage-file"] -runtime-node = ["openhuman-core/runtime-node", "openhuman-tinyhumans/runtime-node"] -media = ["openhuman-core/media", "openhuman-tinyhumans/media"] -flows = ["openhuman-core/flows", "openhuman-tinyhumans/flows"] -skills = ["openhuman-core/skills", "openhuman-tinyhumans/skills"] -mcp = ["openhuman-core/mcp", "openhuman-tinyhumans/mcp"] -channels = ["openhuman-core/channels", "openhuman-tinyhumans/channels"] -whatsapp-web = ["openhuman-core/whatsapp-web", "openhuman-tinyhumans/whatsapp-web"] -e2e-test-support = ["openhuman-core/e2e-test-support"] -file-logging = ["openhuman-core/file-logging", "openhuman-tinyhumans/file-logging"] -scheduler-gate = ["openhuman-core/scheduler-gate", "openhuman-tinyhumans/scheduler-gate"] -# Sentry init in `main.rs` plus the `observability_smoke` test target. -crash-reporting = ["dep:sentry", "openhuman-core/crash-reporting", "openhuman-tinyhumans/crash-reporting"] +storage-sqlite = ["openhuman-rpc/storage-sqlite"] +storage-mongodb = ["openhuman-rpc/storage-mongodb"] +storage-file = ["openhuman-rpc/storage-file"] +runtime-node = ["openhuman-rpc/runtime-node"] +media = ["openhuman-rpc/media"] +flows = ["openhuman-rpc/flows"] +skills = ["openhuman-rpc/skills"] +mcp = ["openhuman-rpc/mcp"] +channels = ["openhuman-rpc/channels"] +whatsapp-web = ["openhuman-rpc/whatsapp-web"] +e2e-test-support = ["openhuman-rpc/e2e-test-support"] +file-logging = ["openhuman-rpc/file-logging"] +scheduler-gate = ["openhuman-rpc/scheduler-gate"] +# Sentry init in `main.rs` (through `embed::process::sentry`) plus the +# `observability_smoke` test target. +crash-reporting = ["openhuman-rpc/crash-reporting"] # CLI argument parsing + logger init for `openhuman-fleet`. bin-tools = ["dep:clap"] diff --git a/crates/openhuman-cli/README.md b/crates/openhuman-cli/README.md index fdea27dece8..84be2686bbc 100644 --- a/crates/openhuman-cli/README.md +++ b/crates/openhuman-cli/README.md @@ -1,8 +1,8 @@ # openhuman-cli -This crate builds the `openhuman-core` binary, a handful of developer and -benchmark binaries, and every root `tests/*.rs` and `examples/*.rs` target in -the repository. The core ([`crates/openhuman-core`](../openhuman-core/), package `openhuman`) is a +This crate builds the `openhuman-core` binary, two ops binaries +(`openhuman-fleet`, `test-mcp-stub`), and every root `tests/*.rs` and +`examples/*.rs` target in the repository. The core ([`crates/openhuman-core`](../openhuman-core/), package `openhuman`) is a library with no backend client and no JSON-RPC server, and it declares no bin, test or example targets of its own. Anything that must run as a process against the hosted backend, or test the core the way a host boots it, lives @@ -13,27 +13,27 @@ here. ### Where it sits ```text - openhuman-cli - (bins, tests, examples) - | | | - v v v - openhuman-tinyhumans openhuman-rpc openhuman-core - (SDK transport, (JSON-RPC (domains, CLI - hosted proxies, server, dispatcher, - session owner) run_server*) controller registry) - | | - v | - openhuman-embed | - | | - +--------+---------+ - v - openhuman-core + openhuman-cli (openhuman-core binary, ops bins) + | + v the only normal OpenHuman dependency + openhuman-rpc (host::cli, JSON-RPC server, session store) + | + v + openhuman-tinyhumans (SDK transport, hosted proxies, session owner) + | + v + openhuman-embed (Runtime/RuntimeBuilder, process helpers, facades) + | + v + openhuman-core (domains, CLI dispatcher, controller registry) ``` -The crate depends on the core directly (every bin and test names -`openhuman_core::` paths), on `openhuman-tinyhumans` for the backend -transport, and on `openhuman-rpc` (with its `server` feature) for the -JSON-RPC server that `run` and `serve` start. +The binary depends on `openhuman-rpc` alone +(`scripts/ci/check-crate-chain.mjs` enforces it), and every feature gate +forwards to `openhuman-rpc/`. The root tests and examples still reach +into the core: `openhuman-core`, `openhuman-embed` and `openhuman-tinyhumans` +are **dev-dependencies**, so they are compiled into test and example targets +and never into the shipped binary. ### Startup of `openhuman-core` @@ -42,27 +42,21 @@ JSON-RPC server that `run` and `serve` start. 1. `restore_default_sigpipe()` resets `SIGPIPE` to the default on unix, so piping output into `head` ends the process quietly instead of panicking on `EPIPE`. -2. `dotenvy::dotenv()` loads a repo-local `.env` before Sentry starts, so a - DSN defined only there is visible. The CLI dispatcher later runs - `load_dotenv_for_cli`, which honors `OPENHUMAN_DOTENV_PATH`. -3. With the `crash-reporting` feature, `sentry::init` resolves the DSN from +2. `embed::process::load_dotenv_for_cli()` loads `.env` (or + `OPENHUMAN_DOTENV_PATH`) before Sentry starts, so a DSN defined only there + is visible. Variables already in the environment win. +3. With the `crash-reporting` feature, Sentry starts with + `embed::process::sentry::client_options`: the DSN from `OPENHUMAN_CORE_SENTRY_DSN`, then `OPENHUMAN_SENTRY_DSN`, at runtime and - then at compile time. Its `before_send` drops known-noise event classes - through the `openhuman_core::core::observability::is_*_event` predicates - (transient provider failures, budget and credit exhaustion, session - expiry, connectivity blips, stale releases, and others), strips - `server_name`, attaches only the account id as the Sentry user, and - scrubs secrets from messages and exception values with - `openhuman_core::core::log_redaction::scrub_secrets`. The release tag is - `openhuman@[+]`, matching the frontend. -4. `openhuman_tinyhumans::install(InstallOptions::default())` installs the - SDK-backed backend transport, registers the hosted RPC proxies and the - Jev ranker. A failure here exits with status 1. -5. `openhuman_rpc::server::install_cli_server()` hands the core the JSON-RPC - server launcher, which the core cannot depend on. -6. `openhuman_core::run_core_from_args(&args)` loads `.env` again with the - CLI rules, applies any startup restart delay, initializes the keyring - master key and dispatches the subcommand. + then at compile time; the release tag `openhuman@[+]`; and + embed's shared `before_send` chain (the same one the desktop shell and the + TUI install), which drops known-noise classes, strips `server_name`, falls + back to the credential identity for the user id, and scrubs secrets. +4. `openhuman_rpc::host::cli(&args)` does the rest: it connects the + TinyHumans backend (SDK transport, hosted RPC proxies, Jev ranker), puts + the JSON-RPC server behind `run` / `serve`, registers the `http_host` + controllers, and runs the core's dispatcher. A failure exits with + status 1. ### Subcommands @@ -129,7 +123,7 @@ targets: `observability_smoke` (`crash-reporting`), `computer_bali_live_e2e` In-process suites that reach the (mock) backend call `tinyhumans_boot::boot()` from [`tests/support/tinyhumans_boot.rs`](../../tests/support/tinyhumans_boot.rs) first, which -runs `openhuman_tinyhumans::install`. Without it every backend call answers +runs `openhuman_tinyhumans::install` (a dev-dependency here). Without it every backend call answers `BACKEND_UNAVAILABLE:`. Suites that spawn the `openhuman-core` binary get the transport from `main.rs`. @@ -139,9 +133,9 @@ transport from `main.rs`. | --- | --- | | [`Cargo.toml`](Cargo.toml) | All `[[bin]]`, `[[test]]` and `[[example]]` targets, feature forwarding. | | [`src/main.rs`](src/main.rs) | The `openhuman-core` binary entry point described above. | -| [`src/bin/`](src/bin/README.md) | Developer binaries: `test-mcp-stub`, `openhuman-fleet`. The benchmark binaries live in [openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) (`profile/`). | +| [`src/bin/`](src/bin/README.md) | Ops binaries: `test-mcp-stub`, `openhuman-fleet`. The benchmark binaries live in [openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) (`profile/`, #6944). | | `../../tests/*.rs` | 27 `[[test]]` targets, including the two aggregators. See [`tests/README.md`](../../tests/README.md). | -| `../../examples/*.rs` | 2 `[[example]]` targets: `embed_headless` and `embed_kernel`. | +| `../../examples/*.rs` | 2 `[[example]]` targets on the `openhuman_embed::Runtime` API: `embed_headless` and `embed_kernel`. | | `../../build.rs` | Shared build script: generates the `raw_coverage_all` and `in_process_all` module lists and exports `OPENHUMAN_REPOSITORY_ROOT`. | ## Targets @@ -154,19 +148,18 @@ transport from `main.rs`. ## Features -The default set mirrors the core's contributor default plus -`openhuman-tinyhumans/default` and `jev`. Every core gate (`http-server`, -`inference`, `voice`, `web3`, `channels`, `media`, `modules`, and the rest) is -forwarded to both `openhuman-core` and `openhuman-tinyhumans`, so the product -lanes' feature list resolves here unchanged; [`scripts/ci/check-feature-forwarding.mjs`](../../scripts/ci/check-feature-forwarding.mjs) -checks the chain. `e2e-test-support` forwards to the core only; `rss-bench` is -not forwarded, since the benchmark crate that uses it enables it on the core directly. -Gates local to this crate: +The default set is the core's contributor default (listed by name, because +`openhuman-rpc`'s own default carries only its server and client) plus +`jev`. Every gate (`http-server`, `inference`, `voice`, `web3`, `storage-*`, +`channels`, `media`, `modules`, `e2e-test-support`, and the rest) forwards to +`openhuman-rpc/`, which forwards it down the chain, so the product +lanes' feature list resolves here unchanged; +[`scripts/ci/check-feature-forwarding.mjs`](../../scripts/ci/check-feature-forwarding.mjs) checks the link. `rss-bench` is +not forwarded, since the benchmark crate that uses it enables it on the core +directly. Gates local to this crate: | Feature | Purpose | | --- | --- | -| `jev` | The Jev `tool_search` ranker (`openhuman-tinyhumans/jev`). | -| `crash-reporting` | Sentry init in `main.rs` and the `observability_smoke` target. | | `bin-tools` | `clap` for `openhuman-fleet`. | ## Boundaries @@ -179,8 +172,12 @@ Gates local to this crate: [`crates/openhuman-rpc`](../openhuman-rpc/). The backend transport and login belong to [`crates/openhuman-tinyhumans`](../openhuman-tinyhumans/). - The terminal UI is [`crates/openhuman-tui`](../openhuman-tui/); the desktop host is - [`crates/openhuman-app`](../openhuman-app/), which embeds the core directly and does not use - this binary. + [`crates/openhuman-app`](../openhuman-app/). Both are hosts on `openhuman-rpc` like this one and + do not use this binary (the app's `core` subcommand runs the same + `host::cli`). +- The binary never names the core, embed or tinyhumans directly; reach new + core behavior through embed's facade (re-exported as + `openhuman_rpc::embed`). ## Gotchas @@ -203,8 +200,8 @@ pnpm test:rust # scripts/test-rust-with-mock.sh, the canonical runner pnpm debug rust ``` -`main_tests.rs` (built with `crash-reporting`) covers the secret scrubbing -that `before_send` applies (bearer tokens and provider API keys). +`main_tests.rs` (built with `crash-reporting`) covers the environment and +release resolution, and pins that the shared chain still scrubs secrets. ## Further reading diff --git a/crates/openhuman-cli/src/bin/README.md b/crates/openhuman-cli/src/bin/README.md index 6e03ae11cfd..0402d721ad1 100644 --- a/crates/openhuman-cli/src/bin/README.md +++ b/crates/openhuman-cli/src/bin/README.md @@ -4,11 +4,17 @@ Auxiliary binaries declared as `[[bin]]` targets in [`crates/openhuman-cli/Cargo.toml`](../../Cargo.toml), next to the primary `openhuman-core` binary ([`src/main.rs`](../main.rs), described in the [crate README](../../README.md)). One is a test fixture and the other an experimental multi-tenant supervisor. -Neither ships in the desktop product. The benchmark and profiling binaries -that used to live here (`tool-search-bench`, `tool-dialect-bench`, -`rss-bench`, `library-profile`) moved to the `profile/` crate of -[openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks), which builds them against a vendored checkout -of this workspace. +Neither ships in the desktop product, and neither names an OpenHuman crate: +the CLI's normal dependencies stop at `openhuman-rpc`. + +The benchmark and profiling binaries that used to live here +(`tool-search-bench`, `tool-dialect-bench`, `rss-bench`, `library-profile`) +reached deep into core internals (`agent::harness`, `platform::proc_metrics`, +`flows`, provider factories) that the curated facade does not expose. They +moved, with their `scripts/profile/` drivers, to the `profile/` crate of +[openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) (#6944), which builds them against a vendored +checkout of this workspace; the `rss-bench` / `rss-bench-dhat` gates went +with them. ## How it works @@ -93,7 +99,7 @@ cargo build -p openhuman-cli --features bin-tools --bin openhuman-fleet - The manifest sets `autobins = false`, so a `.rs` file in this directory is a binary only when it has a `[[bin]]` entry. That is what lets [`fleet_tests.rs`](fleet_tests.rs) sit here beside its binary - without Cargo trying to build them as executables. A new binary needs both + without Cargo trying to build it as an executable. A new binary needs both the file and the manifest entry. ## Tests diff --git a/crates/openhuman-cli/src/main.rs b/crates/openhuman-cli/src/main.rs index 4d62d6776f0..ff7f1c154ee 100644 --- a/crates/openhuman-cli/src/main.rs +++ b/crates/openhuman-cli/src/main.rs @@ -1,292 +1,33 @@ -//! The entry point for the OpenHuman core application. +//! The `openhuman-core` binary. //! -//! This file is responsible for: -//! - Initializing error tracking with Sentry. -//! - Setting up secret scrubbing for outgoing error reports. -//! - Dispatching command-line arguments to the core logic in `openhuman_core`. +//! A host like the desktop app and the TUI: it depends on `openhuman-rpc` +//! alone and boots through the shared host entry +//! [`openhuman_rpc::host::cli`], which connects the TinyHumans backend, +//! registers the server launcher and the `http_host` controllers, and runs +//! the core's command-line dispatcher. This file only owns what is specific +//! to the process: SIGPIPE, the early `.env`, and (with `crash-reporting`) +//! the Sentry client, built from embed's shared `before_send` chain. + +use openhuman_rpc::embed::process; -/// Main application entry point. -/// -/// It initializes the Sentry SDK for error monitoring, ensuring that sensitive -/// information is redacted before being sent to the server. After setup, it -/// delegates execution to the core library based on CLI arguments. fn main() { restore_default_sigpipe(); - // Load `.env` before `sentry::init` so a DSN defined only in the dotenv - // file is visible to the Sentry client at startup. `dotenvy::dotenv()` is - // a no-op for variables already present in the process environment, and - // the CLI dispatcher later calls `load_dotenv_for_cli` which honors - // `OPENHUMAN_DOTENV_PATH`; this early call handles the common default - // case (repo-local `.env`) so startup-time consumers (Sentry, config - // overrides) see the same values as runtime RPC handlers. - let _ = dotenvy::dotenv(); + // Load `.env` before Sentry so a DSN defined only in the dotenv file is + // visible at startup. Honors `OPENHUMAN_DOTENV_PATH` and never overwrites + // variables already in the environment; the dispatcher loads it again + // with the same rules, which is a no-op by then. + if let Err(err) = process::load_dotenv_for_cli() { + // Not fatal here: the dispatcher re-runs the load and reports it. + log::debug!("[cli] early dotenv load failed: {err}"); + } - // Initialize Sentry as the very first operation so the guard outlives everything. - // Resolves the core Sentry DSN by checking, in order: - // 1. `OPENHUMAN_CORE_SENTRY_DSN` at runtime (preferred, namespaced name) - // 2. `OPENHUMAN_SENTRY_DSN` at runtime (legacy unprefixed name — kept - // so existing CI vars and contributor `.env` files keep working until - // the GH org-level variable can be renamed) - // 3. Each of the same names baked at compile time via `option_env!` - // If none resolve to a non-empty value, `sentry::init` returns a no-op guard. - // - // The whole init (guard + secret-scrubbing `before_send`) is gated on the - // `crash-reporting` feature; a slim build compiles it out entirely. + // The guard must outlive everything, so it is bound in `main`. #[cfg(feature = "crash-reporting")] - let _sentry_guard = sentry::init(sentry::ClientOptions { - dsn: std::env::var("OPENHUMAN_CORE_SENTRY_DSN") - .ok() - .filter(|s| !s.is_empty()) - .or_else(|| std::env::var("OPENHUMAN_SENTRY_DSN").ok()) - .filter(|s| !s.is_empty()) - .or_else(|| option_env!("OPENHUMAN_CORE_SENTRY_DSN").map(|s| s.to_string())) - .filter(|s| !s.is_empty()) - .or_else(|| option_env!("OPENHUMAN_SENTRY_DSN").map(|s| s.to_string())) - .filter(|s| !s.is_empty()) - .and_then(|s| s.parse().ok()), - release: Some(std::borrow::Cow::Owned(build_release_tag())), - environment: Some(std::borrow::Cow::Owned(resolve_environment())), - send_default_pii: false, - before_send: Some(std::sync::Arc::new(|mut event| { - // Defense-in-depth: drop transient-upstream provider failures that - // slipped past the call-site classifier. The reliable-provider - // layer already retries 429/408/502/503/504 with backoff + - // fallback, and the aggregate "all providers exhausted" event - // still fires for genuine outages. Per-attempt reports flood - // Sentry — see OPENHUMAN-TAURI-2E (~1393 events), -84 (~1050), - // -T (~871). The primary fix lives in - // `openhuman::inference::provider::ops::should_report_provider_http_failure` - // (transient codes excluded). This filter catches any future call - // site that bypasses it. - if openhuman_core::core::observability::is_transient_provider_http_failure(&event) { - return None; - } - if openhuman_core::core::observability::is_all_transient_provider_exhaustion_event( - &event, - ) { - return None; - } - // Defense-in-depth: drop managed-backend `errorCode` events (#870) - // the backend owns (F2/F4) — primary suppression lives in - // `api_error` / the streaming gates and the `web_channel` - // re-report classifier. The malformed `BAD_REQUEST` carve-out - // (F8) is excluded by the underlying decision, so a client-built - // bad payload still pages. - if openhuman_core::core::observability::is_backend_error_code_event(&event) { - return None; - } - // Defense-in-depth: drop transient streaming transport blips - // (domain=llm_provider, failure=transport) — flaky-network - // timeouts/resets recovered by retry/fallback (F7). The primary - // gate lives at the `stream_chat` / `stream_chat_history` emit - // sites. - if openhuman_core::core::observability::is_transient_provider_transport_failure(&event) - { - return None; - } - // Defense-in-depth for budget-exhausted 400s. Emit sites demote the - // known backend responses before they hit Sentry; this catches any - // future non_2xx/status=400 event that carries the same tight body - // phrases. - if openhuman_core::core::observability::is_budget_event(&event) { - return None; - } - // Defense-in-depth for insufficient-credits 402s. The native_chat - // emit site demotes them, but the compatible provider reports the - // same out-of-balance 402 from chat_with_system / chat_with_history - // / the streaming gates / api_error too; this is the single net - // that catches every path (TAURI-RUST-C62). - if openhuman_core::core::observability::is_insufficient_credits_event(&event) { - return None; - } - // Drop provider monthly-quota exhausted events — third-party plan - // allotment spent (e.g. Kiro `MONTHLY_REQUEST_COUNT`, sometimes - // wrapped in a 500 envelope so the 402-gated credits filter above - // misses it). No local lever (TAURI-RUST-C9A). - if openhuman_core::core::observability::is_quota_exhausted_event(&event) { - return None; - } - // Defense-in-depth for Ollama Cloud hosted-inference 500s. The - // native_chat / streaming_chat / api_error emit sites demote them and - // the agent re-report routes through `TransientUpstreamHttp`, but the - // compatible provider can report the same `Internal Server Error - // (ref: …)` body from other paths; this is the single net that - // catches every path (TAURI-RUST-5MV). - if openhuman_core::core::observability::is_ollama_cloud_internal_500_event(&event) { - return None; - } - // Defense-in-depth: drop Windows `ERROR_FILE_SYSTEM_LIMITATION` - // (os error 665) — a persistent host-filesystem condition with - // zero local lever and no Sentry remediation path. The primary - // suppression lives at the emit site via `expected_error_kind` → - // `ExpectedErrorKind::WindowsFileSystemLimitation`; this catches - // any call site that uses `report_error` directly instead of - // `report_error_or_expected` (TAURI-RUST-QT0: 6,050 events / 1 user). - if openhuman_core::core::observability::is_windows_file_system_limitation_event(&event) - { - log::debug!( - "[sentry-fs-limitation-filter] dropping Windows file-system-limitation event (os error 665) event_id={:?}", - event.event_id - ); - return None; - } - // Defense-in-depth: drop max-tool-iterations cap events that - // slipped past the call-site filters in - // `agent::session_host::runtime::run_single`, - // `channels::runtime::dispatch`, and - // `web_chat::run_chat_task`. The cap is a - // deterministic agent-state outcome surfaced to the user via - // the chat-rendered "Error: …" message — Sentry is the wrong - // surface for it (OPENHUMAN-TAURI-99 / -98). - if openhuman_core::core::observability::is_max_iterations_event(&event) { - return None; - } - if openhuman_core::core::observability::is_transient_backend_api_failure(&event) - || openhuman_core::core::observability::is_transient_integrations_failure(&event) - || openhuman_core::core::observability::is_updater_transient_event(&event) - || openhuman_core::core::observability::is_skill_install_user_fetch_failure(&event) - { - return None; - } - // Defense-in-depth: drop skill-install fetch 4xx (esp. 404/410) — - // a missing/renamed catalog `SKILL.md` is expected user-input state - // surfaced to the UI, not a Sentry-actionable defect. Primary - // suppression lives at the `install_workflow_from_url_with_home` - // emit site; this catches any future skills call site that reports - // a 4xx. 5xx (genuine remote failure) still reports. TAURI-RUST-CGE. - if openhuman_core::core::observability::is_skills_install_client_error_event(&event) { - return None; - } - // Defense-in-depth: 404 on PATCH/DELETE to a channel-message path - // is an expected state (provider-side delete or backend GC). Primary - // suppression lives in `authed_json`; this catches any future call - // site that bypasses it. Targets OPENHUMAN-TAURI-R7 (28 events). - if openhuman_core::core::observability::is_channel_message_not_found_event(&event) { - return None; - } - // Drop 401 "Session expired. Please log in again." bodies surfaced - // by llm_provider / backend_api, plus pre-flight "no session token - // stored" guards from the rpc dispatcher. Primary suppression - // lives at the call sites (`openhuman::inference::provider::ops::api_error` - // publishes a SessionExpired event_bus signal and short-circuits; - // the rpc dispatcher's `is_session_expired_error` skip-path in - // `crates/openhuman-rpc/src/server/http/rpc_handler.rs` redirects to a tracing::info). This - // filter catches any future call site that re-emits the same - // shape — keeping OPENHUMAN-TAURI-25 / -1Q / -27 / -1G off - // Sentry permanently (~185 events/day combined). - // Defense-in-depth: drop user-config provider errors that - // slipped past the call-site classifiers — 4xx client errors, - // subscription/payment issues, embedding API authorization - // failures. These are user misconfigurations, not application - // bugs (targets ~22 Sentry issues / ~26k events from the issue - // audit). Primary suppression lives at individual emit sites; - // this catch-all net catches any future new path that bypasses - // those gates. - if openhuman_core::core::observability::is_user_config_provider_event(&event) { - log::debug!( - "[sentry-user-config-filter] dropping user-config provider event event_id={:?}", - event.event_id - ); - return None; - } - // Defense-in-depth: drop connectivity / network flakiness events - // that escaped the call-site classifiers — "Failed to fetch", - // connection refused, gateway 502/504, HTTP 401 from frontend - // connectivity. These are transient self-resolving conditions, - // not actionable code defects (targets ~8 Sentry issues). - if openhuman_core::core::observability::is_connectivity_event(&event) { - log::debug!( - "[sentry-connectivity-filter] dropping connectivity event event_id={:?}", - event.event_id - ); - return None; - } - // Drop events from stale releases (clients running versions - // more than 6 minor versions behind the current build). Errors - // from ancient code are not actionable against the current - // codebase (targets ~4 Sentry issues). - if openhuman_core::core::observability::is_stale_release_event(&event) { - log::debug!( - "[sentry-stale-release-filter] dropping stale release event event_id={:?}", - event.event_id - ); - return None; - } - if openhuman_core::core::observability::is_session_expired_event(&event) { - // Metadata-only log shape — `event.message` carries the raw - // backend response body (often a JSON envelope with the - // session JWT context attached) which CLAUDE.md forbids from - // local logs. `event.event_id` is a correlation-safe Sentry - // uuid that lets triage match the dropped event against the - // breadcrumb chain without leaking the payload. - log::debug!( - "[sentry-session-expired-filter] dropping session-expired event_id={:?}", - event.event_id - ); - return None; - } - // Strip server_name (hostname) to avoid leaking machine identity - event.server_name = None; - // Attach the cached account uid so Sentry can count unique users - // affected by an issue. We only carry `id` — never email, name, - // or IP — so this stays consistent with `send_default_pii: false`. - // - // Issue #3135: the primary source for `event.user` is now the - // Sentry scope, bound proactively at session boundaries - // (credentials::set_credential / clear_credential) and at server - // boot (run_server_inner). The credential identity slot is kept as - // a fallback so any pre-boot / pre-login event that still rides - // the legacy path retains its previous attribution behaviour — - // but we only consult it when the scope hasn't already bound a - // user, otherwise we'd silently clobber the scope binding when - // the slot is empty (root cause of the original userCount=0). - if event.user.is_none() { - event.user = - openhuman_core::security::credentials::identity::peek_credential_user_identity( - ) - .and_then(|identity| identity.id) - .map(|id| sentry::User { - id: Some(id), - ..Default::default() - }); - } - // Scrub secrets from exception values and top-level message. - for exc in &mut event.exception.values { - if let Some(ref value) = exc.value { - exc.value = Some(scrub_secrets(value)); - } - } - if let Some(msg) = event.message.take() { - event.message = Some(scrub_secrets(&msg)); - } - Some(event) - })), - sample_rate: 1.0, - transport: Some(std::sync::Arc::new( - openhuman_core::core::sentry_transport::factory, - )), - ..sentry::ClientOptions::default() - }); + let _sentry_guard = init_sentry(); - // Collect command-line arguments, skipping the binary name. let args: Vec = std::env::args().skip(1).collect(); - - // The core carries no backend client; every CLI subcommand that reaches - // the hosted backend (serve, auth, billing, composio, ...) needs the - // `openhuman-tinyhumans` transport installed before it dispatches. - if let Err(err) = openhuman_tinyhumans::install(openhuman_tinyhumans::InstallOptions::default()) - { - eprintln!("failed to install the TinyHumans backend transport: {err}"); - std::process::exit(1); - } - - // `run` / `serve` start the JSON-RPC server from `openhuman-rpc`, which the - // core cannot depend on; hand it the launcher before dispatching. - openhuman_rpc::server::install_cli_server(); - - // Delegate to the core library to handle the command. - if let Err(err) = openhuman_core::run_core_from_args(&args) { + if let Err(err) = openhuman_rpc::host::cli(&args) { eprintln!("{err}"); std::process::exit(1); } @@ -305,36 +46,46 @@ fn restore_default_sigpipe() { #[cfg(not(unix))] fn restore_default_sigpipe() {} -// --------------------------------------------------------------------------- -// Release / environment resolution for Sentry -// --------------------------------------------------------------------------- +/// Start the Sentry client with embed's shared filter chain. The fallback +/// user id is the core's credential identity slot (the default of +/// [`SentryConfig::new`](process::sentry::SentryConfig::new)); the primary +/// source is the Sentry scope the core binds at login and server boot. +#[cfg(feature = "crash-reporting")] +fn init_sentry() -> process::sentry::sdk::ClientInitGuard { + let config = process::sentry::SentryConfig::new( + sentry_dsn(), + process::sentry::release_tag( + env!("CARGO_PKG_VERSION"), + option_env!("OPENHUMAN_BUILD_SHA"), + ), + resolve_environment(std::env::var("OPENHUMAN_APP_ENV").ok()), + ); + log::debug!( + "[cli] sentry init environment={} dsn_set={}", + config.environment, + config.dsn.is_some() + ); + process::sentry::sdk::init(process::sentry::client_options(config)) +} -/// Canonical release tag: `openhuman@[+]`. -/// -/// Matches the string the frontend reports (`SENTRY_RELEASE` in -/// `app/src/utils/config.ts`) so events from every surface group under -/// the same release in the Sentry dashboard and benefit from the same -/// source-map upload. +/// The core DSN: `OPENHUMAN_CORE_SENTRY_DSN`, then the legacy +/// `OPENHUMAN_SENTRY_DSN`, at runtime and then baked in at compile time. +/// `None` gives a client that sends nothing. #[cfg(feature = "crash-reporting")] -fn build_release_tag() -> String { - let version = env!("CARGO_PKG_VERSION"); - let sha = option_env!("OPENHUMAN_BUILD_SHA").unwrap_or("").trim(); - let sha_short: String = sha.chars().take(12).collect(); - if sha_short.is_empty() { - format!("openhuman@{version}") - } else { - format!("openhuman@{version}+{sha_short}") - } +fn sentry_dsn() -> Option { + process::sentry::first_non_blank([ + std::env::var("OPENHUMAN_CORE_SENTRY_DSN").ok(), + std::env::var("OPENHUMAN_SENTRY_DSN").ok(), + option_env!("OPENHUMAN_CORE_SENTRY_DSN").map(str::to_owned), + option_env!("OPENHUMAN_SENTRY_DSN").map(str::to_owned), + ]) } -/// Resolve the deployment environment reported to Sentry. -/// -/// Honors `OPENHUMAN_APP_ENV` at runtime (`staging` / `production`) so the -/// same binary could in principle be redeployed between environments; falls -/// back to debug/release detection when unset. +/// The deployment environment: `OPENHUMAN_APP_ENV` (lower-cased) when set, +/// else `development` for debug builds and `production` for release builds. #[cfg(feature = "crash-reporting")] -fn resolve_environment() -> String { - if let Ok(value) = std::env::var("OPENHUMAN_APP_ENV") { +fn resolve_environment(app_env: Option) -> String { + if let Some(value) = app_env { let trimmed = value.trim().to_ascii_lowercase(); if !trimmed.is_empty() { return trimmed; @@ -347,19 +98,6 @@ fn resolve_environment() -> String { } } -// --------------------------------------------------------------------------- -// Secret scrubbing -// --------------------------------------------------------------------------- - -/// Sentry `before_send` secret scrubbing. Delegates to the shared, always-on -/// [`openhuman_core::core::log_redaction::scrub_secrets`] so the redaction -/// patterns stay a single source of truth (the same pass also runs on the -/// always-on diagnostic logs in `core::observability`). -#[cfg(feature = "crash-reporting")] -fn scrub_secrets(input: &str) -> String { - openhuman_core::core::log_redaction::scrub_secrets(input) -} - #[cfg(all(test, feature = "crash-reporting"))] #[path = "main_tests.rs"] mod tests; diff --git a/crates/openhuman-cli/src/main_tests.rs b/crates/openhuman-cli/src/main_tests.rs index 31a564137c4..904901d3561 100644 --- a/crates/openhuman-cli/src/main_tests.rs +++ b/crates/openhuman-cli/src/main_tests.rs @@ -1,55 +1,37 @@ use super::*; #[test] -fn scrubs_bearer_token() { - assert_eq!( - scrub_secrets("Authorization: Bearer abc123xyz"), - "Authorization: Bearer [REDACTED]" - ); +fn environment_prefers_app_env_lowercased() { + assert_eq!(resolve_environment(Some(" Staging ".into())), "staging"); } #[test] -fn scrubs_api_key() { - assert_eq!(scrub_secrets("api_key=sk-abc123"), "api_key=[REDACTED]"); +fn environment_falls_back_on_blank_or_missing() { + let expected = if cfg!(debug_assertions) { + "development" + } else { + "production" + }; + assert_eq!(resolve_environment(None), expected); + assert_eq!(resolve_environment(Some(" ".into())), expected); } #[test] -fn scrubs_anthropic_key() { - assert_eq!( - scrub_secrets("key: sk-ant-api03-abcdefghijklmnop"), - "key: [REDACTED]" - ); +fn release_tag_uses_the_crate_version() { + let tag = process::sentry::release_tag(env!("CARGO_PKG_VERSION"), None); + assert_eq!(tag, format!("openhuman@{}", env!("CARGO_PKG_VERSION"))); } #[test] -fn scrubs_openai_admin_key() { +fn the_shared_chain_scrubs_secrets() { + // The CLI no longer carries its own scrubber; this pins that the one it + // installs (embed's) still redacts the shapes the old one did. assert_eq!( - scrub_secrets("key: sk-admin-abcdefghijkl"), - "key: [REDACTED]" + process::sentry::scrub_secrets("Authorization: Bearer abc123xyz"), + "Authorization: Bearer [REDACTED]" ); -} - -#[test] -fn scrubs_openai_proj_key() { assert_eq!( - scrub_secrets("key: sk-proj-abcdefghijkl"), - "key: [REDACTED]" + process::sentry::scrub_secrets("api_key=sk-abc123"), + "api_key=[REDACTED]" ); } - -#[test] -fn scrubs_generic_sk_key() { - assert_eq!(scrub_secrets("sk-abcdefghijklmnopqrstuvwx"), "[REDACTED]"); -} - -#[test] -fn token_word_boundary_no_false_positive() { - let input = "cancellation_token=abc123 next_page_token=xyz789"; - let result = scrub_secrets(input); - assert_eq!(result, input, "should not scrub compound token fields"); -} - -#[test] -fn standalone_token_is_scrubbed() { - assert_eq!(scrub_secrets("token=secret_value_here"), "token=[REDACTED]"); -} diff --git a/crates/openhuman-embed/Cargo.toml b/crates/openhuman-embed/Cargo.toml index 105e9427040..2a40c6a41c1 100644 --- a/crates/openhuman-embed/Cargo.toml +++ b/crates/openhuman-embed/Cargo.toml @@ -32,6 +32,10 @@ channels = ["openhuman-core/channels"] whatsapp-web = ["openhuman-core/whatsapp-web"] file-logging = ["openhuman-core/file-logging"] scheduler-gate = ["openhuman-core/scheduler-gate"] +# Exposes the destructive `openhuman.test_reset` RPC for E2E builds only. +# Default-OFF everywhere; forwarded so the desktop E2E build can reach it +# through the chain instead of naming the core directly. +e2e-test-support = ["openhuman-core/e2e-test-support"] [dependencies] async-trait = "0.1" diff --git a/crates/openhuman-embed/README.md b/crates/openhuman-embed/README.md index 38eb157e569..e5566d8c6b9 100644 --- a/crates/openhuman-embed/README.md +++ b/crates/openhuman-embed/README.md @@ -535,18 +535,12 @@ has the resulting binary sizes and per-agent memory. dropped. - Build the tokio runtime yourself. A turn is a deep async state machine, and a nested sub-agent overflows tokio's default 2 MiB worker stack, so - `#[tokio::main]` is not enough. The constants live in the core crate, so - the host needs the `openhuman` package as a dependency to name them: + `#[tokio::main]` is not enough. `embed::process` builds one sized for + turns (`AGENT_WORKER_STACK_BYTES`, `MAX_BLOCKING_THREADS`, both re-exported + there), so the host needs no core dependency to name them: ```rust,no_run - use openhuman_core::core::runtime::{AGENT_WORKER_STACK_BYTES, MAX_BLOCKING_THREADS}; - - let runtime = tokio::runtime::Builder::new_multi_thread() - .enable_all() - .thread_stack_size(AGENT_WORKER_STACK_BYTES) - .max_blocking_threads(MAX_BLOCKING_THREADS) - .build() - .expect("tokio runtime"); + let runtime = openhuman_embed::process::tokio_runtime().expect("tokio runtime"); ``` - Access has two halves. The autonomy tier drives `SecurityPolicy`, and the @@ -585,8 +579,8 @@ cargo test -p openhuman-embed --features inference,mcp,skills --test runtime_age ``` The repository-root [`examples/embed_headless.rs`](../../examples/embed_headless.rs) and [`examples/embed_kernel.rs`](../../examples/embed_kernel.rs) -drive `CoreBuilder` directly, without this crate. They are `[[example]]` -targets of `openhuman-cli`: +use this crate's `Runtime` (`Runtime::builder()`, then `core_runtime().invoke` +for raw RPC methods). They are `[[example]]` targets of `openhuman-cli`: `cargo run -p openhuman-cli --example embed_headless`. ## Further reading diff --git a/crates/openhuman-embed/src/runtime/build.rs b/crates/openhuman-embed/src/runtime/build.rs index b301f1749e8..b15bd509e39 100644 --- a/crates/openhuman-embed/src/runtime/build.rs +++ b/crates/openhuman-embed/src/runtime/build.rs @@ -223,7 +223,17 @@ impl RuntimeBuilder { } if seams.has_pending_live_policy() { - seams.install_live_policy(&base_config.workspace_dir, &base_config.action_dir); + if config_unavailable.is_some() { + // The base config is a placeholder: its directories are the + // default root, not the operator's install, so a policy + // scoped to them would guard the wrong tree. Keep the policy + // the core's bootstrap installed. + log::warn!( + "[embed][runtime] live policy not installed: discovered config unavailable" + ); + } else { + seams.install_live_policy(&base_config.workspace_dir, &base_config.action_dir); + } } if let Some(session) = self.session.take() { diff --git a/crates/openhuman-embed/src/runtime/mod.rs b/crates/openhuman-embed/src/runtime/mod.rs index 2eab94e084b..c3b8ed7dccc 100644 --- a/crates/openhuman-embed/src/runtime/mod.rs +++ b/crates/openhuman-embed/src/runtime/mod.rs @@ -367,9 +367,19 @@ impl Runtime { .expect("runtime core is present until the last guard owner drops") } - /// The core runtime under this handle, for the transport layer - /// (`openhuman-rpc` serves it) — not for turns, which belong to agents. - #[doc(hidden)] + /// The core runtime under this handle: the controller registry a host + /// dispatches JSON-RPC methods through in-process + /// ([`CoreRuntime::invoke`]). + /// + /// This is the operator-host escape hatch. The JSON-RPC server serves it, + /// and the terminal UI drives its threads, config and auth screens with + /// it. Library embedders should prefer [`Runtime::agent`] for turns and + /// [`Runtime::core`] for typed config/auth access, because a raw invoke + /// carries no agent's provider route or access tier. + /// + /// The handle stays valid while this `Runtime` is alive. Keep the + /// `Runtime` for the whole session: dropping it tears the core down even + /// if a clone of this `Arc` is still held. pub fn core_runtime(&self) -> &Arc { self.core_ref().raw() } diff --git a/crates/openhuman-rpc/Cargo.toml b/crates/openhuman-rpc/Cargo.toml index 262c1e42ad7..2bfe3a08227 100644 --- a/crates/openhuman-rpc/Cargo.toml +++ b/crates/openhuman-rpc/Cargo.toml @@ -52,6 +52,11 @@ hosting = ["openhuman-tinyhumans/hosting"] modules = ["openhuman-tinyhumans/modules"] voice = ["openhuman-tinyhumans/voice"] web3 = ["openhuman-tinyhumans/web3"] +# Storage drivers for `[storage] url` (see the core `storage` domain and +# `session_store::install_for_host`). +storage-sqlite = ["openhuman-tinyhumans/storage-sqlite"] +storage-mongodb = ["openhuman-tinyhumans/storage-mongodb"] +storage-file = ["openhuman-tinyhumans/storage-file"] runtime-node = ["openhuman-tinyhumans/runtime-node"] media = ["openhuman-tinyhumans/media"] flows = ["openhuman-tinyhumans/flows"] @@ -64,6 +69,9 @@ channels = ["openhuman-tinyhumans/channels"] whatsapp-web = ["openhuman-tinyhumans/whatsapp-web"] file-logging = ["openhuman-tinyhumans/file-logging"] scheduler-gate = ["openhuman-tinyhumans/scheduler-gate"] +# E2E builds only (`openhuman.test_reset`); default-OFF. The desktop app +# and the CLI forward it from their own `e2e-test-support` gate. +e2e-test-support = ["openhuman-tinyhumans/e2e-test-support"] [dependencies] # The only openhuman crate this layer depends on. Core internals the server diff --git a/crates/openhuman-rpc/README.md b/crates/openhuman-rpc/README.md index 4dd9b908055..b51c062c948 100644 --- a/crates/openhuman-rpc/README.md +++ b/crates/openhuman-rpc/README.md @@ -20,8 +20,9 @@ openhuman-core -> openhuman-embed -> openhuman-tinyhumans -> openhuman-rpc -> ap Its only openhuman dependency is `openhuman-tinyhumans`. Core internals the server needs come through embed's doc-hidden `__host` list (aliased crate-privately as `core_host` in `src/lib.rs`); nothing here re-exports the -core. Hosts get `openhuman_rpc::tinyhumans` (and through it -`tinyhumans::embed`) as their configuration facade. +core. Hosts get `openhuman_rpc::tinyhumans` and `openhuman_rpc::embed` (the +same crate as `tinyhumans::embed`) as their curated facade, and this crate is +the only OpenHuman crate they depend on (`scripts/ci/check-crate-chain.mjs`). ## How it works @@ -156,21 +157,24 @@ The root workspace declares this crate with `default-features = false`, so each consumer names what it needs. `openhuman-cli` enables `server`, `openhuman-tui` enables `session-store` only (it runs the core without a server), and [`crates/openhuman-app/Cargo.toml`](../openhuman-app/Cargo.toml), outside the workspace, -enables `http-client` and `server`. +enables `http-client`, `server`, `jev` and the product gates. Each host also +forwards its own feature gates here, 1:1, so `e2e-test-support`, the +`storage-*` drivers and every product gate reach the core through this crate. ## Consumers - [`crates/openhuman-app`](../openhuman-app/): `core_process.rs` runs the embedded server - (`run_server_embedded_with_ready`) and reads its `EmbeddedReadySignal`; + (`host::desktop`) and reads its `EmbeddedReadySignal`; `core_rpc.rs` wraps `post_json_rpc` to reach the embedded core and self-hosted runtimes (#3865); `session/link.rs` and `local_data_reset.rs` build requests with `request_body` and decode with `decode_response`; - `lib.rs` calls `install_cli_server` before `run_core_from_args`. -- [`crates/openhuman-cli`](../openhuman-cli/): `main.rs` calls `install_cli_server`; root + `lib.rs::run_core_from_args` calls `host::cli`; the rest of the shell uses + the `embed` and `tinyhumans` re-exports. +- [`crates/openhuman-cli`](../openhuman-cli/): `main.rs` calls `host::cli`; root `tests/*.rs` suites (for example `json_rpc_e2e.rs`) build the router with `build_core_http_router`. - [`crates/openhuman-tui`](../openhuman-tui/): `unwrap_rpc` is its decode point, and - `runner.rs` calls `session_store::install()`. + `runner.rs` boots with `host::tui()`. ## Boundaries diff --git a/crates/openhuman-rpc/src/host.rs b/crates/openhuman-rpc/src/host.rs index e6fd212f0c5..21ddb06383c 100644 --- a/crates/openhuman-rpc/src/host.rs +++ b/crates/openhuman-rpc/src/host.rs @@ -5,7 +5,7 @@ //! |---|---|---| //! | [`cli`] | `tinyhumans::install` → `server::install_cli_server` → `run_core_from_args` | [`cli_builder`]: the `cli` preset, connected, with the server launcher and the `http_host` controllers | //! | [`desktop`] | `tinyhumans::install` + `server::run_server_embedded_with_ready` | [`desktop_builder`]: the `desktop` preset, connected, with the bearer, listener, services, server launcher and `http_host` controllers | -//! | [`tui`] | `tinyhumans::install` + `session_store::install` + `CoreBuilder(full, none)` | [`tui_builder`]: the `tui` preset, connected, with the on-disk session store | +//! | [`tui`] | `tinyhumans::install` + `session_store::install_for_host` + `CoreBuilder(full, none)` | [`tui_builder`]: the `tui` preset, connected, with the on-disk session store ([`tui`] swaps in the configured storage URL's store) | //! //! Each `*_builder` returns a [`tinyhumans::RuntimeBuilder`] so a host can //! adjust it (product identity, hooks, a different ranker) before handing it @@ -20,7 +20,9 @@ //! - The desktop and CLI servers install the on-disk session store for the //! life of the process (see `server::shims::build_and_serve`), exactly as the //! `run_server*` shims do; the TUI hands it to the builder, which restores -//! the previous provider when its runtime drops at exit. +//! the previous provider when its runtime drops at exit. Both honour a +//! storage URL (`OPENHUMAN_STORAGE_URL`, else `[storage] url`): with one, +//! conversations live in that backend instead of the on-disk layout. //! - Builder seams follow embed's install/restore rules: the hosted and //! `http_host` controllers and the server launcher stay for the process; the //! Jev ranker is restored when the runtime drops. A desktop server that @@ -183,14 +185,27 @@ pub fn tui_builder() -> RuntimeBuilder { /// core for the session; `Runtime::core_runtime` hands the TUI its /// `CoreRuntime`. /// +/// Conversations stay in the classic on-disk layout, as the desktop keeps +/// them, unless a storage URL (`OPENHUMAN_STORAGE_URL` / `[storage] url`) is +/// set: then [`tui_builder`]'s on-disk store is replaced with the storage-backed +/// one ([`crate::session_store::provider_for_host`]). +/// /// # Errors /// -/// The transport or runtime could not be built. +/// A configured storage URL could not be opened, or the transport or runtime +/// could not be built. #[cfg(feature = "session-store")] -pub async fn tui( -) -> Result { +pub async fn tui() -> anyhow::Result { log::debug!("[rpc:host] tui: building connected runtime"); - let runtime = tui_builder().build().await?; + let session_store = crate::session_store::provider_for_host().await?; + let runtime = tui_builder() + .session_store(session_store) + .build() + .await + .map_err(|error| { + log::warn!("[rpc:host] tui: runtime build failed: {error}"); + anyhow::Error::new(error) + })?; log::info!("[rpc:host] tui: core built (DomainSet::full, ServiceSet::none)"); Ok(runtime) } diff --git a/crates/openhuman-rpc/src/lib.rs b/crates/openhuman-rpc/src/lib.rs index 2d6534e6123..f6263f5ea5d 100644 --- a/crates/openhuman-rpc/src/lib.rs +++ b/crates/openhuman-rpc/src/lib.rs @@ -28,9 +28,19 @@ //! - [`host`]: the shared host boot, one entry per host shape //! ([`host::cli`], [`host::desktop`], [`host::tui`]). //! - [`tinyhumans`] (and through it `tinyhumans::embed`): the curated library -//! facade hosts configure a runtime with. +//! facade hosts configure a runtime with. [`embed`] is the same crate as +//! `tinyhumans::embed`, re-exported here so a host (app, CLI, TUI) that +//! depends on this crate alone names it in one step. +//! +//! Hosts depend on `openhuman-rpc` and nothing else from this repository +//! (`scripts/ci/check-crate-chain.mjs` enforces it). What they reach is the +//! curated surface above; the doc-hidden `__host` list stays internal to the +//! layers. pub use openhuman_tinyhumans as tinyhumans; +/// The embed facade (`openhuman_embed`): runtime builder, process helpers, +/// config/artifact/chat-surface facades. Same crate as `tinyhumans::embed`. +pub use openhuman_tinyhumans::embed; /// Core internals for this crate's own modules, through embed's doc-hidden /// `__host` list. Crate-private: never re-exported on a public path. diff --git a/crates/openhuman-rpc/src/server/serve.rs b/crates/openhuman-rpc/src/server/serve.rs index f21710b6665..604004a0cde 100644 --- a/crates/openhuman-rpc/src/server/serve.rs +++ b/crates/openhuman-rpc/src/server/serve.rs @@ -205,7 +205,7 @@ pub async fn serve( runtime.exit_cleanup().await; // Close the per-workspace background-completion logs (they replay from disk // on the next boot) so a data reset can delete the workspace directory. - openhuman_core::agent::orchestration::release_background_completion_stores().await; + crate::core_host::agent::orchestration::release_background_completion_stores().await; served?; Ok(()) diff --git a/crates/openhuman-rpc/src/session_store/README.md b/crates/openhuman-rpc/src/session_store/README.md index 16258e05a76..e9fc263e5a4 100644 --- a/crates/openhuman-rpc/src/session_store/README.md +++ b/crates/openhuman-rpc/src/session_store/README.md @@ -39,9 +39,9 @@ run-ledger rows a dead process left running are settled. The `run_server*` shims in [`../server/`](../server/README.md) (and so `host::desktop`) call it for the life of the process. `provider()` returns the same provider for a runtime builder's `session_store` option, which installs it on build and -restores the previous one on drop; `host::tui` uses that. The TUI itself -still calls `install()` from [`crates/openhuman-tui/src/runner.rs`](../../../openhuman-tui/src/runner.rs) until it -moves to `host::tui`. +restores the previous one on drop; `host::tui` uses that (through +`provider_for_host()`, below), and the TUI +([`crates/openhuman-tui/src/runner.rs`](../../../openhuman-tui/src/runner.rs)) boots through `host::tui`. `install()` builds the provider with `resolving(context_workspace_dir)`, so the workspace is looked up on every `for_agent` call rather than fixed at @@ -69,7 +69,9 @@ tinyagents_store/{kv,journal}/ run status, goals, todos, ## A storage-backed store instead -`install_for_host()` is what the server shims and the TUI call. With no +`install_for_host()` is what the server shims call; `host::tui` calls +`provider_for_host()`, which resolves the same URL and returns the provider +for its runtime builder instead of installing it process-wide. With no storage URL configured (`OPENHUMAN_STORAGE_URL`, else `[storage] url` in `config.toml`) it is `install()` above, unchanged. With one, it opens that backend (`openhuman_core::storage::open`), makes it the process's storage @@ -90,14 +92,15 @@ each other's transcripts, turn states, records or journals. A single-process backend (SQLite, memory, files) interrupts an agent's in-flight turns the first time it is opened; MongoDB does not, because another process may own them. A URL that cannot be parsed or opened fails the boot rather than -falling back to local files. `install_for_url` is the same with the URL -already resolved, for tests and hosts that read it themselves. +falling back to local files. `install_for_url` / `provider_for_url` are the +same with the URL already resolved, for tests and hosts that read it +themselves. ## Layout | File | What it does | | --- | --- | -| [`mod.rs`](mod.rs) | `SqliteSessionStores` (`at`, `resolving`), its `SessionStoreProvider` impl (`for_agent`, `recover`, `destination_key`, `workspace_dir`), `install()`, and `install_for_host()` / `install_for_url()`, which install `DriverSessionStores` when a storage URL is configured. | +| [`mod.rs`](mod.rs) | `SqliteSessionStores` (`at`, `resolving`), its `SessionStoreProvider` impl (`for_agent`, `recover`, `destination_key`, `workspace_dir`), `install()`, `install_for_host()` / `install_for_url()`, which install `DriverSessionStores` when a storage URL is configured, and `provider_for_host()` / `provider_for_url()`, which return that provider for a runtime builder. | ## Key types and entry points diff --git a/crates/openhuman-rpc/src/session_store/mod.rs b/crates/openhuman-rpc/src/session_store/mod.rs index bd5b8cfecda..b59a6ccabf1 100644 --- a/crates/openhuman-rpc/src/session_store/mod.rs +++ b/crates/openhuman-rpc/src/session_store/mod.rs @@ -168,7 +168,29 @@ pub fn provider() -> Arc { /// When a URL is configured but cannot be parsed or opened. A deployment that /// asked for a backend must not quietly fall back to local files. pub async fn install_for_host() -> anyhow::Result<()> { - let url = match std::env::var(crate::core_host::storage::STORAGE_URL_VAR) { + install_for_url(configured_storage_url().await).await +} + +/// The session store the host's configuration asks for, as a provider a +/// runtime builder's `session_store` option takes (what [`crate::host::tui`] +/// wires). [`install_for_host`] installs the same provider process-wide +/// instead. +/// +/// Resolves the storage URL exactly as [`install_for_host`] does and has the +/// same side effect on the process's storage backend: a configured backend is +/// opened and installed, and with no URL any earlier backend is cleared. +/// +/// # Errors +/// +/// When a URL is configured but cannot be parsed or opened. +pub async fn provider_for_host() -> anyhow::Result> { + provider_for_url(configured_storage_url().await).await +} + +/// The storage URL the host asks for: `OPENHUMAN_STORAGE_URL`, else +/// `[storage] url` from the config, else `None` (the classic layout). +async fn configured_storage_url() -> Option { + match std::env::var(crate::core_host::storage::STORAGE_URL_VAR) { Ok(url) if !url.trim().is_empty() => Some(url.trim().to_string()), _ => match crate::core_host::config::rpc::load_config_with_timeout().await { Ok(config) => crate::core_host::storage::configured_url(&config), @@ -176,14 +198,13 @@ pub async fn install_for_host() -> anyhow::Result<()> { // layout, as it always has. Remote deployments pin the backend // with `OPENHUMAN_STORAGE_URL`, which never reads the config. Err(error) => { - tracing::warn!( - "[session_store] config unavailable ({error}); keeping the on-disk layout" + log::warn!( + "[rpc:session_store] config unavailable ({error}); keeping the on-disk layout" ); None } }, - }; - install_for_url(url).await + } } /// [`install_for_host`] with the URL already resolved: `None` installs the @@ -194,14 +215,31 @@ pub async fn install_for_host() -> anyhow::Result<()> { /// /// When `url` cannot be parsed or opened. pub async fn install_for_url(url: Option) -> anyhow::Result<()> { + let provider = provider_for_url(url).await?; + log::debug!("[rpc:session_store] installing process-wide (host configuration)"); + crate::core_host::agent::session_store::install(provider); + Ok(()) +} + +/// [`provider_for_host`] with the URL already resolved: `None` is the classic +/// on-disk [`provider`] (any earlier storage backend cleared), a URL opens +/// that backend, makes it the process's storage backend and returns +/// `DriverSessionStores` over it. +/// +/// # Errors +/// +/// When `url` cannot be parsed or opened. +pub async fn provider_for_url( + url: Option, +) -> anyhow::Result> { use anyhow::Context as _; let Some(url) = url else { // Drop a backend an earlier call installed, so storage operations do // not keep writing to it while the classic layout is in force. crate::core_host::storage::clear(); - install(); - return Ok(()); + log::debug!("[rpc:session_store] no storage url; classic on-disk layout"); + return Ok(provider()); }; let backend = crate::core_host::storage::open(&url) .await @@ -212,13 +250,11 @@ pub async fn install_for_url(url: Option) -> anyhow::Result<()> { .recover_on_open(single_process); // Only a fully working bridge makes the backend the process's storage. crate::core_host::storage::install(backend); - tracing::info!( - target: "openhuman_rpc::session_store", - recover_on_open = single_process, - "[session_store] installed the storage-backed session store" + log::info!( + "[rpc:session_store] opened the storage-backed session store \ + recover_on_open={single_process}" ); - crate::core_host::agent::session_store::install(Arc::new(provider)); - Ok(()) + Ok(Arc::new(provider)) } #[cfg(test)] diff --git a/crates/openhuman-rpc/src/session_store/mod_tests.rs b/crates/openhuman-rpc/src/session_store/mod_tests.rs index 106f03bf426..d0d1db7d4fb 100644 --- a/crates/openhuman-rpc/src/session_store/mod_tests.rs +++ b/crates/openhuman-rpc/src/session_store/mod_tests.rs @@ -166,3 +166,66 @@ async fn the_host_reads_the_storage_url_from_the_environment() { assert_eq!(driver, Some("memory")); assert!(refused.is_err(), "an unusable env URL fails the boot"); } + +#[tokio::test] +async fn provider_for_url_returns_the_store_without_installing_it() { + let _turn = SLOTS.lock().await; + let previous = crate::core_host::agent::session_store::installed(); + let previous_backend = crate::core_host::storage::installed(); + crate::core_host::agent::session_store::restore(None); + + let backed = provider_for_url(Some("memory".into())).await.unwrap(); + let backed_key = backed.destination_key(); + let backend_driver = crate::core_host::storage::installed().map(|b| b.driver()); + let left_uninstalled = crate::core_host::agent::session_store::installed().is_none(); + + let classic = provider_for_url(None).await.unwrap(); + let cleared = crate::core_host::storage::installed().is_none(); + + restore_backend(previous_backend); + crate::core_host::agent::session_store::restore(previous); + + assert!( + backed_key + .as_deref() + .is_some_and(|key| key.starts_with("memory://")), + "{backed_key:?}" + ); + assert_eq!(backend_driver, Some("memory"), "the backend is installed"); + assert!( + left_uninstalled, + "the provider is handed back, not installed" + ); + assert!( + classic.workspace_dir().is_some(), + "no URL is the file layout" + ); + assert!(cleared, "no URL clears an earlier backend"); +} + +#[tokio::test] +async fn provider_for_host_reads_the_storage_url_from_the_environment() { + let _turn = SLOTS.lock().await; + let previous_backend = crate::core_host::storage::installed(); + let var = crate::core_host::storage::STORAGE_URL_VAR; + let old = std::env::var_os(var); + + std::env::set_var(var, "memory"); + let booted = provider_for_host().await.map(|p| p.destination_key()); + std::env::set_var(var, "ftp://nowhere"); + let refused = provider_for_host().await; + + match old { + Some(value) => std::env::set_var(var, value), + None => std::env::remove_var(var), + } + restore_backend(previous_backend); + + let key = booted.unwrap(); + assert!( + key.as_deref() + .is_some_and(|key| key.starts_with("memory://")), + "{key:?}" + ); + assert!(refused.is_err(), "an unusable env URL fails the TUI boot"); +} diff --git a/crates/openhuman-tinyhumans/Cargo.toml b/crates/openhuman-tinyhumans/Cargo.toml index da04b85ba0a..e2822687f3c 100644 --- a/crates/openhuman-tinyhumans/Cargo.toml +++ b/crates/openhuman-tinyhumans/Cargo.toml @@ -39,6 +39,8 @@ channels = ["openhuman-embed/channels"] whatsapp-web = ["openhuman-embed/whatsapp-web"] file-logging = ["openhuman-embed/file-logging"] scheduler-gate = ["openhuman-embed/scheduler-gate"] +# E2E builds only (`openhuman.test_reset`); default-OFF. +e2e-test-support = ["openhuman-embed/e2e-test-support"] [dependencies] # The only openhuman crate this layer depends on. Core internals the diff --git a/crates/openhuman-tinyhumans/src/jev/README.md b/crates/openhuman-tinyhumans/src/jev/README.md index c15b37fff62..46c40c3abd1 100644 --- a/crates/openhuman-tinyhumans/src/jev/README.md +++ b/crates/openhuman-tinyhumans/src/jev/README.md @@ -125,8 +125,9 @@ default is 3 s. [`ranker_tests.rs`](ranker_tests.rs), [`route_tests.rs`](route_tests.rs) and [`evaluator_tests.rs`](evaluator_tests.rs) sit beside their modules and use the config and env seams rather than the process -environment. The ranker comparison harness is a separate binary, `tool-search-bench`, in -the `profile/` crate of [openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) +environment. The ranker comparison harness (`tool-search-bench`) moved out of +`openhuman-cli` with the other benchmarks (#6944), to the `profile/` crate of +[openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks) (`cargo run --manifest-path profile/Cargo.toml --bin tool-search-bench`). ```bash diff --git a/crates/openhuman-tui/Cargo.toml b/crates/openhuman-tui/Cargo.toml index af4605dbe70..91052d788b4 100644 --- a/crates/openhuman-tui/Cargo.toml +++ b/crates/openhuman-tui/Cargo.toml @@ -7,27 +7,65 @@ license.workspace = true repository.workspace = true publish = false +# Every gate forwards to `openhuman-rpc` alone (chain: core -> embed -> +# tinyhumans -> rpc -> app/cli/tui); `scripts/ci/check-feature-forwarding.mjs` +# checks the link. `default` is the set the TUI ships with, named explicitly: +# the core's contributor gates minus `http-server` (the TUI drives the core +# in-process and never serves it), plus Sentry. It used to arrive implicitly +# through a bare workspace `openhuman-core` dependency, which turned on the +# core's whole `default`. [features] -default = ["crash-reporting"] -crash-reporting = ["dep:dotenvy", "dep:sentry", "openhuman-core/crash-reporting"] +default = [ + "crash-reporting", + "media", + "skills", + "flows", + "mcp", + "channels", + "scheduler-gate", + "file-logging", + "modules", +] +crash-reporting = ["openhuman-rpc/crash-reporting"] +jev = ["openhuman-rpc/jev"] +http-server = ["openhuman-rpc/http-server"] +inference = ["openhuman-rpc/inference"] +documents = ["openhuman-rpc/documents"] +hosting = ["openhuman-rpc/hosting"] +modules = ["openhuman-rpc/modules"] +voice = ["openhuman-rpc/voice"] +web3 = ["openhuman-rpc/web3"] +# Storage drivers for `[storage] url` / `OPENHUMAN_STORAGE_URL`. +storage-sqlite = ["openhuman-rpc/storage-sqlite"] +storage-mongodb = ["openhuman-rpc/storage-mongodb"] +storage-file = ["openhuman-rpc/storage-file"] +runtime-node = ["openhuman-rpc/runtime-node"] +media = ["openhuman-rpc/media"] +flows = ["openhuman-rpc/flows"] +skills = ["openhuman-rpc/skills"] +mcp = ["openhuman-rpc/mcp"] +channels = ["openhuman-rpc/channels"] +whatsapp-web = ["openhuman-rpc/whatsapp-web"] +file-logging = ["openhuman-rpc/file-logging"] +scheduler-gate = ["openhuman-rpc/scheduler-gate"] +e2e-test-support = ["openhuman-rpc/e2e-test-support"] [[bin]] name = "openhuman-tui" path = "src/main.rs" [dependencies] -openhuman-core.workspace = true +# The only openhuman crate the TUI names: its `host::tui` boot, the embed +# facades (`openhuman_rpc::embed`) and the session owner +# (`openhuman_rpc::tinyhumans`) all come through it. openhuman-rpc = { workspace = true, features = ["session-store"] } -openhuman-tinyhumans.workspace = true anyhow = "1" base64 = "0.22" chrono = "0.4" crossterm = "0.29" -dotenvy = { version = "0.15", optional = true } log = "0.4" ratatui = "0.30" serde_json = "1" -sentry = { version = "0.47.0", default-features = false, optional = true, features = ["backtrace", "contexts", "panic", "tracing", "debug-images", "httpdate"] } async-trait = "0.1" tokio = { version = "1", features = ["rt-multi-thread", "sync", "time"] } unicode-width = "0.2" diff --git a/crates/openhuman-tui/README.md b/crates/openhuman-tui/README.md index 74c950482dc..d6e1199891e 100644 --- a/crates/openhuman-tui/README.md +++ b/crates/openhuman-tui/README.md @@ -16,36 +16,31 @@ the `openhuman-tui` binary next to `openhuman-core`. `crash-reporting` feature is on) and keeps its guard alive, then calls `run_from_cli` with the arguments. `runner.rs` does the rest, in order: -1. Loads `.env` (`core::cli::load_dotenv_for_cli`) and applies any startup - restart delay. +1. Loads `.env` (`embed::process::load_dotenv_for_cli`) and applies any + startup restart delay. 2. Parses flags. `--help` prints usage and returns before anything boots; an unknown `-` flag is an error. 3. Sets transient provider and model overrides - (`core::cli::set_transient_inference_overrides`). + (`embed::process::set_transient_inference_overrides`). 4. Starts file-only logging under the data dir - (`core::logging::init_for_tui`). The TUI owns the terminal, so nothing is + (`embed::process::init_for_tui`). The TUI owns the terminal, so nothing is written to stderr from here on. 5. Initializes the keyring master key. -6. Builds a multi-thread tokio runtime with the core's - `AGENT_WORKER_STACK_BYTES` stack size, so nested agent turns do not +6. Builds `embed::process::tokio_runtime()`, a multi-thread runtime with the + core's `AGENT_WORKER_STACK_BYTES` stack size, so nested agent turns do not overflow the default 2 MiB stack. -7. Inside the runtime (`async_main`): installs the TinyHumans backend - transport, installs the on-disk session store - (`openhuman_rpc::session_store::install`), and builds the core: - - ```rust - CoreBuilder::new(HostKind::detect_standalone()) - .domains(DomainSet::full()) - .services(ServiceSet::none()) - .backend_transport(backend_transport) - .build() - ``` - - `DomainSet::full()` is needed because `channel.web_chat` lives in the +7. Inside the runtime (`async_main`): boots the core with + `openhuman_rpc::host::tui()` — the embed `tui` preset (every domain, no + background services), connected to the TinyHumans backend, with the + on-disk session store — and keeps the returned `Runtime` alive for the + session. The screens drive it through `Runtime::core_runtime()` + (`CoreRuntime::invoke`). + + Every domain is needed because `channel.web_chat` lives in the Channels domain group. `ServiceSet::none()` skips background services, including channel startup, so the runner registers the approval and artifact surface subscribers itself - (`web_chat::register_approval_surface_subscriber`, + (`embed::chat_surface::register_approval_surface_subscriber`, `register_artifact_surface_subscriber`). 8. Picks a client id (`tui-<12 hex>`), resolves the thread, subscribes to the web-channel broadcast before the first turn, and hands off to `app::run`. @@ -191,16 +186,19 @@ All calls go through `CoreRuntime::invoke`: | Feature | Meaning | | --- | --- | -| `crash-reporting` (default) | Pulls in `sentry` and `dotenvy` and forwards `openhuman-core/crash-reporting`. `init_crash_reporting` loads `.env`, then starts a Sentry client that scrubs secrets from exception values and messages (`core::log_redaction::scrub_secrets`) and uses the core's Sentry transport. | +| `crash-reporting` (default) | Forwards `openhuman-rpc/crash-reporting`. `init_crash_reporting` loads `.env`, then starts a Sentry client from `embed::process::sentry::client_options`: the shared filter chain, secret scrubbing and the core's Sentry transport, as the CLI and the desktop shell use. | +| `media`, `skills`, `flows`, `mcp`, `channels`, `scheduler-gate`, `file-logging`, `modules` (default) | The core's contributor gates, forwarded to `openhuman-rpc`. Named explicitly: they used to arrive implicitly through a bare `openhuman-core` workspace dependency. `http-server` is not in the set; the TUI never serves. | +| every other product gate, `jev`, `e2e-test-support` | Forwarded 1:1 to `openhuman-rpc`, off by default (`scripts/ci/check-feature-forwarding.mjs` checks the link). | ## Boundaries -- The core runs in-process through `openhuman-core`; there is no - `openhuman-core` binary to spawn or connect to. -- `openhuman-rpc` is used for the session store (`session-store` feature) and - `unwrap_rpc`, which strips the optional `result`/`data` envelopes around - RPC payloads. The TUI does not use its server or HTTP client. -- `openhuman-tinyhumans` provides the backend transport and the +- `openhuman-rpc` is the TUI's only OpenHuman dependency + (`scripts/ci/check-crate-chain.mjs`). The core runs in-process through + `host::tui`; there is no `openhuman-core` binary to spawn or connect to. + The TUI also uses `unwrap_rpc` (strips the optional `result`/`data` + envelopes around RPC payloads) and the `embed` facades; it does not use the + server or the HTTP client. +- `openhuman_rpc::tinyhumans` provides the backend transport and the `SessionManager`; auth endpoint behavior belongs there. - The ratatui and crossterm dependencies live only in this crate, which keeps the core free of terminal code. diff --git a/crates/openhuman-tui/src/app.rs b/crates/openhuman-tui/src/app.rs index 85b51eda9b5..6a187a4c448 100644 --- a/crates/openhuman-tui/src/app.rs +++ b/crates/openhuman-tui/src/app.rs @@ -22,9 +22,9 @@ use crossterm::event::{self, Event, KeyCode, KeyEvent, KeyEventKind, KeyModifier use serde_json::json; use tokio::sync::broadcast; -use openhuman_core::core::runtime::CoreRuntime; -use openhuman_core::web_chat; -use openhuman_core::web_chat::WebChannelEvent; +use openhuman_rpc::embed::chat_surface as web_chat; +use openhuman_rpc::embed::chat_surface::WebChannelEvent; +use openhuman_rpc::embed::CoreRuntime; use super::cockpit::{ array_at, row_from_value, Overlay, OverlayKind, OverlayRow, PendingApproval, PendingPlanReview, diff --git a/crates/openhuman-tui/src/controls.rs b/crates/openhuman-tui/src/controls.rs index dd592c9da4c..4a8c2f36238 100644 --- a/crates/openhuman-tui/src/controls.rs +++ b/crates/openhuman-tui/src/controls.rs @@ -6,7 +6,7 @@ use crossterm::event::{KeyCode, KeyEvent, KeyModifiers}; use serde_json::json; use zeroize::Zeroize; -use openhuman_core::core::runtime::CoreRuntime; +use openhuman_rpc::embed::CoreRuntime; use super::ui_state::{ConfigKey, SettingsAction, UiState}; diff --git a/crates/openhuman-tui/src/crash_reporting.rs b/crates/openhuman-tui/src/crash_reporting.rs index 96b3ff5dc93..9f3c8a93600 100644 --- a/crates/openhuman-tui/src/crash_reporting.rs +++ b/crates/openhuman-tui/src/crash_reporting.rs @@ -1,77 +1,58 @@ //! Crash-reporting client ownership for the standalone terminal binary. +//! +//! The client options come from embed's shared chain +//! (`openhuman_rpc::embed::process::sentry`): the same noise filters, secret +//! scrubbing, PII-off defaults and transport as the CLI and the desktop shell. +//! Only the DSN, release and environment are resolved here. + +#[cfg(feature = "crash-reporting")] +use openhuman_rpc::embed::process::{self, sentry}; /// Initialize the Sentry client before the TUI installs its tracing subscriber. /// /// The returned guard must live for the whole process. A build without the /// `crash-reporting` feature retains the same call site and compiles to a no-op. #[cfg(feature = "crash-reporting")] -pub fn init_crash_reporting() -> sentry::ClientInitGuard { +pub fn init_crash_reporting() -> sentry::sdk::ClientInitGuard { // Match the core binary: startup consumers must see a repository-local // `.env`, while explicit process variables continue to take precedence. - let _ = dotenvy::dotenv(); + // Failure is not fatal here; `run_from_cli` loads it again and reports it. + let _ = process::load_dotenv_for_cli(); - sentry::init(sentry::ClientOptions { - dsn: sentry_dsn(), - release: Some(std::borrow::Cow::Owned(build_release_tag())), - environment: Some(std::borrow::Cow::Owned(resolve_environment())), - send_default_pii: false, - before_send: Some(std::sync::Arc::new(|mut event| { - event.server_name = None; - for exception in &mut event.exception.values { - if let Some(value) = exception.value.take() { - exception.value = - Some(openhuman_core::core::log_redaction::scrub_secrets(&value)); - } - } - if let Some(message) = event.message.take() { - event.message = Some(openhuman_core::core::log_redaction::scrub_secrets(&message)); - } - Some(event) - })), - sample_rate: 1.0, - transport: Some(std::sync::Arc::new( - openhuman_core::core::sentry_transport::factory, - )), - ..sentry::ClientOptions::default() - }) + let config = sentry::SentryConfig::new( + sentry_dsn(), + build_release_tag(), + resolve_environment(std::env::var("OPENHUMAN_APP_ENV").ok()), + ); + sentry::sdk::init(sentry::client_options(config)) } #[cfg(not(feature = "crash-reporting"))] pub fn init_crash_reporting() {} +/// `OPENHUMAN_CORE_SENTRY_DSN`, then the legacy `OPENHUMAN_SENTRY_DSN`, at +/// runtime and then baked in at compile time. #[cfg(feature = "crash-reporting")] -fn sentry_dsn() -> Option { - std::env::var("OPENHUMAN_CORE_SENTRY_DSN") - .ok() - .filter(|value| !value.is_empty()) - .or_else(|| std::env::var("OPENHUMAN_SENTRY_DSN").ok()) - .filter(|value| !value.is_empty()) - .or_else(|| option_env!("OPENHUMAN_CORE_SENTRY_DSN").map(str::to_owned)) - .filter(|value| !value.is_empty()) - .or_else(|| option_env!("OPENHUMAN_SENTRY_DSN").map(str::to_owned)) - .filter(|value| !value.is_empty()) - .and_then(|value| value.parse().ok()) +fn sentry_dsn() -> Option { + sentry::first_non_blank([ + std::env::var("OPENHUMAN_CORE_SENTRY_DSN").ok(), + std::env::var("OPENHUMAN_SENTRY_DSN").ok(), + option_env!("OPENHUMAN_CORE_SENTRY_DSN").map(str::to_owned), + option_env!("OPENHUMAN_SENTRY_DSN").map(str::to_owned), + ]) } #[cfg(feature = "crash-reporting")] fn build_release_tag() -> String { - let version = env!("CARGO_PKG_VERSION"); - let short_sha: String = option_env!("OPENHUMAN_BUILD_SHA") - .unwrap_or("") - .trim() - .chars() - .take(12) - .collect(); - if short_sha.is_empty() { - format!("openhuman@{version}") - } else { - format!("openhuman@{version}+{short_sha}") - } + sentry::release_tag( + env!("CARGO_PKG_VERSION"), + option_env!("OPENHUMAN_BUILD_SHA"), + ) } #[cfg(feature = "crash-reporting")] -fn resolve_environment() -> String { - if let Ok(value) = std::env::var("OPENHUMAN_APP_ENV") { +fn resolve_environment(app_env: Option) -> String { + if let Some(value) = app_env { let value = value.trim().to_ascii_lowercase(); if !value.is_empty() { return value; diff --git a/crates/openhuman-tui/src/crash_reporting_tests.rs b/crates/openhuman-tui/src/crash_reporting_tests.rs index 9914d6c528b..5677642e63c 100644 --- a/crates/openhuman-tui/src/crash_reporting_tests.rs +++ b/crates/openhuman-tui/src/crash_reporting_tests.rs @@ -13,3 +13,14 @@ fn release_tag_uses_the_package_version_without_a_build_sha() { ); } } + +#[test] +fn environment_prefers_app_env_lowercased() { + assert_eq!(resolve_environment(Some(" Staging ".into())), "staging"); + let fallback = if cfg!(debug_assertions) { + "development" + } else { + "production" + }; + assert_eq!(resolve_environment(Some(" ".into())), fallback); +} diff --git a/crates/openhuman-tui/src/lib.rs b/crates/openhuman-tui/src/lib.rs index 04a3f38cc29..81e20711eb1 100644 --- a/crates/openhuman-tui/src/lib.rs +++ b/crates/openhuman-tui/src/lib.rs @@ -5,10 +5,12 @@ //! agent/skill/MCP/artifact views, Git review, and a multiline composer. //! Chat uses the **same `web_chat` surface** the desktop app drives (`openhuman.channel_web_chat` / //! `openhuman.channel_web_cancel` + -//! [`web_chat::subscribe_web_channel_events`](openhuman_core::web_chat::subscribe_web_channel_events)). -//! It boots the core in-process — no HTTP, no sockets — via -//! `CoreBuilder::new(HostKind::Cli).domains(DomainSet::full()).services(ServiceSet::none())` -//! and streams a live transcript in the terminal. +//! [`web_chat::subscribe_web_channel_events`](openhuman_rpc::embed::chat_surface::subscribe_web_channel_events)). +//! It boots the core in-process — no HTTP, no sockets — through +//! `openhuman_rpc::host::tui` (the embed `tui` preset: every domain, no +//! background services, the on-disk session store, connected to the +//! TinyHumans backend) and streams a live transcript in the terminal. +//! `openhuman-rpc` is its only openhuman dependency. //! //! The terminal dependencies and UI code live entirely in this crate, keeping //! the shared core crate free of terminal-specific dependencies. diff --git a/crates/openhuman-tui/src/render.rs b/crates/openhuman-tui/src/render.rs index 7e789f18335..09d59cb9f12 100644 --- a/crates/openhuman-tui/src/render.rs +++ b/crates/openhuman-tui/src/render.rs @@ -81,7 +81,7 @@ fn draw_chat(frame: &mut Frame, area: Rect, state: &TranscriptState, ui: &UiStat } fn draw_logs(frame: &mut Frame, area: Rect, ui: &UiState) { - let lines = openhuman_core::core::logging::tui_log_lines(); + let lines = openhuman_rpc::embed::process::tui_log_lines(); let text = if lines.is_empty() { Text::from("Core logs will appear here as OpenHuman starts.") } else { diff --git a/crates/openhuman-tui/src/runner.rs b/crates/openhuman-tui/src/runner.rs index 8bfe6664a9c..59939a204c2 100644 --- a/crates/openhuman-tui/src/runner.rs +++ b/crates/openhuman-tui/src/runner.rs @@ -1,19 +1,19 @@ //! CLI entry point for the tabbed terminal UI (`openhuman` / `tui` / `chat`). //! //! Parses flags, initializes **file-only** logging (the TUI owns the terminal — -//! see `logging::init_for_tui`), boots the core in-process with no transport and -//! no background services, resolves the target thread, and hands off to the -//! event loop in [`super::app`]. +//! see `embed::process::init_for_tui`), boots the core in-process through +//! [`openhuman_rpc::host::tui`] (no transport, no background services), +//! resolves the target thread, and hands off to the event loop in +//! [`super::app`]. use std::path::PathBuf; use std::sync::Arc; use serde_json::{json, Value}; -use openhuman_core::core::runtime::{ - CoreBuilder, CoreRuntime, DomainSet, ServiceSet, AGENT_WORKER_STACK_BYTES, MAX_BLOCKING_THREADS, -}; -use openhuman_core::core::types::HostKind; +use openhuman_rpc::embed::chat_surface; +use openhuman_rpc::embed::process; +use openhuman_rpc::embed::CoreRuntime; /// Entry point for the `openhuman-tui` executable. /// @@ -26,8 +26,8 @@ use openhuman_core::core::types::HostKind; /// * a positional prompt — send immediately after startup. /// * `-v` / `--verbose` — debug-level file logging. pub fn run_from_cli(args: &[String]) -> anyhow::Result<()> { - openhuman_core::core::cli::load_dotenv_for_cli()?; - openhuman_core::platform::service::apply_startup_restart_delay_from_env(); + process::load_dotenv_for_cli()?; + process::apply_startup_restart_delay_from_env(); let mut thread_id: Option = None; let mut force_new = false; @@ -100,17 +100,14 @@ pub fn run_from_cli(args: &[String]) -> anyhow::Result<()> { } } - openhuman_core::core::cli::set_transient_inference_overrides( - provider.as_deref(), - model.as_deref(), - ); + process::set_transient_inference_overrides(provider.as_deref(), model.as_deref()); // File-only logging — never stderr while the TUI owns the terminal. let data_dir = resolve_data_dir(); - let log_dir = openhuman_core::core::logging::init_for_tui(&data_dir, verbose); + let log_dir = process::init_for_tui(&data_dir, verbose); // After argument parsing so `--help` works while a configured master key // is being fixed, and after logging so a rejection reaches the log file. - openhuman_core::security::keyring::init_master_key().map_err(anyhow::Error::msg)?; + process::init_master_key()?; log::info!( "[tui] starting tabbed terminal UI (thread={:?} new={} logs={:?})", thread_id, @@ -121,11 +118,7 @@ pub fn run_from_cli(args: &[String]) -> anyhow::Result<()> { // A chat turn is a large async state machine that can delegate to // sub-agents; give the tokio workers the same roomy stack the server uses // so a nested turn cannot overflow the default 2 MiB stack. - let rt = tokio::runtime::Builder::new_multi_thread() - .enable_all() - .thread_stack_size(AGENT_WORKER_STACK_BYTES) - .max_blocking_threads(MAX_BLOCKING_THREADS) - .build()?; + let rt = process::tokio_runtime()?; let options = super::app::LaunchOptions { initial_prompt: (!prompt_parts.is_empty()).then(|| prompt_parts.join(" ")), resume_picker, @@ -168,41 +161,34 @@ async fn async_main( prefer_existing: bool, options: super::app::LaunchOptions, ) -> anyhow::Result<()> { - // The core reaches the hosted backend (login, billing, integrations) only - // through the transport `openhuman-tinyhumans` installs; bind it to the - // runtime explicitly rather than relying on the process global. - let backend_transport = - openhuman_tinyhumans::install(openhuman_tinyhumans::InstallOptions::default())?; - - // In-process core: full domains (channel.web_chat needs DomainGroup::Channels, - // so harness() is not enough), no RPC transport, no background services. - // Conversations in the classic on-disk layout, as the desktop keeps them, - // unless a storage URL (`OPENHUMAN_STORAGE_URL` / `[storage] url`) is set. - openhuman_rpc::session_store::install_for_host().await?; - let runtime = Arc::new( - CoreBuilder::new(HostKind::detect_standalone()) - .domains(DomainSet::full()) - .services(ServiceSet::none()) - .backend_transport(backend_transport) - .build() - .await?, - ); - log::info!("[tui] core built (DomainSet::full, ServiceSet::none)"); + // In-process core, connected to the TinyHumans backend (login, billing, + // integrations): full domains (channel.web_chat needs the channels family, + // so harness() is not enough), no RPC transport, no background services, + // and conversations in the classic on-disk layout, as the desktop keeps + // them, unless a storage URL (`OPENHUMAN_STORAGE_URL` / `[storage] url`) + // is set. `host` owns the core for the whole session: it must outlive the + // event loop below, so it is held until `app::run` returns. + let host = openhuman_rpc::host::tui().await?; + let runtime = Arc::clone(host.core_runtime()); + log::debug!("[tui] runtime ready runtime_id={}", host.runtime_id()); // ServiceSet::none intentionally skips channel startup. The TUI is itself // an interactive surface, so bridge approval, plan-review, artifact, and // agent progress events onto the same in-process web-channel stream. - openhuman_core::web_chat::register_approval_surface_subscriber(); - openhuman_core::web_chat::register_artifact_surface_subscriber(); + chat_surface::register_approval_surface_subscriber(); + chat_surface::register_artifact_surface_subscriber(); let client_id = format!("tui-{}", short_hex()); let thread_id = resolve_thread(&runtime, thread_flag, force_new, prefer_existing).await?; log::info!("[tui] resolved thread={thread_id} client_id={client_id}"); // Subscribe BEFORE the first turn so no streamed event is missed. - let web_rx = openhuman_core::web_chat::subscribe_web_channel_events(); + let web_rx = chat_surface::subscribe_web_channel_events(); - super::app::run(runtime, client_id, thread_id, web_rx, options).await + let result = super::app::run(runtime, client_id, thread_id, web_rx, options).await; + drop(host); + log::debug!("[tui] runtime released ok={}", result.is_ok()); + result } /// Resolve the thread to open: the `--thread` id (unless `--new`), otherwise a @@ -267,7 +253,7 @@ fn resolve_data_dir() -> PathBuf { return PathBuf::from(workspace); } } - openhuman_core::config::default_root_openhuman_dir() + openhuman_rpc::embed::config::default_root_openhuman_dir() .unwrap_or_else(|_| std::env::temp_dir().join("openhuman")) } diff --git a/crates/openhuman-tui/src/runner_tests.rs b/crates/openhuman-tui/src/runner_tests.rs index 085df170f11..bc59cdc902e 100644 --- a/crates/openhuman-tui/src/runner_tests.rs +++ b/crates/openhuman-tui/src/runner_tests.rs @@ -67,17 +67,17 @@ fn tui_invokes_use_canonical_registered_rpc_method_names() { "openhuman.auth_get_state", // Session ownership lives in `openhuman-tinyhumans`; these are the // credential RPCs its `CoreLink` drives through the in-process link. - openhuman_tinyhumans::link::AUTH_GET_STATE, - openhuman_tinyhumans::link::AUTH_GET_SESSION_TOKEN, - openhuman_tinyhumans::link::AUTH_SET_CREDENTIAL, - openhuman_tinyhumans::link::AUTH_CLEAR_CREDENTIAL, - openhuman_tinyhumans::link::CONFIG_RESOLVE_API_URL, + openhuman_rpc::tinyhumans::link::AUTH_GET_STATE, + openhuman_rpc::tinyhumans::link::AUTH_GET_SESSION_TOKEN, + openhuman_rpc::tinyhumans::link::AUTH_SET_CREDENTIAL, + openhuman_rpc::tinyhumans::link::AUTH_CLEAR_CREDENTIAL, + openhuman_rpc::tinyhumans::link::CONFIG_RESOLVE_API_URL, ]; methods.push("openhuman.skills_list"); methods.push("openhuman.mcp_clients_installed_list"); for method in methods { assert!( - openhuman_core::core::all::schema_for_rpc_method(method).is_some(), + openhuman_rpc::embed::schema_for_rpc_method(method).is_some(), "TUI invokes `{method}`, but it is not a registered RPC method — \ the tabbed terminal UI would fail with `unknown method: {method}`" ); diff --git a/crates/openhuman-tui/src/session.rs b/crates/openhuman-tui/src/session.rs index 02cc6b1676a..85cb0089a73 100644 --- a/crates/openhuman-tui/src/session.rs +++ b/crates/openhuman-tui/src/session.rs @@ -6,8 +6,8 @@ use std::sync::{Arc, OnceLock}; use async_trait::async_trait; -use openhuman_core::core::runtime::CoreRuntime; -use openhuman_tinyhumans::{ClientHeaders, CoreLink, SessionManager}; +use openhuman_rpc::embed::CoreRuntime; +use openhuman_rpc::tinyhumans::{ClientHeaders, CoreLink, SessionManager}; /// `CoreLink` over `CoreRuntime::invoke` — no HTTP, no bearer. pub struct InProcessLink(pub Arc); @@ -30,7 +30,7 @@ static MANAGER: OnceLock>> = OnceLock::new(); /// same manager regardless of the handle they pass. pub fn session_manager(runtime: &Arc) -> Arc> { Arc::clone(MANAGER.get_or_init(|| { - let headers = ClientHeaders::new(openhuman_tinyhumans::product_identity().as_str()) + let headers = ClientHeaders::new(openhuman_rpc::tinyhumans::product_identity().as_str()) .with_core_version(env!("CARGO_PKG_VERSION")); SessionManager::new(Arc::new(InProcessLink(Arc::clone(runtime))), headers) })) diff --git a/crates/openhuman-tui/src/state.rs b/crates/openhuman-tui/src/state.rs index 57708d3670c..8ac5fcf912a 100644 --- a/crates/openhuman-tui/src/state.rs +++ b/crates/openhuman-tui/src/state.rs @@ -10,7 +10,7 @@ //! into the transcript. Events for a different `client_id` are ignored, so a //! process-wide broadcast bus can be drained safely. -use openhuman_core::web_chat::WebChannelEvent; +use openhuman_rpc::embed::chat_surface::WebChannelEvent; /// The kind of a transcript entry — drives colour / prefix in the renderer. #[derive(Debug, Clone, Copy, PartialEq, Eq)] diff --git a/docs/library-minimal-recipe.md b/docs/library-minimal-recipe.md index 04c45ecb4bc..b534d3e021b 100644 --- a/docs/library-minimal-recipe.md +++ b/docs/library-minimal-recipe.md @@ -1,5 +1,10 @@ # Library-minimal feature recipe +> **Note.** The bench bins (`library-profile`, `rss-bench`) referenced below +> moved to the openhuman-benchmarks repository (#6944). The `openhuman-cli` +> `rss-bench` / `rss-bench-dhat` gates are gone; the core keeps its own +> `rss-bench` hook for that repository. + A **supported, measured** compile-time feature recipe for embedding the OpenHuman Rust core as a library in "opencompany" — headless, no RPC server, no Tauri shell, targeting 100-1000 live agents in a 2 GB RAM / 2 vCPU box. diff --git a/examples/embed_headless.rs b/examples/embed_headless.rs index d7f836e3b14..d3bdaa3132a 100644 --- a/examples/embed_headless.rs +++ b/examples/embed_headless.rs @@ -1,14 +1,16 @@ //! Embed the OpenHuman core as a library — no HTTP, no background services. //! -//! Demonstrates the pluggable-core API: build a fully-initialized core with -//! [`ServiceSet::none`] (no ports bound, no cron/channels/login-gated services) AND +//! Demonstrates the library API, `openhuman_embed::Runtime`: build a +//! fully-initialized core with [`ServiceSet::none`] (no ports bound, no cron/channels/login-gated services) AND //! [`DomainSet::harness`] (only the agent + memory + threads + config + security //! domain families are live — the gate families flows/skills/mcp/meet/channels/ //! web3/voice/media and the catch-all `platform` are off, so their controllers //! are unknown-method, their agent tools absent, and their stores/subscribers //! never initialize). Dispatch RPC methods in-process through -//! [`CoreRuntime::invoke`] — the exact same path the HTTP `/rpc` handler and the -//! CLI use. +//! [`Runtime::core_runtime`]'s `invoke` — the exact same path the HTTP `/rpc` +//! handler and the CLI use. The workspace is ephemeral (the library default), +//! so running this touches nothing in `~/.openhuman`. Turns belong to agents: +//! see `Runtime::agent` for the typed path. //! //! Run with: //! @@ -16,15 +18,12 @@ //! cargo run --example embed_headless //! ``` //! -//! To instead expose the core over HTTP for a single-core cloud deployment, -//! swap `ServiceSet::none()` for `ServiceSet::headless_api()`, keep a -//! `CancellationToken`, and call `runtime.serve(None, Some(token)).await` — it -//! binds `127.0.0.1:7788` (override with `.host(..)` / `.port(..)` on the -//! builder, or `OPENHUMAN_CORE_HOST` / `OPENHUMAN_CORE_PORT`) and serves until -//! the token is cancelled. Widen the runtime surface by swapping -//! `DomainSet::harness()` for `DomainSet::full()`. +//! To instead expose the core over HTTP, use `openhuman-rpc`'s host entries +//! (`openhuman_rpc::host::serve_desktop` / the `openhuman-core serve` CLI), +//! which serve a runtime like this one over JSON-RPC. Widen the runtime surface +//! by swapping `DomainSet::harness()` for `DomainSet::full()`. -use openhuman_core::{CoreBuilder, DomainSet, HostKind, ServiceSet}; +use openhuman_embed::{DomainSet, HostKind, Runtime, ServiceSet}; #[tokio::main] async fn main() -> anyhow::Result<()> { @@ -32,28 +31,30 @@ async fn main() -> anyhow::Result<()> { // self-contained. (`RUST_LOG=info cargo run --example embed_headless`) let _ = env_logger::builder().is_test(false).try_init(); - // Initialize the core against the local workspace. `HostKind::Cli` selects + // Initialize the core on an ephemeral workspace. `HostKind::Cli` selects // the standalone (non-desktop) bootstrap path; `DomainSet::harness()` builds // the embeddable agent core; `ServiceSet::none()` means no transport and no // background services are started. - let runtime = CoreBuilder::new(HostKind::Cli) + let runtime = Runtime::builder() + .host_kind(HostKind::Cli) .domains(DomainSet::harness()) .services(ServiceSet::none()) .build() .await?; + let core = runtime.core_runtime(); // Dispatch a couple of RPC methods in-process — no network involved. // `core.version` and `openhuman.ping` (a legacy alias for the built-in // `core.ping`) are always available regardless of the DomainSet — they are // transport built-ins, not domain controllers — so they succeed even under // `harness()`. - let version = runtime + let version = core .invoke("core.version", serde_json::json!({})) .await .map_err(|e| anyhow::anyhow!("core.version failed: {e}"))?; println!("core.version -> {version}"); - let ping = runtime + let ping = core .invoke("openhuman.ping", serde_json::json!({})) .await .map_err(|e| anyhow::anyhow!("openhuman.ping failed: {e}"))?; diff --git a/examples/embed_kernel.rs b/examples/embed_kernel.rs index cb145cf33fa..824e622c21a 100644 --- a/examples/embed_kernel.rs +++ b/examples/embed_kernel.rs @@ -33,7 +33,10 @@ //! cargo run --example embed_kernel //! ``` -use openhuman_core::{CoreBuilder, DomainSet, HostKind, ServiceSet}; +//! Built with the library API (`openhuman_embed::Runtime`) on an ephemeral +//! workspace, so running it touches nothing in `~/.openhuman`. + +use openhuman_embed::{DomainSet, HostKind, Runtime, ServiceSet}; #[tokio::main] async fn main() -> anyhow::Result<()> { @@ -43,21 +46,23 @@ async fn main() -> anyhow::Result<()> { let mut domains = DomainSet::kernel(); domains.memory = true; - let runtime = CoreBuilder::new(HostKind::Cli) + let runtime = Runtime::builder() + .host_kind(HostKind::Cli) .domains(domains) .services(ServiceSet::none()) .build() .await?; + let core = runtime.core_runtime(); // Always available — `core.*` is kernel transport, not a domain. - let version = runtime + let version = core .invoke("core.version", serde_json::json!({})) .await .map_err(|e| anyhow::anyhow!("core.version failed: {e}"))?; println!("core.version -> {version}"); // Enabled: memory was opted in above. - match runtime + match core .invoke("openhuman.memory_list_namespaces", serde_json::json!({})) .await { @@ -69,7 +74,7 @@ async fn main() -> anyhow::Result<()> { // not a registered handler returning "agent disabled". Absence is the // contract: a registered-but-failing method teaches a model the capability // exists and makes it retry. - match runtime + match core .invoke("openhuman.agent_list_definitions", serde_json::json!({})) .await { diff --git a/gitbooks/developing/architecture.md b/gitbooks/developing/architecture.md index 2370715f8a6..d5965a26dee 100644 --- a/gitbooks/developing/architecture.md +++ b/gitbooks/developing/architecture.md @@ -20,20 +20,61 @@ OpenHuman ships for desktop only: Windows, macOS and Linux. Web is not a support | Path | Contents | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `app/` | pnpm workspace `openhuman-app`: Vite/React UI (`app/src/`), Vitest and WDIO tests. The Tauri shell itself is the Rust crate `crates/openhuman-app/` (below). | -| `crates/openhuman-app/` | Thin Tauri v2 desktop host (Cargo package `openhuman-app`). Built from its own manifest and lockfile, excluded from the root workspace; hosts the core as an in-process tokio task (`src/core_process.rs`). | +| `crates/openhuman-app/` | Thin Tauri v2 desktop host (Cargo package `openhuman-app`). Built from its own manifest and lockfile, excluded from the root workspace; depends on `openhuman-rpc` only and hosts the core as an in-process tokio task (`src/core_process.rs`, `openhuman_rpc::host::desktop`). | | `crates/openhuman-core/` | Cargo package `openhuman`, library `openhuman_core`, with no `tinyhumans-sdk` dependency. The hosted backend is reached through the `backend::transport::BackendTransport` port, except hosted memory, which `vendor/tinymemory`'s own HTTP client calls (see `memory/engine.rs`). Flat domain modules directly under `src/` (`agent`, `backend`, `channels`, `config`, `cron`, `desktop`, `flows`, `hooks`, `hosting`, `inference`, `integrations`, `mcp`, `media`, `memory`, `modules`, `platform`, `runtime`, `sandbox`, `search`, `security`, `skills`, `threads`, `tools`, `util`, `voice`, `web3`, `web_chat`, …). `src/core/` holds the CLI (`cli.rs`), dispatch, controller registry (`all.rs`), event bus (`bus.rs`), `runtime/` (`CoreBuilder`/`CoreRuntime`) and `subsystem/`. | | `crates/openhuman-rpc/` | Shared JSON-RPC contracts: `RpcOutcome`, `unwrap_rpc`, `apply_log_envelope`, `StructuredRpcError`. The `http-client` feature (default on; off for root-workspace consumers) adds `post_json_rpc`, `bearer_header`, `redact_url_for_log`, `HttpRpcResponse`. Used by the Tauri shell (`core_rpc.rs` → `relay_http_rpc`) and the TUI (envelope decoding). The core does not depend on this crate and does not re-export it: the controller contract (`Outcome`, `ControllerSchema`, `StructuredRpcError`, params rules, in-process dispatch) lives in `crates/openhuman-core/src/core/`, and this crate sits above it. Behind `server` it also owns the `http_host` static-directory file server and registers its controllers as a core extension. | -| `crates/openhuman-embed/` | Typed library facade (`openhuman_embed::{Harness, Core, CoreBuilder, DomainSet, ServiceSet, HostKind}`) for embedding the core in another product; forwards the core's feature gates. Installs no backend transport itself. | -| `crates/openhuman-tinyhumans/` | The TinyHumans layer above embed: `SdkBackendTransport` (the only crate that depends on the vendored `tinyhumans-sdk`), `install()` for hosts that boot the core themselves, a `RuntimeBuilder` that boots an embed runtime connected, the hosted RPC proxies (`hosted/`, all on the SDK's typed clients through one `hosted::client::HostedClient`: billing, team and usage, referral, announcements, webhook tunnels, managed Telegram/Discord linking, and backend-brokered OAuth), registered into the core's controller registry as an extension, some sharing the core's `auth` / `channels` / `webhooks` namespaces), and the host-side login/session owner (`session/`). A user without a TinyHumans account gets the core's `BACKEND_UNAVAILABLE:` sentinel from these without a request. | -| `crates/openhuman-cli/` | The `openhuman-core` binary (installs the tinyhumans transport, then `run_core_from_args`), the developer/benchmark bins, and every root `tests/*.rs` / `examples/*.rs` target. | -| `crates/openhuman-tui/` | Standalone ratatui terminal frontend. Boots the core in-process via `CoreBuilder` (`DomainSet::full()`, `ServiceSet::none()`), no HTTP. | +| `crates/openhuman-embed/` | Typed library facade (`Runtime`/`RuntimeBuilder` → `Agent`, host presets, `embed::process`, and the `config` / `artifacts` / `chat_surface` / `modules` facades) for embedding the core in another product; forwards the core's feature gates. Installs no backend transport itself. | +| `crates/openhuman-tinyhumans/` | The TinyHumans layer above embed: `SdkBackendTransport` (the only crate that depends on the vendored `tinyhumans-sdk`), `install()` for hosts that boot the core themselves, a `RuntimeBuilder` that boots an embed runtime connected, the hosted RPC proxies (`hosted/`, all on the SDK's typed clients through one `hosted::client::HostedClient`: billing, team and usage, referral, announcements, webhook tunnels, managed Telegram/Discord linking, and backend-brokered OAuth; registered into the core's controller registry as an extension, some sharing the core's `auth` / `channels` / `webhooks` namespaces), and the host-side login/session owner (`session/`). A user without a TinyHumans account gets the core's `BACKEND_UNAVAILABLE:` sentinel from these without a request. | +| `crates/openhuman-cli/` | The `openhuman-core` binary (`openhuman_rpc::host::cli`), the ops bins, and every root `tests/*.rs` / `examples/*.rs` target (which reach the core through dev-dependencies). The benchmark bins live in [openhuman-benchmarks](https://github.com/tinyhumansai/openhuman-benchmarks). | +| `crates/openhuman-tui/` | Standalone ratatui terminal frontend. Boots the core in-process via `openhuman_rpc::host::tui` (every domain, no background services), no HTTP. | | `Cargo.toml` (root) | Virtual workspace for `openhuman-core`, `openhuman-embed`, `openhuman-rpc`, `openhuman-tinyhumans`, `openhuman-cli`, and `openhuman-tui` (`cargo build -p openhuman-cli --bin openhuman-core` builds the standalone CLI/server); `vendor/`, `worktrees/`, `crates/openhuman-app`, `app/src-tauri-mobile`, and `packages/tauri-plugin-ptt` are excluded. Holds the `[patch]` tables for vendored crates. There is no sidecar: the desktop bundle links the core in-process. | | `crates/openhuman-core/src/skills/` | Skill metadata and run orchestration (`ops_create`, `ops_discover`, `ops_install`, `ops_parse`, `catalog/`, `registry`, `runtime/`, `schemas/`, `types`, `bundled/`, `webhooks/`). Skills contribute metadata and tool descriptors that are injected into agent prompts. Tool execution flows through native Rust handlers and Node-backed helpers via `runtime::node` (Cargo feature `runtime-node`). | | `gitbooks/` | This book (public product and contributor documentation). | | `docs/` | Internal maintainer documentation (test-coverage matrix, release smoke checklist, library benchmarking notes). | | `vendor/` | Recursive git submodules for the `tiny*` crate family (`tinyagents`, `tinyflows`, `tinychannels`, `tinyjuice`, `tinymemory`, `tinymcp`, `tinybus`, `tinybox`, `tinycomputer`, `tinyruntime`, `tinydocs`, `tinysearch`, `tinyskills`, `tinyvoice`, `tinywallet`, `tinyhosts`, `tinyconnectors`, `tinyhumans-sdk`) plus `motosan-ai-oauth`. | -The desktop app's webview loads the UI from `app/`. RPC, agents and skills run in the `openhuman_core` core, hosted in-process as a tokio task by the Tauri shell (`crates/openhuman-app/src/core_process.rs`, `run_server_embedded_with_ready`) and reachable over loopback HTTP. The renderer's `coreRpcClient` `fetch()`es `http://127.0.0.1:/rpc` directly. The `relay_http_rpc` Tauri command (backed by `openhuman_rpc::post_json_rpc`) is only the fallback for non-loopback plain-`http://` runtimes that the webview would block as mixed content. The standalone `openhuman-core serve` binary is the CLI/debug path. +The desktop app's webview loads the UI from `app/`. RPC, agents and skills run in the `openhuman_core` core, hosted in-process as a tokio task by the Tauri shell (`crates/openhuman-app/src/core_process.rs`, `openhuman_rpc::host::desktop`) and reachable over loopback HTTP. The renderer's `coreRpcClient` `fetch()`es `http://127.0.0.1:/rpc` directly. The `relay_http_rpc` Tauri command (backed by `openhuman_rpc::post_json_rpc`) is only the fallback for non-loopback plain-`http://` runtimes that the webview would block as mixed content. The standalone `openhuman-core serve` binary is the CLI/debug path. + +### Crate layering + +The Rust crates form a strict chain. Each crate's normal dependencies name only +the layer directly below it, and the three hosts name `openhuman-rpc` alone: + +```text + openhuman-app openhuman-cli openhuman-tui + (desktop shell) (openhuman-core) (terminal UI) + \ | / + +-------------------+-------------------+ + | + v + openhuman-rpc host::{desktop, cli, tui}, + | JSON-RPC server + client, + | session store + v + openhuman-tinyhumans SDK transport, hosted + | proxies, session owner + v + openhuman-embed Runtime/RuntimeBuilder, + | process helpers, facades + v + openhuman-core domains, registry, ports +``` + +- **Hosts boot through `openhuman_rpc::host`.** `host::desktop` is the shell's + embedded server, `host::cli` the `openhuman-core` binary (and the shell's + `core` / `mcp` subcommands), `host::tui` the terminal UI's runtime. Each one + connects the TinyHumans layer and builds an embed `Runtime`; there is one per + process. +- **Hosts reach the core only through the curated facade** rpc re-exports + (`openhuman_rpc::embed`, `openhuman_rpc::tinyhumans`). The layers above embed + use embed's doc-hidden `__host` list for the server and transport internals; + hosts never do. +- **Enforced in CI.** `scripts/ci/check-crate-chain.mjs` (part of + `pnpm rust:layout`) fails on any other normal-dependency edge and on a host + `src/` naming `__host`, `core_host` or `openhuman_core::`. + `scripts/ci/check-feature-forwarding.mjs` checks that every gate travels the + same chain. Dev-dependencies are exempt, so the root tests still reach into + the core. --- diff --git a/gitbooks/developing/architecture/tauri-shell.md b/gitbooks/developing/architecture/tauri-shell.md index a949a413a52..27f8d4045c8 100644 --- a/gitbooks/developing/architecture/tauri-shell.md +++ b/gitbooks/developing/architecture/tauri-shell.md @@ -9,6 +9,44 @@ icon: desktop `crates/openhuman-app/` is the desktop host for OpenHuman. It provides the Tauri v2 webview, IPC commands and window management, and bridges to the embedded `openhuman-core` Rust runtime over core JSON-RPC. It does not duplicate the domain stack. That lives in `crates/openhuman-core` (library `openhuman_core`; the `openhuman-core` binary is `crates/openhuman-cli/src/main.rs`). +### Where the shell sits + +The shell is one of three hosts on top of a strict crate chain, and +`openhuman-rpc` is the only OpenHuman crate in its `Cargo.toml`: + +```text + openhuman-app (this crate) + | [dependencies] openhuman-rpc (http-client, server, jev + the product gates) + v + openhuman-rpc ----------> host::desktop / host::cli, server, client + | re-exports `embed` and `tinyhumans` + v + openhuman-tinyhumans ---> SDK transport, session owner (`session/` uses it) + | + v + openhuman-embed --------> Runtime, process helpers (tokio runtime, logging, + | Sentry options), config/artifacts/modules facades + v + openhuman-core +``` + +| Shell code | Goes through | +| --- | --- | +| Embedded server (`core_process.rs`) | `openhuman_rpc::host::desktop(DesktopOptions { port, rpc_token, .. }, shutdown, ready_tx)` | +| `OpenHuman core …` / `mcp` subcommands (`lib.rs::run_core_from_args`) | `openhuman_rpc::host::cli(args)` | +| Tauri async runtime sizing (`lib.rs::run`) | `openhuman_rpc::embed::process::tokio_runtime()` | +| Sentry client (`lib.rs::run`) | `openhuman_rpc::embed::process::sentry::client_options` (the shared `before_send` chain; fallback user id from `session::peek_user_id`) | +| File logging (`file_logging.rs`), data reset log handle | `openhuman_rpc::embed::process::{init_for_embedded, log_directory, shutdown_file_guard}` | +| Config / workspace paths, artifacts, bundled modules | `openhuman_rpc::embed::{config, artifacts, modules}` | +| Session owner (`session/`) | `openhuman_rpc::tinyhumans` | +| Compile-time asserts | `openhuman_rpc::embed::{VOICE_COMPILED_IN, HTTP_SERVER_COMPILED_IN}` | + +`scripts/ci/check-crate-chain.mjs` fails if the shell's manifest names +another OpenHuman crate or its `src/` names `__host`, `core_host` or +`openhuman_core::`. One embed runtime exists per process: when the shell +restarts its embedded server, `CoreProcessHandle` waits for the old task (and +the runtime it owns) to drop before it spawns the next one. + ## Responsibilities 1. Web UI. Load the Vite build from `app/dist` (or dev server on port 1420). @@ -83,7 +121,7 @@ React (fetch) The renderer talks to the local core directly over HTTP: `app/src/services/coreRpcClient.ts` invokes `core_rpc_url` / `core_rpc_token` once, then issues plain `fetch()` calls. The `relay_http_rpc` Tauri command is a host-side fallback used only when the RPC URL is not a trustworthy origin for the secure `tauri://localhost` webview (for example a self-hosted runtime on a LAN IP, blocked as mixed content). In that case the Rust host delegates to `openhuman_rpc::post_json_rpc` from the shared `crates/openhuman-rpc` crate (feature `http-client`): 30 s timeout, redirects disabled when a bearer is present, status + body mirrored back verbatim as `HttpRpcResponse`. The shell adds only the gateway transport guard (`validate_remote_transport`, feature `gateways`) before delegating. -`CoreProcessHandle` in `core_process.rs` owns the embedded server task (started via `openhuman_rpc::server::run_server_embedded_with_ready` with a per-launch random bearer token) and handles stale-listener/port-conflict recovery. +`CoreProcessHandle` in `core_process.rs` owns the embedded server task (started via `openhuman_rpc::host::desktop` with a per-launch random bearer token) and handles stale-listener/port-conflict recovery. ### Window and tray behavior @@ -159,8 +197,8 @@ install's data, and *this* machine's audio, so routing them to a remote gateway wrong rather than incomplete. Gated by the shell-local `gateways` Cargo feature (default on). That gate is unrelated to -the feature-forwarding rules in `AGENTS.md`, which govern which `openhuman_core` gates the -shell forwards; nothing here belongs in `scripts/ci/product-features.txt`. +the feature-forwarding rules in `AGENTS.md`, which govern which core gates the shell +forwards on its `openhuman-rpc` dependency; nothing here belongs in `scripts/ci/product-features.txt`. Frontend: `app/src/services/gatewayService.ts`, surfaced in Settings → Core connection (`components/settings/panels/core/GatewaySection.tsx`). @@ -301,7 +339,7 @@ The Tauri crate does not embed a duplicate Socket.io server or Telegram client; ### `CoreProcessHandle` (`core_process.rs`) -- Runs the core's HTTP/JSON-RPC server as a tokio task inside the Tauri host via `openhuman_rpc::server::run_server_embedded_with_ready`: no sidecar binary. +- Runs the core's HTTP/JSON-RPC server as a tokio task inside the Tauri host via `openhuman_rpc::host::desktop`: no sidecar binary. The runtime the task builds is dropped when the task ends; `shutdown` and the startup-timeout abort wait for that before a respawn, because a process holds one embed runtime at a time. - Generates a per-launch 256-bit hex bearer token (`generate_rpc_token`) and hands it to the embedded server; the renderer reads it via the `core_rpc_token` command. - Stale-listener policy: if the core port is already occupied, probes whether the listener is an old OpenHuman core (terminate + respawn) or something foreign (surface the conflict). `OPENHUMAN_CORE_REUSE_EXISTING=1` opts back into attach-to-existing for debugging. - Managed as Tauri state in `lib.rs` (`app.manage(core_handle)`). diff --git a/package.json b/package.json index 97df9c64743..a50e9cc5c99 100644 --- a/package.json +++ b/package.json @@ -56,7 +56,7 @@ "test:install-ps1": "pwsh -NoProfile -File scripts/tests/OpenHumanWindowsInstall.Tests.ps1", "rust:check": "pnpm --filter openhuman-app rust:check", "rust:clippy": "cargo clippy -p openhuman -p openhuman-cli -p openhuman-tinyhumans -- -D warnings && pnpm --filter openhuman-app rust:clippy", - "rust:layout": "node scripts/ci/check-openhuman-rust-layout.mjs", + "rust:layout": "node scripts/ci/check-openhuman-rust-layout.mjs && node scripts/ci/check-crate-chain.mjs", "rust:ignored-tests": "node scripts/ci/check-ignored-tests.mjs", "dep:audit": "bash scripts/dep-audit/run.sh", "agent:runtime-boundary": "node scripts/ci/check-agent-runtime-boundary.mjs", diff --git a/scripts/__tests__/check-crate-chain.test.mjs b/scripts/__tests__/check-crate-chain.test.mjs new file mode 100644 index 00000000000..a0916a48468 --- /dev/null +++ b/scripts/__tests__/check-crate-chain.test.mjs @@ -0,0 +1,185 @@ +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { test } from 'node:test'; +import { fileURLToPath } from 'node:url'; + +import { + CHAIN, + checkManifestEdges, + checkRepository, + findForbiddenPaths, + formatReport, + parseNormalDependencies, + parsePackageName, + parseWorkspaceDependencies, + parseWorkspaceMembers, +} from '../ci/check-crate-chain.mjs'; + +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../..'); +const CHECKER = resolve(REPO_ROOT, 'scripts/ci/check-crate-chain.mjs'); + +// ── parsing ──────────────────────────────────────────────────────────────── + +test('reads the package name and workspace members', () => { + assert.equal(parsePackageName('[package]\nname = "openhuman-tui" # the TUI\n'), 'openhuman-tui'); + assert.deepEqual( + parseWorkspaceMembers('[workspace]\nmembers = [\n "crates/a",\n # "crates/old",\n "crates/b",\n]\n'), + ['crates/a', 'crates/b'] + ); +}); + +test('normal dependencies include target tables and sub-tables, not dev/build ones', () => { + const deps = parseNormalDependencies(` +[package] +name = "openhuman-cli" + +[dependencies] +openhuman-rpc = { workspace = true, features = ["server"] } +core-alias = { path = "../openhuman-core", package = "openhuman", features = [ + "voice", +] } +serde = "1" + +[target.'cfg(unix)'.dependencies] +openhuman-embed = { path = "../openhuman-embed" } + +[dependencies.openhuman-tinyhumans] +path = "../openhuman-tinyhumans" + +[dev-dependencies] +openhuman-core = { path = "../openhuman-core", package = "openhuman" } + +[build-dependencies] +openhuman-tui = { path = "../openhuman-tui" } +`); + assert.deepEqual( + deps.map(d => d.package), + ['openhuman-rpc', 'openhuman', 'serde', 'openhuman-embed', 'openhuman-tinyhumans'] + ); + assert.equal(deps[0].workspace, true); +}); + +test('a commented-out dependency is not an edge', () => { + const deps = parseNormalDependencies('[dependencies]\n# openhuman-core = { path = "x" }\nlog = "0.4"\n'); + assert.deepEqual(deps.map(d => d.key), ['log']); +}); + +test('workspace dependencies resolve their package rename', () => { + const ws = parseWorkspaceDependencies(` +[workspace.dependencies] +openhuman-core = { path = "crates/openhuman-core", package = "openhuman" } +openhuman-rpc = { path = "crates/openhuman-rpc", default-features = false } +`); + assert.equal(ws.get('openhuman-core'), 'openhuman'); + assert.equal(ws.get('openhuman-rpc'), 'openhuman-rpc'); +}); + +// ── the chain ────────────────────────────────────────────────────────────── + +test('each layer may name only the layer directly below it', () => { + assert.deepEqual(CHAIN['openhuman-embed'], ['openhuman']); + assert.deepEqual(CHAIN['openhuman-cli'], ['openhuman-rpc']); + assert.deepEqual(checkManifestEdges({ crate: 'openhuman-tui', deps: [{ key: 'openhuman-rpc', package: 'openhuman-rpc', workspace: false }] }), []); +}); + +test('a host depending on the core is a violation, even through a workspace alias', () => { + const workspaceDeps = new Map([['openhuman-core', 'openhuman']]); + const violations = checkManifestEdges({ + crate: 'openhuman-tui', + deps: [ + { key: 'openhuman-rpc', package: 'openhuman-rpc', workspace: true }, + { key: 'openhuman-core', package: 'openhuman-core', workspace: true }, + { key: 'openhuman-tinyhumans', package: 'openhuman-tinyhumans', workspace: false }, + ], + workspaceDeps, + }); + assert.deepEqual( + violations.map(v => v.dependency), + ['openhuman', 'openhuman-tinyhumans'] + ); +}); + +test('the core may not depend on any OpenHuman crate', () => { + const [violation] = checkManifestEdges({ + crate: 'openhuman', + deps: [{ key: 'openhuman-rpc', package: 'openhuman-rpc', workspace: false }], + }); + assert.match(violation.reason, /may not depend on any OpenHuman crate/); +}); + +test('an unknown OpenHuman crate fails until it is placed in the chain', () => { + const [violation] = checkManifestEdges({ crate: 'openhuman-new', deps: [] }); + assert.match(violation.reason, /unknown OpenHuman crate/); +}); + +test('host source naming core internals by path is flagged, with file and line', () => { + const hits = findForbiddenPaths( + 'use openhuman_rpc::embed::config;\nlet x = openhuman_core::config::Config::default();\nuse crate::core_host::agent;\nuse openhuman_embed::__host::core;\n', + 'src/lib.rs' + ); + assert.deepEqual( + hits.map(h => `${h.line}:${h.pattern}`), + ['2:openhuman_core::', '3:core_host', '4:__host'] + ); +}); + +test('a test name that merely contains openhuman_core is not a path', () => { + assert.deepEqual(findForbiddenPaths('fn binary_path_result_contains_openhuman_core() {}', 'x.rs'), []); +}); + +// ── the real repository ──────────────────────────────────────────────────── + +test('the checked-in manifests and host sources hold the chain', () => { + const result = checkRepository(REPO_ROOT); + assert.ok(result.checked.length >= 7, `checked only ${result.checked.join(', ')}`); + assert.ok(result.checked.includes('openhuman-app'), 'the desktop shell is outside the workspace and must still be checked'); + assert.deepEqual(result.edgeViolations, [], formatReport(result)); + assert.deepEqual(result.sourceViolations, [], formatReport(result)); +}); + +test('the CLI exits 1 and names the edge when a host reaches past rpc', () => { + const dir = mkdtempSync(join(tmpdir(), 'crate-chain-')); + try { + writeFileSync( + join(dir, 'Cargo.toml'), + '[workspace]\nmembers = ["crates/openhuman-core", "crates/openhuman-tui"]\n' + ); + const crate = (name, toml) => { + mkdirSync(join(dir, 'crates', name, 'src'), { recursive: true }); + writeFileSync(join(dir, 'crates', name, 'Cargo.toml'), toml); + }; + crate('openhuman-core', '[package]\nname = "openhuman"\n'); + crate( + 'openhuman-tui', + '[package]\nname = "openhuman-tui"\n[dependencies]\nopenhuman-core = { path = "../openhuman-core", package = "openhuman" }\n' + ); + crate('openhuman-app', '[package]\nname = "openhuman-app"\n[dependencies]\nopenhuman-rpc = { path = "../openhuman-rpc" }\n'); + crate('openhuman-cli', '[package]\nname = "openhuman-cli"\n'); + writeFileSync(join(dir, 'crates/openhuman-cli/src/main.rs'), 'fn main() { openhuman_core::run(); }\n'); + const result = spawnSync('node', [CHECKER, dir], { encoding: 'utf8' }); + assert.equal(result.status, 1, result.stdout + result.stderr); + assert.match(result.stderr, /openhuman-tui -> openhuman: may only depend on openhuman-rpc/); + assert.match(result.stderr, /crates\/openhuman-cli\/src\/main\.rs:1: openhuman_core::/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('the CLI exits 2 instead of passing when it cannot read the repository', () => { + const dir = mkdtempSync(join(tmpdir(), 'crate-chain-empty-')); + try { + const result = spawnSync('node', [CHECKER, dir], { encoding: 'utf8' }); + assert.equal(result.status, 2, result.stdout + result.stderr); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('the real repository passes through the CLI', () => { + const result = spawnSync('node', [CHECKER], { encoding: 'utf8' }); + assert.equal(result.status, 0, result.stdout + result.stderr); + assert.match(result.stdout, /OK: every crate depends only on the layer below it/); +}); diff --git a/scripts/__tests__/feature-forwarding.test.mjs b/scripts/__tests__/feature-forwarding.test.mjs index 510bca36f48..cc0db31d115 100644 --- a/scripts/__tests__/feature-forwarding.test.mjs +++ b/scripts/__tests__/feature-forwarding.test.mjs @@ -21,6 +21,8 @@ import { parseProductFeatures, parseShellForwardedFeatures, resolveEnabledFeatures, + rpcForwardedGates, + SHELL_RPC_LOCAL_GATES, stripComments, } from '../lib/feature-forwarding.mjs'; @@ -62,7 +64,7 @@ default = ["voice"] test('parses the shell forwarded list across multiple lines', () => { const toml = ` -openhuman_core = { path = "../..", package = "openhuman", default-features = false, features = [ +openhuman-rpc = { path = "../openhuman-rpc", default-features = false, features = [ "media", "voice", ] } @@ -74,7 +76,7 @@ openhuman_core = { path = "../..", package = "openhuman", default-features = fal }); test('detects when the shell inherits defaults instead of forwarding', () => { - const toml = 'openhuman_core = { path = "../..", package = "openhuman" }\n'; + const toml = 'openhuman-rpc = { path = "../openhuman-rpc" }\n'; assert.deepEqual(parseShellForwardedFeatures(toml), { defaultFeatures: true, features: [] }); }); @@ -86,7 +88,7 @@ test('comment stripping does not truncate on a # inside a quoted value', () => { test('a commented-out gate does not count as forwarded', () => { const toml = ` -openhuman_core = { path = "../..", package = "openhuman", default-features = false, features = [ +openhuman-rpc = { path = "../openhuman-rpc", default-features = false, features = [ # "voice", "media", ] } @@ -204,6 +206,16 @@ test('a gate the shell forwards but the product does not claim is flagged', () = assert.deepEqual(result.unexpected, ['web3']); }); +test("the shell's openhuman-rpc local gates are not read as unexpected product gates", () => { + const result = checkProductForwarding({ + productFeatures: PRODUCT, + coreFeatureNames: CORE_GATES, + shell: { defaultFeatures: false, features: ['server', 'http-client', 'jev', 'media', 'voice'] }, + }); + assert.ok(result.ok, JSON.stringify(result)); + assert.deepEqual(Object.keys(SHELL_RPC_LOCAL_GATES).sort(), ['http-client', 'jev', 'server']); +}); + test('a product gate that is not a real core gate is flagged', () => { const result = checkProductForwarding({ productFeatures: ['voice', 'vioce'], @@ -466,7 +478,7 @@ test('reads TOML literal strings, not just basic strings', () => { assert.deepEqual(parseCoreFeatureGraph("[features]\ndefault = ['documents']\ndocuments = ['modules']\n").get('documents'), ['modules']); assert.deepEqual( parseShellForwardedFeatures( - "openhuman_core = { path = \"../..\", default-features = false, features = ['voice'] }\n", + "openhuman-rpc = { path = \"../openhuman-rpc\", default-features = false, features = ['voice'] }\n", ).features, ['voice'], ); @@ -690,65 +702,61 @@ test('a forward to a gate the crate below no longer has is named as drift', () = crate: 'openhuman-cli', features: parseFeatureTable(` [features] -default = ["openhuman-core/default"] -media = ["openhuman-core/media", "openhuman-tinyhumans/media"] -voice = ["openhuman-core/voice", "openhuman-tinyhumans/voice"] -e2e-test-support = ["openhuman-core/e2e-test-support"] +default = ["media"] +media = ["openhuman-rpc/media"] +voice = ["openhuman-rpc/voice"] `), - sources: [ - coreSource(), - // tinyhumans dropped `voice`; the cli still points at it. - { crate: 'openhuman-tinyhumans', gates: ['media'], required: false }, - ], + // rpc dropped `voice`; the cli still points at it. + sources: [{ crate: 'openhuman-rpc', gates: ['media'], required: true }], }); assert.equal(result.ok, false); - assert.deepEqual(result.dangling, [{ gate: 'voice', item: 'openhuman-tinyhumans/voice' }]); + assert.deepEqual(result.dangling, [{ gate: 'voice', item: 'openhuman-rpc/voice' }]); assert.match(formatChainReport(result), /no longer exists/); }); -test('the cli must forward to tinyhumans as well where tinyhumans has the gate', () => { +test('a host forwarding a gate straight to the core is misrouted, not accepted', () => { + // The hosts sit on rpc alone: a forward that skips the chain is the edge + // `check-crate-chain.mjs` forbids, and this check names it too. const result = diffChainForwarding({ crate: 'openhuman-cli', features: parseFeatureTable(` [features] -default = ["openhuman-core/default"] media = ["openhuman-core/media"] -voice = ["openhuman-core/voice"] -e2e-test-support = ["openhuman-core/e2e-test-support"] +voice = ["openhuman-rpc/voice"] `), - sources: [ - coreSource(), - { crate: 'openhuman-tinyhumans', gates: ['media', 'voice'], required: false }, - ], + sources: [{ crate: 'openhuman-rpc', gates: ['media', 'voice'], required: true }], }); assert.equal(result.ok, false); assert.deepEqual( result.misrouted.map(m => m.expected), - ['openhuman-tinyhumans/media', 'openhuman-tinyhumans/voice'] + ['openhuman-rpc/media'] ); }); -test('an optional source never demands a gate the crate below it does not have', () => { - // `e2e-test-support` is core-only: requiring `openhuman-tinyhumans/e2e-test-support` - // would be a forward to a gate that does not exist, i.e. a cargo error. +test("a host need not forward rpc's own local gates", () => { + const rpc = parseFeatureTable(` +[features] +default = ["http-client", "server"] +http-client = ["dep:reqwest"] +server = ["http-server", "session-store"] +session-store = [] +http-server = ["openhuman-tinyhumans/http-server"] +voice = ["openhuman-tinyhumans/voice"] +`); + assert.deepEqual(rpcForwardedGates(rpc), ['http-server', 'voice']); const result = diffChainForwarding({ - crate: 'openhuman-cli', + crate: 'openhuman-tui', features: parseFeatureTable(` [features] -default = ["openhuman-core/default"] -media = ["openhuman-core/media", "openhuman-tinyhumans/media"] -voice = ["openhuman-core/voice", "openhuman-tinyhumans/voice"] -e2e-test-support = ["openhuman-core/e2e-test-support"] +http-server = ["openhuman-rpc/http-server"] +voice = ["openhuman-rpc/voice"] `), - sources: [ - coreSource(), - { crate: 'openhuman-tinyhumans', gates: ['media', 'voice'], required: false }, - ], + sources: [{ crate: 'openhuman-rpc', gates: rpcForwardedGates(rpc), required: true }], }); assert.ok(result.ok, formatChainReport(result)); }); -test('the checked-in embed, tinyhumans, rpc and cli manifests forward the whole chain', () => { +test('the checked-in embed, tinyhumans, rpc, cli and tui manifests forward the whole chain', () => { const read = name => readFileSync(resolve(REPO_ROOT, `crates/${name}/Cargo.toml`), 'utf8'); const core = parseCoreFeatureNames( readFileSync(resolve(REPO_ROOT, 'crates/openhuman-core/Cargo.toml'), 'utf8') @@ -757,6 +765,7 @@ test('the checked-in embed, tinyhumans, rpc and cli manifests forward the whole const tinyhumans = parseFeatureTable(read('openhuman-tinyhumans')); const cli = parseFeatureTable(read('openhuman-cli')); const rpc = parseFeatureTable(read('openhuman-rpc')); + const tui = parseFeatureTable(read('openhuman-tui')); // Guards the guard: empty tables would make every assertion below vacuous. assert.ok(core.length > 0, 'expected to parse at least one core gate'); for (const [name, table] of [ @@ -764,6 +773,7 @@ test('the checked-in embed, tinyhumans, rpc and cli manifests forward the whole ['openhuman-tinyhumans', tinyhumans], ['openhuman-cli', cli], ['openhuman-rpc', rpc], + ['openhuman-tui', tui], ]) { assert.ok(table.size > 0, `expected to parse features from ${name}`); } @@ -787,10 +797,12 @@ test('the checked-in embed, tinyhumans, rpc and cli manifests forward the whole { crate: 'openhuman-cli', features: cli, - sources: [ - { crate: 'openhuman-core', gates: core, required: true }, - { crate: 'openhuman-tinyhumans', gates: gatesOf(tinyhumans), required: false }, - ], + sources: [{ crate: 'openhuman-rpc', gates: rpcForwardedGates(rpc), required: true }], + }, + { + crate: 'openhuman-tui', + features: tui, + sources: [{ crate: 'openhuman-rpc', gates: rpcForwardedGates(rpc), required: true }], }, ]; for (const link of links) { @@ -871,6 +883,7 @@ test('the checker reports the chain and fails when a layer drops a gate', () => resolve(REPO_ROOT, 'crates/openhuman-tinyhumans/Cargo.toml'), resolve(REPO_ROOT, 'crates/openhuman-cli/Cargo.toml'), resolve(REPO_ROOT, 'crates/openhuman-rpc/Cargo.toml'), + resolve(REPO_ROOT, 'crates/openhuman-tui/Cargo.toml'), ], { encoding: 'utf8' } ); diff --git a/scripts/__tests__/self-hosted-lanes.test.mjs b/scripts/__tests__/self-hosted-lanes.test.mjs index 484a39f1260..bb06b64c598 100644 --- a/scripts/__tests__/self-hosted-lanes.test.mjs +++ b/scripts/__tests__/self-hosted-lanes.test.mjs @@ -191,6 +191,7 @@ test("the lane plan retains the shared CI checks", () => { "bash scripts/ci/check-gated-test-allowlist.sh", "bash scripts/ci/orch-ip-gate.sh", "node scripts/ci/check-feature-forwarding.mjs", + "node scripts/ci/check-crate-chain.mjs", "node scripts/ci/check-module-pins.mjs", "node scripts/ci/check-submodule-monotonic.mjs", "node scripts/ci/check-toolchain-image.mjs", @@ -224,6 +225,7 @@ test("untouched areas leave only the always-on gates", () => { assert.deepEqual(on, [ "static:orch-ip-gate", "static:feature-forwarding", + "static:crate-chain", "static:module-pins", "static:submodule-monotonic", ]); diff --git a/scripts/ci/README.md b/scripts/ci/README.md index 40009e736f9..a756855e864 100644 --- a/scripts/ci/README.md +++ b/scripts/ci/README.md @@ -7,7 +7,8 @@ authoritative spec; this is an index so a CI failure log points somewhere. | File | Checks | | --- | --- | | [`check-openhuman-rust-layout.mjs`](check-openhuman-rust-layout.mjs) (`pnpm rust:layout`) | Every file under `crates/openhuman-core/src` stays under a 750-line ceiling, warning (without failing) past 725; oversized files are pinned at their exact size in `LEGACY_LIMIT_ENTRIES`, once each, and a pinned file that shrinks must lower or drop its pin; forbids inline `#[cfg(test)] mod` blocks and `tests.rs`/`test.rs` filenames; and asserts every root `tests/*.rs` / `examples/*.rs` has a matching `[[test]]`/`[[example]]` entry in `crates/openhuman-cli/Cargo.toml`, and that the core manifest declares no target tables. | -| [`check-feature-forwarding.mjs`](check-feature-forwarding.mjs) | The Tauri shell ([`crates/openhuman-app/Cargo.toml`](../../crates/openhuman-app/Cargo.toml), which sets `default-features = false`) forwards exactly the gates in `product-features.txt`, no more, no less, and the library chain that re-declares the core's gates (`openhuman-embed` → `openhuman-tinyhumans` → `openhuman-cli`) forwards every one of them, by target and not just by name. Logic lives in [`scripts/lib/feature-forwarding.mjs`](../lib/feature-forwarding.mjs) (openhuman#4919, openhuman#6364). | +| [`check-feature-forwarding.mjs`](check-feature-forwarding.mjs) | The Tauri shell ([`crates/openhuman-app/Cargo.toml`](../../crates/openhuman-app/Cargo.toml), which sets `default-features = false`) forwards exactly the gates in `product-features.txt`, no more, no less, and the library chain that re-declares the core's gates (`openhuman-embed` → `openhuman-tinyhumans` → `openhuman-rpc` → `openhuman-cli` / `openhuman-tui`) forwards every one of them, by target and not just by name. The shell's list is read from its `openhuman-rpc` dependency. Logic lives in [`scripts/lib/feature-forwarding.mjs`](../lib/feature-forwarding.mjs) (openhuman#4919, openhuman#6364). | +| [`check-crate-chain.mjs`](check-crate-chain.mjs) | The crate chain holds: each OpenHuman crate's normal dependencies name only the layer below it (core: none; embed: core; tinyhumans: embed; rpc: tinyhumans; app/cli/tui: rpc), and host `src/` never names `__host`, `core_host` or `openhuman_core::`. Dev-dependencies are exempt. Runs as part of `pnpm rust:layout`. | | [`product-features.txt`](product-features.txt) / [`product-features.sh`](product-features.sh) | Source of truth for the shipped desktop product's Cargo feature gates, distinct from `[features] default` (the contributor build). The `.sh` prints them comma-separated for `cargo --features "$(scripts/ci/product-features.sh)"`. | | [`check-module-pins.mjs`](check-module-pins.mjs) + [`module-pin-exemptions.json`](module-pin-exemptions.json) | A loadable module's registry pin ([`crates/openhuman-core/src/modules/registry.rs`](../../crates/openhuman-core/src/modules/registry.rs)) and its `vendor/*` submodule pin must describe the same release (openhuman#5727). The exemptions file records known, expected drift, not a way to silence the gate. Logic lives in [`scripts/lib/module-pins.mjs`](../lib/module-pins.mjs). | | [`check-submodule-monotonic.mjs`](check-submodule-monotonic.mjs) | A `vendor/*` submodule pin may never move backwards onto a commit that is an ancestor of what the base branch already has. | @@ -23,7 +24,7 @@ changed-line diff-cover check (>= 80%) in `ci-lite.yml`. ## Running locally -Only `check-openhuman-rust-layout.mjs` is wired to a pnpm script +Only `check-openhuman-rust-layout.mjs` and `check-crate-chain.mjs` are wired to a pnpm script (`pnpm rust:layout`); the rest are invoked directly, e.g. `node scripts/ci/check-feature-forwarding.mjs` or `bash scripts/ci/orch-ip-gate.sh`, and can be run the same way locally. Every diff --git a/scripts/ci/check-crate-chain.mjs b/scripts/ci/check-crate-chain.mjs new file mode 100644 index 00000000000..b49590ffb72 --- /dev/null +++ b/scripts/ci/check-crate-chain.mjs @@ -0,0 +1,293 @@ +#!/usr/bin/env node +// Enforces the crate chain: core -> embed -> tinyhumans -> rpc -> app/cli/tui. +// +// Each OpenHuman crate may name exactly one OpenHuman crate among its NORMAL +// dependencies — the layer directly below it — and the hosts (desktop app, CLI, +// TUI) name `openhuman-rpc` only. Dev-dependencies and build-dependencies are +// exempt: the CLI's root integration tests reach into the core on purpose, and +// none of that reaches a shipped binary. +// +// A second check covers what a manifest cannot show: host source must not +// reach core internals by path. The layers above embed get core internals +// through embed's doc-hidden `__host` list (`core_host` inside rpc); hosts get +// the curated facade (`openhuman_rpc::embed`, `openhuman_rpc::tinyhumans`) and +// nothing else. So `__host`, `core_host` and `openhuman_core::` in +// `crates/openhuman-{app,cli,tui}/src` fail the check. +// +// Usage: check-crate-chain.mjs [repo-root] +// Exit 0 when the chain holds, 1 on a violation, 2 when the inputs could not +// be read or parsed (the check refuses to pass vacuously). + +import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'; +import { dirname, join, relative, resolve } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +import { stripComments } from '../lib/feature-forwarding.mjs'; + +/** + * Package name -> the OpenHuman packages it may name as normal dependencies. + * A package missing from this table is an unknown OpenHuman crate and fails + * the check until someone places it in the chain on purpose. + */ +export const CHAIN = { + openhuman: [], + 'openhuman-embed': ['openhuman'], + 'openhuman-tinyhumans': ['openhuman-embed'], + 'openhuman-rpc': ['openhuman-tinyhumans'], + 'openhuman-app': ['openhuman-rpc'], + 'openhuman-cli': ['openhuman-rpc'], + 'openhuman-tui': ['openhuman-rpc'], +}; + +/** The host crates whose `src/` must not reach core internals by path. */ +export const HOST_SOURCE_DIRS = [ + 'crates/openhuman-app/src', + 'crates/openhuman-cli/src', + 'crates/openhuman-tui/src', +]; + +/** Paths a host must never name (see the header). */ +export const FORBIDDEN_HOST_PATTERNS = [ + { name: '__host', regex: /\b__host\b/ }, + { name: 'core_host', regex: /\bcore_host\b/ }, + { name: 'openhuman_core::', regex: /\bopenhuman_core::/ }, +]; + +/** Whether a package name belongs to this repository's OpenHuman crates. */ +export function isOpenhumanPackage(name) { + return name === 'openhuman' || name.startsWith('openhuman-'); +} + +function tableBody(text, start) { + const rest = text.slice(start); + const next = rest.search(/^[ \t]*\[/m); + return next === -1 ? rest : rest.slice(0, next); +} + +/** `[package] name = "..."`, or null. */ +export function parsePackageName(toml) { + const text = stripComments(toml); + const header = text.match(/^[ \t]*\[package\][ \t]*$/m); + if (!header) return null; + const body = tableBody(text, header.index + header[0].length); + const name = body.match(/^[ \t]*name[ \t]*=[ \t]*"([^"]+)"/m); + return name ? name[1] : null; +} + +/** Root `[workspace] members = [...]`. */ +export function parseWorkspaceMembers(toml) { + const text = stripComments(toml); + const header = text.match(/^[ \t]*\[workspace\][ \t]*$/m); + if (!header) return []; + const body = tableBody(text, header.index + header[0].length); + const at = body.search(/^[ \t]*members[ \t]*=[ \t]*\[/m); + if (at === -1) return []; + const open = body.indexOf('[', at); + const close = body.indexOf(']', open); + return [...body.slice(open + 1, close).matchAll(/"([^"]+)"/g)].map(m => m[1]); +} + +/** One dependency entry's value text (inline table or string), joined across lines. */ +function entryValue(body, valueStart) { + if (body[valueStart] !== '{') { + const end = body.indexOf('\n', valueStart); + return end === -1 ? body.slice(valueStart) : body.slice(valueStart, end); + } + let depth = 0; + for (let i = valueStart; i < body.length; i++) { + if (body[i] === '{') depth++; + else if (body[i] === '}') { + depth--; + if (depth === 0) return body.slice(valueStart, i + 1); + } + } + return body.slice(valueStart); +} + +/** + * Dependencies declared in one dependency table body, as + * `{ key, package, workspace }`: `package` is the `package = "..."` rename when + * given, else the key; `workspace` marks `key.workspace = true` / + * `{ workspace = true }`, whose package the root `[workspace.dependencies]` + * table decides. + */ +export function parseDependencyEntries(body) { + const entries = []; + for (const match of body.matchAll(/^[ \t]*([A-Za-z0-9_-]+)((?:\.workspace)?)[ \t]*=[ \t]*/gm)) { + const key = match[1]; + const value = entryValue(body, match.index + match[0].length); + const pkg = value.match(/package[ \t]*=[ \t]*"([^"]+)"/); + const workspace = match[2] === '.workspace' || /workspace[ \t]*=[ \t]*true/.test(value); + entries.push({ key, package: pkg ? pkg[1] : key, workspace }); + } + return entries; +} + +/** + * Every NORMAL dependency of a manifest: `[dependencies]`, the + * `[target.'…'.dependencies]` tables, and `[dependencies.]` sub-tables. + * Dev- and build-dependencies are skipped. + */ +export function parseNormalDependencies(toml) { + const text = stripComments(toml); + const deps = []; + for (const header of text.matchAll(/^[ \t]*\[([^\]\n]+)\][ \t]*$/gm)) { + const table = header[1].trim(); + const body = tableBody(text, header.index + header[0].length); + if (table === 'dependencies' || /^target\..+\.dependencies$/.test(table)) { + deps.push(...parseDependencyEntries(body)); + continue; + } + const sub = table.match(/^(?:target\..+\.)?dependencies\.([A-Za-z0-9_-]+)$/); + if (sub) { + const pkg = body.match(/^[ \t]*package[ \t]*=[ \t]*"([^"]+)"/m); + const workspace = /^[ \t]*workspace[ \t]*=[ \t]*true/m.test(body); + deps.push({ key: sub[1], package: pkg ? pkg[1] : sub[1], workspace }); + } + } + return deps; +} + +/** Root `[workspace.dependencies]` as key -> package name. */ +export function parseWorkspaceDependencies(toml) { + const text = stripComments(toml); + const header = text.match(/^[ \t]*\[workspace\.dependencies\][ \t]*$/m); + if (!header) return new Map(); + const body = tableBody(text, header.index + header[0].length); + return new Map(parseDependencyEntries(body).map(entry => [entry.key, entry.package])); +} + +/** + * Violations of {@link CHAIN} for one manifest. Each is + * `{ crate, dependency, reason }`. + */ +export function checkManifestEdges({ crate, deps, workspaceDeps = new Map(), chain = CHAIN }) { + const violations = []; + if (!Object.prototype.hasOwnProperty.call(chain, crate)) { + violations.push({ crate, dependency: null, reason: 'unknown OpenHuman crate: add it to CHAIN' }); + return violations; + } + const allowed = new Set(chain[crate]); + for (const dep of deps) { + const pkg = dep.workspace ? (workspaceDeps.get(dep.key) ?? dep.package) : dep.package; + if (!isOpenhumanPackage(pkg)) continue; + if (!allowed.has(pkg)) { + violations.push({ + crate, + dependency: pkg, + reason: allowed.size + ? `may only depend on ${[...allowed].join(', ')} among OpenHuman crates` + : 'may not depend on any OpenHuman crate', + }); + } + } + return violations; +} + +/** `{ file, line, pattern }` for every forbidden path in one source text. */ +export function findForbiddenPaths(text, file, patterns = FORBIDDEN_HOST_PATTERNS) { + const hits = []; + text.split(/\r?\n/).forEach((line, index) => { + for (const { name, regex } of patterns) { + if (regex.test(line)) hits.push({ file, line: index + 1, pattern: name }); + } + }); + return hits; +} + +function rustFiles(dir) { + const out = []; + for (const entry of readdirSync(dir)) { + const path = join(dir, entry); + if (statSync(path).isDirectory()) out.push(...rustFiles(path)); + else if (entry.endsWith('.rs')) out.push(path); + } + return out; +} + +/** Run both checks against a repository checkout. */ +export function checkRepository(root) { + const rootToml = readFileSync(join(root, 'Cargo.toml'), 'utf8'); + const members = parseWorkspaceMembers(rootToml); + if (members.length === 0) { + throw new Error('parsed zero workspace members from the root Cargo.toml'); + } + const workspaceDeps = parseWorkspaceDependencies(rootToml); + // The desktop shell is its own Cargo world, excluded from the workspace. + const manifestDirs = [...members, 'crates/openhuman-app']; + const edgeViolations = []; + const checked = []; + for (const dir of manifestDirs) { + const manifest = join(root, dir, 'Cargo.toml'); + if (!existsSync(manifest)) throw new Error(`missing manifest ${relative(root, manifest)}`); + const toml = readFileSync(manifest, 'utf8'); + const crate = parsePackageName(toml); + if (!crate) throw new Error(`no [package] name in ${relative(root, manifest)}`); + if (!isOpenhumanPackage(crate)) continue; + checked.push(crate); + edgeViolations.push( + ...checkManifestEdges({ crate, deps: parseNormalDependencies(toml), workspaceDeps }) + ); + } + const sourceViolations = []; + for (const dir of HOST_SOURCE_DIRS) { + const abs = join(root, dir); + if (!existsSync(abs)) throw new Error(`missing host source directory ${dir}`); + for (const file of rustFiles(abs)) { + sourceViolations.push( + ...findForbiddenPaths(readFileSync(file, 'utf8'), relative(root, file)) + ); + } + } + return { checked, edgeViolations, sourceViolations }; +} + +export function formatReport({ checked, edgeViolations, sourceViolations }) { + const lines = [`Crate chain: core -> embed -> tinyhumans -> rpc -> app/cli/tui`]; + lines.push(`Checked ${checked.length} manifests: ${checked.join(', ')}`); + if (edgeViolations.length > 0) { + lines.push('', 'Normal-dependency edges that break the chain:'); + for (const v of edgeViolations) { + lines.push(` - ${v.crate} -> ${v.dependency ?? '?'}: ${v.reason}`); + } + lines.push( + '', + 'Reach the layer below through its curated facade instead. Hosts name', + '`openhuman-rpc` only (`openhuman_rpc::embed`, `openhuman_rpc::tinyhumans`);', + 'a test-only need belongs in [dev-dependencies].' + ); + } + if (sourceViolations.length > 0) { + lines.push('', 'Host source reaching core internals by path:'); + for (const v of sourceViolations) lines.push(` - ${v.file}:${v.line}: ${v.pattern}`); + lines.push( + '', + 'Hosts use the curated facade (`openhuman_rpc::embed::…`). `__host` /', + '`core_host` are internal to the library layers; add what the host needs to', + "embed's public facade instead." + ); + } + if (edgeViolations.length === 0 && sourceViolations.length === 0) { + lines.push('OK: every crate depends only on the layer below it.'); + } + return lines.join('\n'); +} + +function main() { + const root = resolve(process.argv[2] ?? resolve(dirname(fileURLToPath(import.meta.url)), '../..')); + let result; + try { + result = checkRepository(root); + } catch (err) { + console.error(`check-crate-chain: could not check ${root}: ${err.message}`); + process.exit(2); + } + const report = formatReport(result); + const ok = result.edgeViolations.length === 0 && result.sourceViolations.length === 0; + (ok ? console.log : console.error)(report); + process.exit(ok ? 0 : 1); +} + +if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) { + main(); +} diff --git a/scripts/ci/check-feature-forwarding.mjs b/scripts/ci/check-feature-forwarding.mjs index d148acf3ba0..80359a90cd3 100644 --- a/scripts/ci/check-feature-forwarding.mjs +++ b/scripts/ci/check-feature-forwarding.mjs @@ -4,7 +4,8 @@ // // See scripts/lib/feature-forwarding.mjs for the three assertions and why they // are shaped this way (#4919). Short version: the shell sets -// `default-features = false` on `openhuman_core`, so every gate the product +// `default-features = false` on `openhuman-rpc` (its only openhuman +// dependency, which forwards each gate down to the core), so every gate the product // needs must be forwarded by hand. When someone forgets, the domain vanishes // from the shipped app with no build error — that is how #4901 (voice, 56 // users, ~93k Sentry events) and #4918 (tokenjuice-treesitter, silent soft @@ -15,14 +16,14 @@ // deliberately smaller. // // It also checks the library chain the core is re-declared by — embed, -// tinyhumans, rpc and cli (#6364). Those three lists were maintained by hand: a gate +// tinyhumans, rpc, then the cli and tui hosts on rpc (#6364). Those lists were maintained by hand: a gate // dropped from the core and left behind is a cargo error nobody reads as drift // (#6360), and a gate ADDED to the core and forgotten is silent, because the // product lanes only ever resolve names against `openhuman-cli`. // // Usage: check-feature-forwarding.mjs [core-manifest] [shell-manifest] [product-features] // [embed-manifest] [tinyhumans-manifest] [cli-manifest] -// [rpc-manifest] +// [rpc-manifest] [tui-manifest] import { readFileSync } from 'node:fs'; import { dirname, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -42,6 +43,7 @@ import { parseFeatureTable, parseProductFeatures, parseShellForwardedFeatures, + rpcForwardedGates, } from '../lib/feature-forwarding.mjs'; const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../..'); @@ -50,11 +52,11 @@ function usage() { return ( 'Usage: check-feature-forwarding.mjs [core-manifest] [shell-manifest] [product-features]\n' + ' [embed-manifest] [tinyhumans-manifest] [cli-manifest]\n' + - ' [rpc-manifest]' + ' [rpc-manifest] [tui-manifest]' ); } -const [coreArg, shellArg, productArg, embedArg, tinyhumansArg, cliArg, rpcArg, extra] = +const [coreArg, shellArg, productArg, embedArg, tinyhumansArg, cliArg, rpcArg, tuiArg, extra] = process.argv.slice(2); if (coreArg === '--help' || coreArg === '-h') { console.log(usage()); @@ -80,6 +82,7 @@ const tinyhumansPath = tinyhumansArg : resolve(REPO_ROOT, 'crates/openhuman-tinyhumans/Cargo.toml'); const cliPath = cliArg ? resolve(cliArg) : resolve(REPO_ROOT, 'crates/openhuman-cli/Cargo.toml'); const rpcPath = rpcArg ? resolve(rpcArg) : resolve(REPO_ROOT, 'crates/openhuman-rpc/Cargo.toml'); +const tuiPath = tuiArg ? resolve(tuiArg) : resolve(REPO_ROOT, 'crates/openhuman-tui/Cargo.toml'); let coreToml; let shellToml; @@ -88,6 +91,7 @@ let embedToml; let tinyhumansToml; let cliToml; let rpcToml; +let tuiToml; try { coreToml = readFileSync(corePath, 'utf8'); shellToml = readFileSync(shellPath, 'utf8'); @@ -96,6 +100,7 @@ try { tinyhumansToml = readFileSync(tinyhumansPath, 'utf8'); cliToml = readFileSync(cliPath, 'utf8'); rpcToml = readFileSync(rpcPath, 'utf8'); + tuiToml = readFileSync(tuiPath, 'utf8'); } catch (err) { console.error(`Could not read inputs: ${err.message}`); process.exit(2); @@ -148,6 +153,7 @@ const embedFeatures = parseFeatureTable(embedToml); const tinyhumansFeatures = parseFeatureTable(tinyhumansToml); const cliFeatures = parseFeatureTable(cliToml); const rpcFeatures = parseFeatureTable(rpcToml); +const tuiFeatures = parseFeatureTable(tuiToml); // Guard the guard, same as above: a parser that found nothing would turn every // chain assertion into a rubber stamp. @@ -156,6 +162,7 @@ for (const [path, table] of [ [tinyhumansPath, tinyhumansFeatures], [cliPath, cliFeatures], [rpcPath, rpcFeatures], + [tuiPath, tuiFeatures], ]) { if (table.size === 0) { console.error( @@ -168,6 +175,8 @@ for (const [path, table] of [ const embedGates = [...embedFeatures.keys()].filter(name => name !== 'default'); const tinyhumansGates = [...tinyhumansFeatures.keys()].filter(name => name !== 'default'); +// What a host must forward from rpc: its gates minus rpc's own local ones. +const rpcHostGates = rpcForwardedGates(rpcFeatures); const chain = [ { @@ -188,14 +197,15 @@ const chain = [ sources: [{ crate: 'openhuman-tinyhumans', gates: tinyhumansGates, required: true }], }, { + // The hosts sit on rpc alone (no core/tinyhumans edge any more). crate: 'openhuman-cli', features: cliFeatures, - sources: [ - { crate: 'openhuman-core', gates: coreFeatureNames, required: true }, - // Optional: the cli forwards to both parents, but only for the gates - // tinyhumans actually has — `e2e-test-support` is core-only. - { crate: 'openhuman-tinyhumans', gates: tinyhumansGates, required: false }, - ], + sources: [{ crate: 'openhuman-rpc', gates: rpcHostGates, required: true }], + }, + { + crate: 'openhuman-tui', + features: tuiFeatures, + sources: [{ crate: 'openhuman-rpc', gates: rpcHostGates, required: true }], }, ].map(link => diffChainForwarding({ @@ -206,7 +216,7 @@ const chain = [ ); console.log(''); -console.log('Library chain (core -> embed -> tinyhumans -> rpc; core/tinyhumans -> cli):'); +console.log('Library chain (core -> embed -> tinyhumans -> rpc -> cli/tui; app on its rpc dependency):'); for (const result of chain) { console.log( formatChainReport(result, { diff --git a/scripts/ci/product-features.txt b/scripts/ci/product-features.txt index 257911d6a61..b21a1d7fe15 100644 --- a/scripts/ci/product-features.txt +++ b/scripts/ci/product-features.txt @@ -11,7 +11,8 @@ # scripts/ci/product-features.sh, so they keep compiling and testing the # code the product actually ships even though `default` no longer does. # -# Why the split (#4901, #4919): the shell declares `openhuman_core` with +# Why the split (#4901, #4919): the shell declares its core dependency (now +# `openhuman-rpc`, which forwards each gate down the chain to the core) with # `default-features = false`, so it never inherited `default` anyway. Before # this file, the guard worked by diffing the shell's list against `default` — # which meant SHRINKING `default` made the guard pass vacuously and silently diff --git a/scripts/ci/self-hosted/lanes-plan.mjs b/scripts/ci/self-hosted/lanes-plan.mjs index 027a12e711e..22eb4172a5b 100644 --- a/scripts/ci/self-hosted/lanes-plan.mjs +++ b/scripts/ci/self-hosted/lanes-plan.mjs @@ -180,6 +180,13 @@ export function buildPlan({ profile, areas, env = {}, isPullRequest = true }) { when: true, run: "node scripts/ci/check-feature-forwarding.mjs", }, + { + // core -> embed -> tinyhumans -> rpc -> app/cli/tui; cheap, and a + // manifest edit anywhere can break it, so always on. + name: "crate-chain", + when: true, + run: "node scripts/ci/check-crate-chain.mjs", + }, { name: "module-pins", when: true, diff --git a/scripts/lib/feature-forwarding.mjs b/scripts/lib/feature-forwarding.mjs index bf54b770d3d..f5f05a51bde 100644 --- a/scripts/lib/feature-forwarding.mjs +++ b/scripts/lib/feature-forwarding.mjs @@ -17,7 +17,9 @@ // reached zero gates — silently re-arming the exact failure described below. // That is why assertion 1 exists and why it compares an explicit list. // -// Why this exists (#4919): the shell declares `openhuman_core` with +// Why this exists (#4919): the shell declares its one openhuman dependency +// (`openhuman_core` then; `openhuman-rpc` now, which forwards each gate down +// the chain rpc -> tinyhumans -> embed -> core) with // `default-features = false`, so it does NOT inherit the core's `default` list. // Every default-ON gate must be forwarded by hand, and nothing enforced that. // When the two drift, the domain is compiled out of the shipped desktop app and @@ -52,6 +54,20 @@ export const INTENTIONALLY_NOT_FORWARDED = { // 'some-gate': 'Reason it must not ship in the desktop build.', }; +/** + * Gates the shell turns on on its `openhuman-rpc` dependency that are NOT + * core gates, mapped to why. They are rpc's (and, for `jev`, tinyhumans') + * own, so they never appear in `scripts/ci/product-features.txt`, and the + * product assertion must not read them as "unexpected". + */ +export const SHELL_RPC_LOCAL_GATES = { + 'http-client': + "openhuman-rpc's authenticated JSON-RPC client; the shell relays non-loopback runtimes through it.", + server: + "openhuman-rpc's JSON-RPC server and `host::desktop`, which the shell boots its embedded core with.", + jev: 'The Jev-backed `tool_search` ranker (openhuman-tinyhumans), shipped by the product.', +}; + /** * Strip TOML `#` comments while respecting quoted strings, so a `#` inside a * value (or an issue number in a comment) can't truncate a real line. @@ -132,13 +148,14 @@ export function parseCoreDefaultFeatures(coreToml) { } /** - * What the shell forwards on its `openhuman_core` dependency. + * What the shell forwards on its `openhuman-rpc` dependency — the only + * openhuman crate it names (`scripts/ci/check-crate-chain.mjs`). * * Returns `{ defaultFeatures, features }`. `defaultFeatures: true` means the * shell inherits the core's defaults and forwarding is moot — there is nothing * to drift. */ -export function parseShellForwardedFeatures(shellToml, depName = 'openhuman_core') { +export function parseShellForwardedFeatures(shellToml, depName = 'openhuman-rpc') { const text = stripComments(shellToml); // `[ \t]` not `\s`, for the same newline-matching reason as above. const declAt = text.search(new RegExp(`^[ \\t]*${depName}[ \\t]*=[ \\t]*\\{`, 'm')); @@ -260,7 +277,12 @@ export function parseProductFeatures(text) { * dropping a gate from the shell fails on `missing`, and adding one the * product never agreed to fails on `unexpected`. */ -export function checkProductForwarding({ productFeatures, coreFeatureNames, shell }) { +export function checkProductForwarding({ + productFeatures, + coreFeatureNames, + shell, + localGates = SHELL_RPC_LOCAL_GATES, +}) { const empty = { missing: [], unexpected: [], unknown: [] }; if (shell === null) return { ok: false, reason: 'dependency-not-found', ...empty }; if (shell.defaultFeatures) { @@ -276,7 +298,11 @@ export function checkProductForwarding({ productFeatures, coreFeatureNames, shel const missing = productFeatures.filter(gate => !forwarded.has(gate)); // Forwarded by the shell but not in the product set → the product grew a // gate without anyone editing the file that says what the product is. - const unexpected = shell.features.filter(gate => !product.has(gate)); + // `openhuman-rpc`'s own gates (server, client, Jev ranker) are not core + // gates and never belong in the product file; they are exempt by name. + const unexpected = shell.features.filter( + gate => !product.has(gate) && !Object.prototype.hasOwnProperty.call(localGates, gate) + ); // Named in the product set but not declared by the core → typo, or a gate // renamed/deleted without updating this list. const unknown = productFeatures.filter(gate => !known.has(gate)); @@ -291,11 +317,11 @@ export function checkProductForwarding({ productFeatures, coreFeatureNames, shel export function formatProductReport(result, { productFeatures, shell }) { if (result.reason === 'dependency-not-found') { - return 'FAIL: could not find the `openhuman_core` dependency in the shell manifest.\nThe guard cannot verify forwarding — fix the parser or the manifest.'; + return 'FAIL: could not find the `openhuman-rpc` dependency in the shell manifest.\nThe guard cannot verify forwarding — fix the parser or the manifest.'; } if (result.reason === 'shell-inherits-defaults') { return [ - 'FAIL: the shell no longer sets `default-features = false` on `openhuman_core`.', + 'FAIL: the shell no longer sets `default-features = false` on `openhuman-rpc`.', 'It would inherit `[features] default`, which is the CONTRIBUTOR set and is', 'deliberately smaller than the product — voice, web3, documents, meet, contacts', 'and crash-reporting would vanish from the shipped app.', @@ -316,7 +342,7 @@ export function formatProductReport(result, { productFeatures, shell }) { lines.push( '', 'Each of these is compiled OUT of the shipped desktop app, silently.', - 'Add it to the `openhuman_core` features list in crates/openhuman-app/Cargo.toml.', + 'Add it to the `openhuman-rpc` features list in crates/openhuman-app/Cargo.toml.', 'See #4901 (voice, 56 users) and #4918 (tokenjuice-treesitter).' ); } @@ -366,7 +392,7 @@ export function diffForwarding({ coreDefaults, shell, allowlist = {} }) { export function formatReport(result, { coreDefaults, shell, allowlist = {} }) { if (result.reason === 'dependency-not-found') { - return 'FAIL: could not find the `openhuman_core` dependency in the shell manifest.\nThe guard cannot verify forwarding — fix the parser or the manifest.'; + return 'FAIL: could not find the `openhuman-rpc` dependency in the shell manifest.\nThe guard cannot verify forwarding — fix the parser or the manifest.'; } if (result.reason === 'inherits-defaults') { return 'OK: the shell inherits the core default features (no `default-features = false`), so no forwarding is required.'; @@ -388,7 +414,7 @@ export function formatReport(result, { coreDefaults, shell, allowlist = {} }) { lines.push( '', 'Each of these is compiled OUT of the shipped desktop app, silently.', - 'Fix by adding the gate to the `openhuman_core` features list in', + 'Fix by adding the gate to the `openhuman-rpc` features list in', 'crates/openhuman-app/Cargo.toml — or, if the exclusion is deliberate, add it to', 'INTENTIONALLY_NOT_FORWARDED in scripts/ci/check-feature-forwarding.mjs', 'with a reason. See #4901 (voice) and #4918 (tokenjuice-treesitter).' @@ -399,15 +425,19 @@ export function formatReport(result, { coreDefaults, shell, allowlist = {} }) { return lines.join('\n'); } -// ── the library chain: core → embed → tinyhumans → rpc, and cli (#6364) ─── +// ── the library chain: core → embed → tinyhumans → rpc → cli/tui (#6364) ── // // The shell is not the only manifest that re-declares the core's gates. The -// library layers forward them 1:1 three more times: +// library layers and the other hosts forward them 1:1: // // openhuman-embed = ["openhuman-core/"] // openhuman-tinyhumans = ["openhuman-embed/"] // openhuman-rpc = ["openhuman-tinyhumans/"] -// openhuman-cli = ["openhuman-core/", "openhuman-tinyhumans/"] +// openhuman-cli = ["openhuman-rpc/"] +// openhuman-tui = ["openhuman-rpc/"] +// +// (The desktop shell forwards on its `openhuman-rpc` dependency line instead +// of a features table; `checkProductForwarding` covers it.) // // Nothing enforced those three lists, and they fail in both directions: // @@ -433,14 +463,8 @@ export function formatReport(result, { coreDefaults, shell, allowlist = {} }) { */ export const CHAIN_GATES_NOT_FORWARDED = { 'openhuman-embed': { - 'e2e-test-support': - 'Exposes the destructive `openhuman.test_reset` RPC for the E2E build only. An embedder must never be able to turn a data wipe on.', 'rss-bench': - 'Library-side hook for the embedded-RSS benchmark (#5046). Its binaries live in tinyhumansai/openhuman-benchmarks (`profile/`), which enables the core gate directly.', - }, - 'openhuman-cli': { - 'rss-bench': - 'The benchmark binaries that used it moved to tinyhumansai/openhuman-benchmarks (`profile/`), which enables the core gate directly.', + 'Library-side hook for the embedded-RSS benchmark (#5046). Its binaries live in tinyhumansai/openhuman-benchmarks (`profile/`, #6944), which enables the core gate directly; no host in this repository turns it on.', }, }; @@ -486,11 +510,10 @@ export function parseFeatureTable(toml) { * * `{ crate, gates, required: true }` — every gate must be declared here and * must forward to `/`. That is core → embed, embed → - * tinyhumans, tinyhumans → rpc, core → cli. + * tinyhumans, tinyhumans → rpc, rpc → cli and rpc → tui. * `{ crate, gates, required: false }` — only gates this crate ALREADY - * declares have to carry the forward. That is tinyhumans → cli: the cli - * forwards to both parents, but only for the gates tinyhumans has, and a - * core-only gate like `e2e-test-support` must not be demanded of it. + * declares have to carry the forward: for a crate that forwards to a + * second parent only where that parent has the gate. * * A forward is checked by TARGET, not just by name: a gate declared as * `voice = ["openhuman-core/web3"]` is as broken as a missing one and looks @@ -640,3 +663,16 @@ export function formatChainReport(result, { notForwarded = {} } = {}) { if (result.ok) lines.push(` OK: forwards every gate of the crates below it.`); return lines.join('\n'); } + +/** + * The gates a host must forward from `openhuman-rpc`: rpc's gates minus its + * own local ones (`http-client`, `server`, `session-store`). A host turns those + * on on its dependency line when it needs them; they are not product gates and + * there is nothing to forward. + */ +export function rpcForwardedGates(rpcFeatures) { + const local = CHAIN_LOCAL_GATES['openhuman-rpc'] ?? {}; + return [...rpcFeatures.keys()].filter( + name => name !== 'default' && !Object.prototype.hasOwnProperty.call(local, name) + ); +} diff --git a/tests/README.md b/tests/README.md index 20010120550..e8309e1869e 100644 --- a/tests/README.md +++ b/tests/README.md @@ -22,8 +22,10 @@ member, don't add one. ## Registering a new target (read this before adding a file) These targets belong to **[`crates/openhuman-cli`](../crates/openhuman-cli/README.md)**: the crate that owns the -`openhuman-core` binary and depends on `openhuman-tinyhumans` for the backend -transport the core library does not carry. Its manifest sets +`openhuman-core` binary. The binary depends on `openhuman-rpc` alone; the +tests reach the core, `openhuman-embed` and `openhuman-tinyhumans` (the +backend transport the core library does not carry) through that crate's +**dev-dependencies**, so none of them becomes an edge of the shipped binary. Its manifest sets `autotests = false` and `autoexamples = false`, because Cargo's autodiscovery only scans beside the manifest and the tests live at the repo root instead. That means `cargo test` silently runs **nothing** for a new `tests/.rs` diff --git a/tests/fixtures/tool_search/intents.jsonl b/tests/fixtures/tool_search/intents.jsonl deleted file mode 100644 index 6a0073d4500..00000000000 --- a/tests/fixtures/tool_search/intents.jsonl +++ /dev/null @@ -1,162 +0,0 @@ -# One JSON object per line: {intent, expected (tool name or "none"), family?}. -# Hand-written paraphrases over the real orchestrator registry and the recorded Composio catalogues; read by `tool-search-bench`. -{"intent": "ping alex by email that the deck is ready", "expected": "GMAIL_SEND_EMAIL"} -{"intent": "shoot a quick mail to the team about tomorrow's standup being cancelled", "expected": "GMAIL_SEND_EMAIL"} -{"intent": "what's new in my inbox this morning?", "expected": "GMAIL_FETCH_EMAILS"} -{"intent": "did anyone from Acme write to me this week?", "expected": "GMAIL_FETCH_EMAILS"} -{"intent": "draft a reply to Sarah's proposal but don't send it yet", "expected": "GMAIL_CREATE_EMAIL_DRAFT"} -{"intent": "answer that thread from the landlord saying yes", "expected": "GMAIL_REPLY_TO_THREAD"} -{"intent": "what labels do I have set up in gmail?", "expected": "GMAIL_LIST_LABELS"} -{"intent": "pull the PDF attached to the invoice email", "expected": "GMAIL_GET_ATTACHMENT"} -{"intent": "bin that spammy newsletter email", "expected": "GMAIL_MOVE_TO_TRASH"} -{"intent": "tag the flight confirmation with my Travel label", "expected": "GMAIL_ADD_LABEL_TO_EMAIL"} -{"intent": "look up Priya's email address from my contacts", "expected": "GMAIL_SEARCH_PEOPLE"} -{"intent": "ping alex on slack that I'm ten minutes late", "expected": "SLACK_SEND_MESSAGE"} -{"intent": "tell #eng in slack the deploy is done", "expected": "SLACK_SEND_MESSAGE"} -{"intent": "drop a \ud83d\udc4d on the last message in #general", "expected": "SLACK_ADD_REACTION_TO_AN_ITEM"} -{"intent": "what did people say in #design yesterday?", "expected": "SLACK_FETCH_CONVERSATION_HISTORY"} -{"intent": "find the slack thread where we discussed the pricing page", "expected": "SLACK_SEARCH_MESSAGES"} -{"intent": "make a new slack channel called launch-week", "expected": "SLACK_CREATE_CHANNEL"} -{"intent": "list all the channels in our workspace", "expected": "SLACK_LIST_ALL_CHANNELS"} -{"intent": "add maria to the #ops channel", "expected": "SLACK_INVITE_USER_TO_CHANNEL"} -{"intent": "schedule a slack message to #team for monday 9am saying happy new week", "expected": "SLACK_SCHEDULE_MESSAGE"} -{"intent": "pin that announcement in the channel", "expected": "SLACK_PIN_ITEM"} -{"intent": "post the roadmap PDF into #product on slack", "expected": "SLACK_UPLOAD_OR_CREATE_A_FILE_IN_SLACK"} -{"intent": "file a bug on the repo about the login page crash", "expected": "GITHUB_CREATE_AN_ISSUE"} -{"intent": "open a github issue: dark mode toggle is broken", "expected": "GITHUB_CREATE_AN_ISSUE"} -{"intent": "open a PR from fix/login into main", "expected": "GITHUB_CREATE_A_PULL_REQUEST"} -{"intent": "leave a comment on issue 42 saying we'll pick it up next sprint", "expected": "GITHUB_CREATE_AN_ISSUE_COMMENT"} -{"intent": "what PRs are open on the backend repo?", "expected": "GITHUB_FIND_PULL_REQUESTS"} -{"intent": "show me the recent commits on main", "expected": "GITHUB_LIST_COMMITS"} -{"intent": "cut a v1.2.0 release on github", "expected": "GITHUB_CREATE_A_RELEASE"} -{"intent": "label issue 17 as bug and p1", "expected": "GITHUB_ADD_LABELS_TO_AN_ISSUE"} -{"intent": "fork the tinytools repo into my account", "expected": "GITHUB_CREATE_A_FORK"} -{"intent": "has PR 88 been merged yet?", "expected": "GITHUB_CHECK_IF_PULL_REQUEST_HAS_BEEN_MERGED"} -{"intent": "get the readme of the openhuman repository", "expected": "GITHUB_GET_A_REPOSITORY_README"} -{"intent": "approve pull request 12 with a review", "expected": "GITHUB_CREATE_A_REVIEW_FOR_A_PULL_REQUEST"} -{"intent": "make a notion page for the offsite agenda", "expected": "NOTION_CREATE_NOTION_PAGE"} -{"intent": "find my notion page about hiring", "expected": "NOTION_SEARCH_NOTION_PAGE"} -{"intent": "append these meeting notes to the notion page", "expected": "NOTION_ADD_PAGE_CONTENT"} -{"intent": "set up a notion database to track candidates", "expected": "NOTION_CREATE_DATABASE"} -{"intent": "which rows in the notion tasks database are overdue?", "expected": "NOTION_QUERY_DATABASE_WITH_FILTER"} -{"intent": "rename the notion page to Q4 planning", "expected": "NOTION_UPDATE_PAGE"} -{"intent": "archive the old roadmap page in notion", "expected": "NOTION_ARCHIVE_NOTION_PAGE"} -{"intent": "leave a comment on the notion spec asking about scope", "expected": "NOTION_CREATE_COMMENT"} -{"intent": "upload the contract to my google drive", "expected": "GOOGLEDRIVE_UPLOAD_FILE"} -{"intent": "where's the budget spreadsheet in my drive?", "expected": "GOOGLEDRIVE_FIND_FILE"} -{"intent": "create a Receipts folder in drive", "expected": "GOOGLEDRIVE_CREATE_FOLDER"} -{"intent": "download the onboarding doc from google drive", "expected": "GOOGLEDRIVE_DOWNLOAD_FILE"} -{"intent": "share the pitch deck in drive with tom@example.com", "expected": "GOOGLEDRIVE_CREATE_PERMISSION"} -{"intent": "move the photos folder into Archive on drive", "expected": "GOOGLEDRIVE_MOVE_FILE"} -{"intent": "make a copy of the template doc in drive", "expected": "GOOGLEDRIVE_COPY_FILE"} -{"intent": "delete the duplicate file from google drive", "expected": "GOOGLEDRIVE_DELETE_FILE"} -{"intent": "read the values in A1:D20 of the sales sheet", "expected": "GOOGLESHEETS_BATCH_GET"} -{"intent": "add a new tab called July to the expenses spreadsheet", "expected": "GOOGLESHEETS_ADD_SHEET"} -{"intent": "append a row with today's numbers to the metrics sheet", "expected": "GOOGLESHEETS_SPREADSHEETS_VALUES_APPEND"} -{"intent": "start a fresh google sheet for the vendor list", "expected": "GOOGLESHEETS_CREATE_GOOGLE_SHEET1"} -{"intent": "wipe the values in the scratch range of the sheet", "expected": "GOOGLESHEETS_CLEAR_VALUES"} -{"intent": "replace every 'TBD' with 'done' in the tracker spreadsheet", "expected": "GOOGLESHEETS_FIND_REPLACE"} -{"intent": "post to r/rust about our new crate", "expected": "REDDIT_CREATE_REDDIT_POST"} -{"intent": "search reddit for threads about the M4 macbook", "expected": "REDDIT_SEARCH_ACROSS_SUBREDDITS"} -{"intent": "what are the comments on that reddit post?", "expected": "REDDIT_RETRIEVE_POST_COMMENTS"} -{"intent": "reply to the top comment on my reddit post", "expected": "REDDIT_POST_REDDIT_COMMENT"} -{"intent": "publish a post on our facebook page about the sale", "expected": "FACEBOOK_CREATE_POST"} -{"intent": "how are our facebook page posts performing?", "expected": "FACEBOOK_GET_PAGE_INSIGHTS"} -{"intent": "which facebook pages do I manage?", "expected": "FACEBOOK_LIST_MANAGED_PAGES"} -{"intent": "show me the comments on my latest instagram post", "expected": "INSTAGRAM_GET_POST_COMMENTS"} -{"intent": "how many followers and reach did my instagram get this week?", "expected": "INSTAGRAM_GET_USER_INSIGHTS"} -{"intent": "prepare an instagram post with this photo", "expected": "INSTAGRAM_CREATE_MEDIA_CONTAINER"} -{"intent": "open src/main.rs and show me the contents", "expected": "file_read"} -{"intent": "what's in the README?", "expected": "file_read"} -{"intent": "find every place we call parse_config", "expected": "grep"} -{"intent": "which files mention TODO in the crates folder?", "expected": "grep"} -{"intent": "list all the .toml files in the repo", "expected": "glob"} -{"intent": "what's in the current directory?", "expected": "list"} -{"intent": "write a hello world script to hello.py", "expected": "file_write"} -{"intent": "apply this diff to the parser", "expected": "apply_patch"} -{"intent": "run the test suite", "expected": "shell"} -{"intent": "execute npm install", "expected": "shell"} -{"intent": "commit these changes with the message fix typo", "expected": "git_operations"} -{"intent": "what's the latest news about the fed rate decision?", "expected": "web_search_tool"} -{"intent": "google who won the champions league", "expected": "web_search_tool"} -{"intent": "fetch https://example.com/pricing and summarize it", "expected": "web_fetch"} -{"intent": "call the weather API at api.weather.example/today", "expected": "http_request"} -{"intent": "remember that I prefer short answers", "expected": "save_preference"} -{"intent": "what did I tell you about my dog?", "expected": "memory_recall"} -{"intent": "store the fact that my flight is on the 14th", "expected": "memory_store"} -{"intent": "forget what I said about the old address", "expected": "memory_forget"} -{"intent": "what time is it in Tokyo right now?", "expected": "current_time"} -{"intent": "what's next friday's date?", "expected": "resolve_time"} -{"intent": "remind me every morning at 8 to drink water", "expected": "cron_add"} -{"intent": "what scheduled jobs do I have?", "expected": "cron_list"} -{"intent": "cancel the weekly report cron", "expected": "cron_remove"} -{"intent": "add buy milk to my todo list", "expected": "todo"} -{"intent": "show me my todos", "expected": "todo"} -{"intent": "draw a picture of a cat astronaut", "expected": "create_image"} -{"intent": "make a short video of waves at sunset", "expected": "create_video"} -{"intent": "what's in this screenshot?", "expected": "analyze_image"} -{"intent": "do a deep dive on competitors in the AI note-taking space", "expected": "web_search_tool"} -{"intent": "write and run a python script that sums a list", "expected": "python_exec"} -{"intent": "run this python snippet: print(2**10)", "expected": "python_exec"} -{"intent": "is there a skill for summarizing PDFs?", "expected": "skill_search"} -{"intent": "search the skill registry for a stripe integration", "expected": "skill_registry_search"} -{"intent": "install the skill from https://github.com/x/y", "expected": "install_workflow_from_url"} -{"intent": "what workflows have I saved?", "expected": "list_workflows"} -{"intent": "run the weekly digest workflow", "expected": "run_workflow"} -{"intent": "check if there's an app update", "expected": "update_check"} -{"intent": "update the app to the newest version", "expected": "update_apply"} -{"intent": "set a goal to ship v2 by december", "expected": "goal_set"} -{"intent": "what's my current goal?", "expected": "goal_get"} -{"intent": "is the app healthy? run diagnostics", "expected": "doctor_health"} -{"intent": "how much have I spent on AI this month?", "expected": "cost_get_summary"} -{"intent": "find an MCP server for postgres", "expected": "mcp_registry_search"} -{"intent": "call the query tool on the postgres mcp server", "expected": "mcp_registry_tool_call"} -{"intent": "connect my hubspot account", "expected": "oauth_connect_url"} -{"intent": "export these results as a csv", "expected": "csv_export"} -{"intent": "send a pushover notification to my phone", "expected": "pushover"} -{"intent": "unsubscribe me from all these marketing emails", "expected": "gmail_unsubscribe"} -{"intent": "swap 0.1 eth for usdc", "expected": "web3_swap_quote"} -{"intent": "schedule the report to run tomorrow at noon", "expected": "cron_add"} -{"intent": "how do I set up the proxy in openhuman?", "expected": "gitbooks_search"} -{"intent": "what does the persona file say about me?", "expected": "workspace_read_persona"} -{"intent": "what did we discuss in last month's emails about the merger?", "expected": "retrieve_memory"} -{"intent": "turn this into a reusable skill", "expected": "create_skill"} -{"intent": "are the background services running?", "expected": "service_status"} -{"intent": "run three research tasks in parallel on rust, go and zig", "expected": "spawn_parallel_agents"} -{"intent": "ask me which option I want before continuing", "expected": "ask_user_clarification"} -{"intent": "build a workflow that emails me the top HN posts daily", "expected": "build_workflow"} -{"intent": "suggest workflows I could automate", "expected": "suggest_workflows"} -{"intent": "pull my open tasks from linear", "expected": "task_source_list_tasks"} -{"intent": "what facets have you learned about me?", "expected": "learning_list_facets"} -{"intent": "list the artifacts from earlier today", "expected": "artifact_list"} -{"intent": "set my status to away", "expected": "none"} -{"intent": "hi!", "expected": "none"} -{"intent": "thanks, that's all", "expected": "none"} -{"intent": "what's the capital of australia?", "expected": "none"} -{"intent": "explain the difference between tcp and udp", "expected": "none"} -{"intent": "tell me a joke", "expected": "none"} -{"intent": "how are you today?", "expected": "none"} -{"intent": "what's 17 times 23?", "expected": "none"} -{"intent": "translate 'good morning' into spanish", "expected": "none"} -{"intent": "write a haiku about autumn", "expected": "none"} -{"intent": "who wrote pride and prejudice?", "expected": "none"} -{"intent": "summarize what you just said in one sentence", "expected": "none"} -{"intent": "what's a good name for a golden retriever?", "expected": "none"} -{"intent": "can you rephrase that more formally?", "expected": "none"} -{"intent": "why is the sky blue?", "expected": "none"} -{"intent": "give me three tips for better sleep", "expected": "none"} -{"intent": "ok", "expected": "none"} -{"intent": "what does HTTP stand for?", "expected": "none"} -{"intent": "is 97 a prime number?", "expected": "none"} -{"intent": "define the word 'ephemeral'", "expected": "none"} -{"intent": "what year did the berlin wall fall?", "expected": "none"} -{"intent": "help me think through whether to take the job offer", "expected": "none"} -{"intent": "write a limerick about rust borrow checker", "expected": "none"} -{"intent": "what's the plural of octopus?", "expected": "none"} -{"intent": "convert 5 miles to kilometers", "expected": "none"} -{"intent": "recommend a sci-fi novel", "expected": "none"} -{"intent": "sorry, ignore that last message", "expected": "none"} -{"intent": "what's your name?", "expected": "none"} -{"intent": "how many days are in a leap year?", "expected": "none"} -{"intent": "explain recursion like I'm five", "expected": "none"} -{"intent": "good night", "expected": "none"}