The skill OS that stands next to the robot.
Runs like your bruhhh.
bruhos.ai · @bruhos_ai · Design doc (中文)
bruhOS is a partner OS for robots — it hears you, runs skills, and leaves a trace.
Open brains already exist: NVIDIA Isaac GR00T, Cosmos, Isaac Lab, Hugging Face LeRobot, openpi, Qwen-VL. What's missing is the system between the model and the body. bruhOS plugs those brains into a TonyPi humanoid as planners and data engines, while the robot itself only runs whitelisted skills, speaks English, gets back up, and leaves a hashable trace.
Not a brain in the cloud. A bruh on the body.
English command
│
▼
┌────────────────────────┐
│ bruhOS Brain │
│ planner adapters │
│ Qwen-VL / OpenAI API │
│ optional: GR00T S2 │
│ optional: LeRobot svc │
└────────────┬───────────┘
│ skill JSON
▼
┌────────────────────────┐
│ bruhOS Runtime (Pi 5) │
│ observe · dispatch │
│ recover · trace │
└────────────┬───────────┘
│
┌──────────────┼──────────────┐
▼ ▼ ▼
TonyPi body Trace Writer Offline stack
action groups hash / sinks Isaac + Cosmos
LeRobot datasets
- Skills, not chat. The unit of the system is a skill call. Anything outside the whitelist is rejected before it reaches the body.
- Brains are pluggable. Every adapter emits the same skill JSON; swapping the brain never renames a skill.
- NVIDIA in the right layer. Isaac / Cosmos are for data and simulation, not the control loop.
GR00T is used as System 2 (task decomposition) only — its joint-action head is never routed to
servos, and the runtime raises
IntegrationErrorif a brain tries. - Compressed observation. The brain gets a small JSON of detections, IMU and line state — not raw video.
- Failure is first-class. Falls are detected from the IMU, the robot stands up, and the interrupted skill resumes. Recovery is scored separately from completion.
- Execution is data. Every episode is a hash-chained trace with a Merkle
trace_root, exportable to LeRobot-aligned frames, and optionally anchored on-chain (hashes only — never video, never control).
bruhOS is pre-alpha (0.1.0.dev0). This is what is real today and what is not:
| Part | State |
|---|---|
| Runtime: plan → validate → dispatch → fall recovery → replan → trace | Working, tested in sim |
| Skill whitelist (10 skills) and arg validation | Working, tested |
| Kinematic tabletop sim with fall injection | Working |
| 20-prompt English eval (completion + recovery reported separately) | Working in sim |
rules brain (offline English planner, no model) |
Working |
openai brain (any OpenAI-compatible endpoint) |
Implemented; tested against a mocked endpoint |
qwen_vl brain (local vLLM, optional camera frame) |
Implemented; tested against a mocked endpoint |
lfm3_omni brain (Liquid LFM3-Omni-2B in llama.cpp on the Pi 5, optional camera frame) |
Implemented; sim eval run against the real model on a Pi 5 |
groot_s2 brain |
Stub: contract + joint-action guard; real GR00T wrapper is v0.2 |
lerobot_policy brain |
Interface: HTTP client for a skill-JSON policy server |
| Trace hashing, Merkle root, verify, local sink | Working, tested |
| IPFS sink (Kubo HTTP API) | Implemented, not covered by tests |
| Chain sink | Builds the anchor payload; transaction submit is pluggable, enabled in v0.3 |
| LeRobot export | Field names / shapes aligned; writing a LeRobotDataset is v0.2 |
| TonyPi body (Hiwonder action groups, OpenCV color blobs, MPU-6050) | Implemented, not yet validated on a robot — action-group names must be checked against your image |
elizaOS robot bridge body (--body eliza, elizaOS/research) |
Implemented; tested against the bridge's mock backend. Its MuJoCo backend currently refuses motion commands |
| Isaac Lab twin / Cosmos augmentation | Scene spec only (bruhos/sim/tabletop_task.toml); v0.2 |
Requires Python 3.11+. The core has no third-party dependencies.
git clone https://github.com/bruhosdotai/bruhOS.git && cd bruhOS
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
bruhos run "Put the red one in the box." # sim body + offline rules brain
bruhos eval # 20 English prompts in sim
bruhos eval --fall-prob 0.05 --seed 1 # same prompts with random falls
bruhos skills # the whitelist + skillpack hash
bruhos verify traces/<episode_id> # re-hash a trace
bruhos lerobot traces/<episode_id> # LeRobot-aligned frames
pytestExample run:
0 skill detect(target=red) -> ok found red_block
1 skill walk_toward(target=red, stop_distance_m=0.1) -> ok reached red_block
2 skill grab() -> ok holding red_block
3 skill walk_toward(target=box, stop_distance_m=0.1) -> ok reached box
4 skill release() -> ok released red_block
Sim baseline for the rules brain (kinematic sim, not hardware):
| Setting | Completion | Falls | Recovered |
|---|---|---|---|
| no falls | 20 / 20 | 1 (scripted, "Stand up.") | 1 |
--fall-prob 0.05 --seed 1 |
16 / 20 | 28 | 24 |
Same prompts with the lfm3_omni brain: LFM3-Omni-2B Q4_0 in llama.cpp on a Raspberry Pi 5 (8 GB, CPU
only), 3–12 s per plan depending on its length:
| Setting | Completion | Falls | Recovered |
|---|---|---|---|
| no falls | 18 / 20 | 1 (scripted, "Stand up.") | 1 |
--fall-prob 0.05 --seed 1 |
15 / 20 | 26 | 23 |
It misses "Grab the blue one." (grabs before walking to it) and "Place the green cube into the box."
(asks for green_cube; the sim calls it green_block).
# OpenAI or any OpenAI-compatible server
export OPENAI_API_KEY=...
bruhos run "Wave hello." --brain openai
# Local Qwen-VL on a desktop GPU (not on the Pi)
vllm serve Qwen/Qwen2.5-VL-7B-Instruct --port 8000
export BRUHOS_LLM_BASE_URL=http://<gpu-host>:8000/v1
bruhos run "Kick the ball." --brain qwen_vl
# Liquid LFM3-Omni-2B on the Pi itself: CPU only, ~3 GB RAM, no GPU host
llama-server -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_0 --jinja --swa-full -ub 64 -cms 64 \
-t 3 -tb 4 -c 8192 --temp 0 --port 8080
bruhos run "Kick the ball." --brain lfm3_omniBrain HTTP calls never follow redirects, so an API key cannot be forwarded to another origin: point
BRUHOS_LLM_BASE_URL (or base_url) at the final endpoint. A real API key is only sent over
https:// or to this machine (127.0.0.1, localhost); a LAN server without a key, like the vLLM
example above, works over plain http://. For a keyed server on a trusted network, set
allow_insecure = true under [brain], or use an SSH tunnel.
The lfm3_omni brain turns LFM3's thinking off (plans in a few seconds instead of ~50 s on a Pi 5) and
constrains the reply with a JSON schema built from the skill catalog, so the model cannot produce a
skill, arg or choice outside the whitelist. A 2B model plans better from a short observation (object
names, distances, what it holds) and a few worked examples, so that is what it gets; the examples use
objects that are not in the sim or the eval. --swa-full keeps the catalog prompt cached between
plans. Weights: LFM Open License v1.0.
All brains receive the skill catalog and the observation JSON and must answer with:
{"plan": [{"skill": "walk_toward", "args": {"target": "red_block"}}, {"skill": "grab", "args": {}}]}GR00T System-2 contract (v0.1 stub):
GR00T_S2(image, "Put the red one in the box.")
→ ["detect", "walk_toward(red_block)", "grab", "walk_toward(box)", "release"]
→ Runtime executes them one by one
GR00T action chunk → 16 servos is a wrong integration and is rejected.
# on the robot (Raspberry Pi 5)
pip install -e ".[vision,imu]"
bruhos run "Wave hello." --body tonypi --config configs/tonypi.tomlBefore the first run, open configs/tonypi.toml (or tonypi_pro.toml) and check every name under
[actions] against the ActionGroups/ directory on your robot, plus sdk_path for the Hiwonder
SDK. Unmapped primitives (e.g. grab on a TonyPi without hands) make the skill return failed
instead of guessing.
| Skill | Args | What it does |
|---|---|---|
look_at |
target |
Turn to face a target |
detect |
target? |
List visible objects, or scan until target is in view |
walk_toward |
target, stop_distance_m=0.1 |
Walk until the target is in reach |
follow_line |
steps=20 |
Follow floor tape |
grab |
— | Close the hand on the object in reach |
release |
— | Open the hand |
kick |
foot=left|right |
Kick |
wave |
— | Wave |
stand_up |
— | Recover from a fall (front/back from the IMU) |
stop |
— | Halt now. Local only; never waits on brain, sinks or chain |
Brains must output this schema. The runtime is never changed to fit a checkpoint's action space.
Each step is hashed over canonical JSON and chained to the previous step; the episode's
trace_root is the Merkle root of the step hashes. Sinks (LocalSink, IpfsSink, ChainSink)
run on a background thread, so a slow or failing sink never blocks the control loop. What goes
on-chain is trace_root plus the skillpack and brain version hashes — not video, not control.
bruhos run "Wave hello." --traces traces # simulator; no hardware required
bruhos dashboard --traces tracesOpen http://127.0.0.1:8765/. Use the same trace directory as the runtime; --port
changes the listening port. The read-only console refreshes every two seconds and
shows the latest 12 readable episodes, outcome filters and expandable trace details.
It opens no hardware and changes no runtime settings.
The server binds to localhost. For remote viewing, forward the port with
ssh -N -L 8876:127.0.0.1:8765 pi@ROBOT_IP, then open http://127.0.0.1:8876/.
Verification uses the existing trace hash chain, which does not cover all header
metadata or independently prove physical completion. Live hardware status is not
reported. Assets work offline; system fonts require no downloads.
bruhos/
runtime/ observe · dispatch · recover · trace
brain/adapters/ rules · openai · qwen_vl · lfm3_omni · groot_s2 (stub) · lerobot_policy
skills/ whitelist + implementations (written once against body primitives)
body/ Body interface · TonyPi · perception (OpenCV) · MPU-6050
speech/ English-first ASR / TTS
trace/ hashing · writer · sinks (local | ipfs | chain) · LeRobot frames
sim/ kinematic tabletop · Isaac Lab scene spec
schemas/ obs · skill · trace
eval/ 20 English prompts + runner
configs/ tonypi · tonypi_pro
docs/ design doc
tests/
| Version | Integration | Product |
|---|---|---|
| v0.1 | TonyPi + Qwen-VL/API + LeRobot schema + local trace | English skill OS, 20-prompt on-robot eval |
| v0.2 | Isaac Lab regression; Cosmos failure data; GR00T S2 adapter | Change the brain without re-testing on hardware every time |
| v0.3 | Two bodies: TonyPi skills + arm with π0.5 / SmolVLA; ChainSink on | Walk, see, talk — and grasp — as separate layers |
| later | On-device VLA only with Jetson-class hardware | Still no GR00T whole-body head on a Pi 5 |
PRs welcome, including through Slop. Rules in CONTRIBUTING.md.
Apache-2.0. See LICENSE. Third-party models and datasets keep their own licenses; non-commercial datasets are not used in the default product path.




