Skip to content

Commit 369e70c

Browse files
committed
Document the Unikraft sandbox backend
How to run it from Rust and Python, the images it takes, and how it differs from the other backends: one guest memory size, host-mounted /input and /output checked after each run and cleared before it, socket-level networking by host name only, data exchanged in pieces past the 64 KiB host call limit, CPython exit statuses, and hyperlight-unikraft's guest limits. Signed-off-by: danbugs <danilochiarlone@gmail.com>
1 parent 59f5f01 commit 369e70c

1 file changed

Lines changed: 45 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Supported backends:
1212

1313
- [Wasm Component Sandbox](#wasm-component-sandbox) (Python/Javascript or provide your own)
1414
- [HyperlightJS Sandbox](#hyperlightjs-sandbox)
15+
- [Unikraft Sandbox](#unikraft-sandbox) (full CPython, with numpy/pandas/scipy/scikit-learn)
1516

1617
## Overview
1718

@@ -195,6 +196,50 @@ console.log('10 + 20 = ' + sum);
195196

196197
See [examples](./src/javascript_sandbox/examples/) for file I/O and network demos.
197198

199+
### Unikraft Sandbox
200+
201+
Runs full CPython 3.12 in a [Unikraft](https://unikraft.org) unikernel inside a Hyperlight micro-VM, via [hyperlight-unikraft](https://github.com/hyperlight-dev/hyperlight-unikraft). Use it when guest code needs native extensions the Wasm guest cannot load: the published `agent` image ships numpy, pandas, scipy, scikit-learn, matplotlib and more. Guest code gets the same globals as the Wasm Python guest (`call_tool`, `http_get`, `http_post`), `/input` and `/output`, and snapshot/restore of the whole micro-VM.
202+
203+
The module path is a rootfs image (initrd CPIO). `cached_rootfs("agent")` finds the one the [`hluk`](https://github.com/hyperlight-dev/hyperlight-unikraft) CLI caches (`just unikraft pull-images` pulls `python` and `agent`).
204+
205+
```rust
206+
use hyperlight_sandbox::{SandboxBuilder, ToolRegistry};
207+
use hyperlight_unikraft_sandbox::{Unikraft, cached_rootfs};
208+
209+
fn main() -> anyhow::Result<()> {
210+
let mut tools = ToolRegistry::new();
211+
tools.register_unvalidated("fetch_data", |_| Ok(serde_json::json!([1, 2, 3, 4])));
212+
213+
let mut sandbox = SandboxBuilder::new()
214+
.module_path(cached_rootfs("agent")?.display().to_string())
215+
.with_tools(tools)
216+
.guest(Unikraft::new())
217+
.build()?;
218+
219+
let result = sandbox.run(r#"
220+
import numpy as np
221+
print(np.mean(call_tool('fetch_data')))
222+
"#)?;
223+
print!("{}", result.stdout);
224+
Ok(())
225+
}
226+
```
227+
228+
From Python: `Sandbox(backend="unikraft")` (the `agent` image by default; `module="python"` or `module_path=...` for another), installed with `hyperlight-sandbox[unikraft]`.
229+
230+
How it differs from the other backends:
231+
232+
- Only the `python`, `python-shell` and `agent` images are supported, and they must be the hyperlight-unikraft release the crate is built against (`IMAGE_RELEASE`).
233+
- Guest memory is one size, the image's tested default (1.5 GiB for `agent`); set it with `Unikraft::scratch_mb` (`scratch_size="2Gi"` in Python, `WithScratchSize("2Gi")` in .NET). Heap and stack sizes do not apply: the Python and .NET SDKs refuse them, and the Rust builder's are ignored.
234+
- `/input` and `/output` are host directories mounted into the guest when it boots, so the guest writes `/output` directly. `/output` is read-write if its permissions allow creating and writing files, else read-only; it cannot be write-only. As with the other backends, files in `/output` persist from run to run, and restoring a snapshot clears them; files the guest has open are interpreter state, so they stay open across runs and through a restore. The filesystem limits are checked after each run: a run that exceeded them, or that left a symbolic or hard link, a special file, or a tree deeper than 64 levels or over 100,000 entries in `/output`, fails, and `/output` is cleared of every file and link (earlier runs' included) and of the directories made during the run; directories it had when the run started are kept, emptied. The bytes did reach the host disk first, so put the output directory on quota-controlled storage. The same check runs before each run, for what the host put in `/output` since: a run that would start on such things is refused, and they are left in place. The guest keeps the directories it booted with; if either is replaced on the host after the sandbox is built, the re-boot an allowlist change makes (with `native_network`) fails until the original is put back.
235+
- `http_get` / `http_post` go through the host and the `allow_domain` allowlist, as in the Wasm guest. Guest sockets (`requests`, `urllib`) are off unless `Unikraft::native_network(true)` (`native_network=True`), and are then allowed to the `allow_domain` hosts by name only: scheme, port, path and method filters apply to `http_get` / `http_post` only. Those names are resolved when the allowlist changes; one that does not resolve is left out (the next run's stderr says so).
236+
- One host call carries at most 64 KiB; larger code, tool arguments and results, and HTTP bodies are chunked transparently. The host holds at most 64 MiB of them unread at a time, so a single tool result or HTTP response can be up to that size.
237+
- Exit statuses are those a CPython process exits with, 0 to 255 (`sys.exit(-1)` is 255, `sys.exit(256)` is 0), so -1 always means the guest itself failed.
238+
- Guest code can run threads and child processes (`subprocess`, and shell pipelines in the `agent` and `python-shell` images, which have a `/bin/sh`). Output written straight to the file descriptors (by child processes, C extensions, or the guest kernel, whose messages are standard error) reaches the run's `stdout` and `stderr` too.
239+
- hyperlight-unikraft's guest limits apply: a single vCPU and no AVX.
240+
241+
See [examples](./src/unikraft_sandbox/examples/) and [the Python example](./src/sdk/python/core/examples/unikraft_basics.py).
242+
198243
### Writable filesystem limits
199244

200245
Writable output is bounded by default to 5 MiB per file, 20 MiB of cumulative

0 commit comments

Comments
 (0)