Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]

### Changed
- **Breaking: provider routing now uses independent object capabilities.** The
legacy broad `Routing` interface and `routing` graft are replaced by
provider discovery through `routing::Finder` (`routing-finder`) and host
PeerID announcement through `routing::Announcer` (`routing-announcer`).
Finder queries enforce the requested unique-provider count, deduplicate WAN
and LAN results, retain at most `min(count, 16)` selected results behind a
single-slot handoff, and use a per-request token to cancel after sink
callback failure, epoch expiry, or a 30-second deadline that includes
swarm-command admission. Announcer
registrations and republication now end with their final owning authority
epoch. Guest IPNS resolution/publication and persistent UnixFS mutation are
removed. Canonical CIDv1/raw/BLAKE3 routing-key derivation is now the
optional pure `wetware:routing/key@0.1.0` WIT import.
- **Wetware now owns default IPNS signing and can follow an IPNS deployment
Stem.** `ww run --ipns-stem <name>` uses locally verified raw records as the
authoritative deployment source. Signed EOL revokes the current generation
Expand All @@ -17,7 +30,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
`Gateway.ExposeRoutingAPI` and use the Gateway listener, which defaults to
`http://localhost:8080` and can be set with `IPFS_ROUTING_API` or
`--ipns-routing-url`. Wetware no longer creates or requires Kubo's `"ww"`
signing key. Guest `Routing.publish` is unchanged. `rust-ipns` is temporarily
signing key. The host publisher remains separate from guest provider routing.
`rust-ipns` is temporarily
pinned to reviewed commit `02c5ae7bf3f9568c7dbbb1308ae9299cfc7ba2d9`
pending upstream PR #503 or a release that contains its V2-only validation
fix.
Expand Down
11 changes: 10 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ members = [
"crates/membrane",
"crates/rpc",
"crates/guest/auth",
"crates/guest/routing-key",
"std/system",
"examples/chess",
"examples/discovery",
Expand Down Expand Up @@ -40,6 +41,7 @@ libp2p-stream = "0.4.0-alpha"
log = "0.4"
rand = "0.9.2"
reqwest = { version = "0.12.24", default-features = false }
routing-key = { path = "crates/guest/routing-key" }
serde = "1.0.228"
serde_json = "1.0.145"
tempfile = "3.23.0"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ an image. A published guest requires an application-specific composition path.
- [Capabilities](doc/capabilities.md): the capability model and Cap'n Proto schemas
- [CLI reference](doc/cli.md): full command-line usage
- [Image layout](doc/images.md): FHS convention, mounts, on-chain coordination
- [Routing](doc/routing.md): Kademlia DHT and peer discovery
- [Provider routing](doc/routing.md): independent discovery and host-PeerID announcement capabilities
- [Keys & identity](doc/keys.md): Ed25519 identity management
- [RPC transport](doc/rpc-transport.md): transport plumbing and scheduling model
- [Guest runtime](doc/guest-runtime.md): async runtime for WASM guests
Expand Down
2 changes: 1 addition & 1 deletion TODOS.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ usable by trusted FHS configuration or a future Warrant/ICME adapter.
## mDNS for Kubo-less LAN peer discovery
**What:** Add `libp2p::mdns::tokio::Behaviour` to `host::Behaviour` to discover LAN peers without Kubo. mDNS is a **peer discovery source** that feeds the LAN DHT routing table — not a routing primitive. It does not touch Cap'n Proto or the guest API.
**Why:** The dual DHT bootstraps the LAN routing table from Kubo's swarm peers. Without Kubo (or in environments where Kubo has no private-address peers), the LAN DHT starts empty. mDNS enables zero-config LAN discovery. Note: mDNS does NOT work in cloud/container environments (no multicast). Kubo bootstrap is the fallback/primary for those environments. Dual DHT and mDNS are orthogonal — can be built and merged independently.
**Context:** mDNS adds ~25-40 lines (config, event handling, address reconciliation). CI consideration: GitHub Actions runners may not support mDNS multicast, so mDNS-dependent tests should be `#[ignore]` or gated behind an env check. All critical logic remains testable via `LocalRouting` and mock swarm channels.
**Context:** mDNS adds ~25-40 lines (config, event handling, address reconciliation). CI consideration: GitHub Actions runners may not support mDNS multicast, so mDNS-dependent tests should be `#[ignore]` or gated behind an env check. All critical logic remains testable via `LocalProviderRouting` and mock swarm channels.
**Effort:** S (CC: ~30 min)
**Priority:** P3
**Depends on:** Dual DHT (architecturally orthogonal but LAN DHT should exist first so mDNS has a routing table to feed)
Expand Down
3 changes: 2 additions & 1 deletion capnp/membrane.capnp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ interface Membrane @0xdb52c25106bc2c5e {
# Pure capability provisioning (ocap model). Having a Membrane reference IS
# authorization — no signer needed. Wrap in Terminal(Membrane) to gate access.
#
# Canonical names: "identity", "host", "runtime", "routing", "http-client", "ipfs".
# Canonical names: "identity", "host", "runtime", "routing-finder",
# "routing-announcer", "http-client", "ipfs".
# Trusted pid0 may also receive explicitly configured extras.
#
# Listener/Dialer accessed via host.network().
Expand Down
69 changes: 22 additions & 47 deletions capnp/routing.capnp
Original file line number Diff line number Diff line change
@@ -1,61 +1,36 @@
# Content routing capability backed by the in-process Kademlia client.
# Narrow provider-routing capabilities backed by the in-process Kademlia client.
#
# Mirrors Go's coreiface.RoutingAPI (provide/findProviders only).
# Data transfer flows through the WASI virtual filesystem, not a capability.
# DHT key-value store (putValue/getValue) is deferred.
# Finder and Announcer are independent object capabilities. Canonical routing-key
# derivation is a pure Component Model import, not a Cap'n Proto capability.
#
# Epoch-scoped: the host wraps the implementation with an EpochGuard so all
# methods fail with stale-epoch once the epoch advances.
# Both capabilities are epoch-scoped. The host wraps each implementation with an
# EpochGuard so calls fail after the authority epoch advances.

@0xa7c3e8f1d4b29065;
@0xbb7178bb658e44b6;

struct ProviderInfo {
peerId @0 :Data; # libp2p peer ID, serialized.
addrs @1 :List(Data); # Multiaddrs for this provider, each serialized.
peerId @0 :Data; # Serialized libp2p PeerID.
addrs @1 :List(Data); # Serialized multiaddrs for the provider.
}

interface ProviderSink {
provider @0 (info :ProviderInfo) -> stream;
# Called once per discovered provider. -> stream enables
# Cap'n Proto flow control (backpressure).
provider @0 (info :ProviderInfo) -> ();
# Called once per unique provider. Each response applies backpressure and
# reports sink closure before Finder requests another result.

done @1 ();
# Signals that the search is complete. Errors from earlier
# provider() calls surface here.
# Signals that the finite lookup has completed.
}

interface Routing {
provide @0 (key :Text) -> ();
# Announce this node as a provider for the given CID.

findProviders @1 (key :Text, count :UInt32, sink :ProviderSink) -> ();
# Stream providers for a CID into the caller-supplied sink.

hash @2 (data :Data) -> (key :Text);
# Compute a deterministic CID (v1, raw codec, sha256) from data.
# Local operation — does not touch the network or Kubo.

resolve @3 (name :Text) -> (path :Text);
# Resolve an IPNS name to an IPFS path via Kubo.
# Returns e.g. "/ipfs/bafyrei..."

mkdir @4 (baseCid :Text, path :Text, parents :Bool) -> (rootCid :Text);
# Build a new UnixFS directory root by creating `path` relative to
# `baseCid`. Returns the new root CID. No global mutable root is used.

writeFile @5 (baseCid :Text, path :Text, data :Data, createParents :Bool) -> (rootCid :Text);
# Build a new UnixFS root by writing file bytes at `path` relative to
# `baseCid` (overwrite if present). Returns the new root CID.

remove @6 (baseCid :Text, path :Text, recursive :Bool) -> (rootCid :Text);
# Build a new UnixFS root by removing `path` relative to `baseCid`.
# Returns the new root CID.
interface Finder @0xebb8ace9d47ae6a8 {
findProviders @0 (key :Text, count :UInt32, sink :ProviderSink) -> ();
# `count` is the requested maximum number of unique provider peers.
# A count of zero launches no network query and returns no peers.
# Wetware applies a host maximum of 16 provider results.
}

publish @7 (name :Text, cid :Text, expectedCurrent :Text) -> (publishedPath :Text);
# Publish `/ipfs/<cid>` under IPNS `name`.
#
# Conflict semantics:
# - if `expectedCurrent` is empty, publish unconditionally.
# - if set, `name` must currently resolve to `expectedCurrent` (or fail).
# This is a compare-and-set guard to avoid silent last-write-wins.
interface Announcer @0xf52674c78f631b2f {
provide @0 (key :Text) -> ();
# Announce the Wetware host PeerID as a provider for the CID. Local provider
# registration and republication stop after the owning authority epoch ends.
}
51 changes: 27 additions & 24 deletions crates/atom/tests/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ impl GraftBuilder for StubSessionBuilder {
}

// ---------------------------------------------------------------------------
// Stub servers for all 5 graft capabilities (Identity, Host, Runtime, Routing, HttpClient)
// Stub servers for all 6 graft capabilities
// ---------------------------------------------------------------------------

/// Stub Identity: returns unimplemented for all methods.
Expand Down Expand Up @@ -163,37 +163,35 @@ impl http_capnp::http_client::Server for StubHttpClient {
}
}

/// Stub Routing: returns unimplemented for all methods.
pub struct StubRouting;
/// Stub Finder: returns unimplemented for provider discovery.
pub struct StubFinder;

#[allow(refining_impl_trait)]
impl routing_capnp::routing::Server for StubRouting {
fn provide(
self: capnp::capability::Rc<Self>,
_params: routing_capnp::routing::ProvideParams,
_results: routing_capnp::routing::ProvideResults,
) -> Promise<(), capnp::Error> {
Promise::err(capnp::Error::unimplemented("stub routing".into()))
}

impl routing_capnp::finder::Server for StubFinder {
fn find_providers(
self: capnp::capability::Rc<Self>,
_params: routing_capnp::routing::FindProvidersParams,
_results: routing_capnp::routing::FindProvidersResults,
_params: routing_capnp::finder::FindProvidersParams,
_results: routing_capnp::finder::FindProvidersResults,
) -> Promise<(), capnp::Error> {
Promise::err(capnp::Error::unimplemented("stub routing".into()))
Promise::err(capnp::Error::unimplemented("stub finder".into()))
}
}

/// Stub Announcer: returns unimplemented for provider announcement.
pub struct StubAnnouncer;

fn hash(
#[allow(refining_impl_trait)]
impl routing_capnp::announcer::Server for StubAnnouncer {
fn provide(
self: capnp::capability::Rc<Self>,
_params: routing_capnp::routing::HashParams,
_results: routing_capnp::routing::HashResults,
_params: routing_capnp::announcer::ProvideParams,
_results: routing_capnp::announcer::ProvideResults,
) -> Promise<(), capnp::Error> {
Promise::err(capnp::Error::unimplemented("stub routing".into()))
Promise::err(capnp::Error::unimplemented("stub announcer".into()))
}
}

/// GraftBuilder that populates ALL 5 graft capabilities with stubs (Export list format).
/// GraftBuilder that populates all six graft capabilities with stubs.
/// Used to verify that graft() returns every capability field.
pub struct FullStubSessionBuilder;

Expand All @@ -208,10 +206,11 @@ impl GraftBuilder for FullStubSessionBuilder {
let runtime: system_capnp::runtime::Client = capnp_rpc::new_client(StubRuntime {
guard: guard.clone(),
});
let routing: routing_capnp::routing::Client = capnp_rpc::new_client(StubRouting);
let finder: routing_capnp::finder::Client = capnp_rpc::new_client(StubFinder);
let announcer: routing_capnp::announcer::Client = capnp_rpc::new_client(StubAnnouncer);
let http_client: http_capnp::http_client::Client = capnp_rpc::new_client(StubHttpClient);

let mut caps = builder.reborrow().init_caps(5);
let mut caps = builder.reborrow().init_caps(6);

let mut e = caps.reborrow().get(0);
e.set_name("identity");
Expand All @@ -226,10 +225,14 @@ impl GraftBuilder for FullStubSessionBuilder {
e.init_cap().set_as_capability(runtime.client.hook);

let mut e = caps.reborrow().get(3);
e.set_name("routing");
e.init_cap().set_as_capability(routing.client.hook);
e.set_name("routing-finder");
e.init_cap().set_as_capability(finder.client.hook);

let mut e = caps.reborrow().get(4);
e.set_name("routing-announcer");
e.init_cap().set_as_capability(announcer.client.hook);

let mut e = caps.reborrow().get(5);
e.set_name("http-client");
e.init_cap().set_as_capability(http_client.client.hook);

Expand Down
15 changes: 8 additions & 7 deletions crates/atom/tests/membrane_integration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -366,14 +366,14 @@ async fn test_terminal_missing_signer_rejected() {
assert!(result.get_session().is_err());
}

/// Helper: create a Membrane client with all 5 capabilities populated.
/// Helper: create a Membrane client with all six capabilities populated.
fn full_stub_membrane(rx: watch::Receiver<Epoch>) -> membrane_capnp::membrane::Client {
new_client(MembraneServer::new(rx, FullStubSessionBuilder))
}

/// Verify that graft() returns all 5 capabilities: identity, host, runtime, routing, http-client.
/// Verify that graft() returns both routing capabilities as independent refs.
#[tokio::test]
async fn test_graft_returns_all_five_capabilities() {
async fn test_graft_returns_all_six_capabilities() {
let epoch = Epoch {
seq: 1,
head: b"head".to_vec(),
Expand All @@ -392,16 +392,17 @@ async fn test_graft_returns_all_five_capabilities() {
let results = graft_resp.get().expect("graft results");
let caps = results.get_caps().expect("caps");

// All 5 capabilities must be present by name.
assert_eq!(caps.len(), 5, "expected 5 capabilities");
assert_eq!(caps.len(), 6, "expected 6 capabilities");
let _identity: auth_capnp::identity::Client =
get_graft_cap(&caps, "identity").expect("identity capability should be present");
let _host: system_capnp::host::Client =
get_graft_cap(&caps, "host").expect("host capability should be present");
let runtime: system_capnp::runtime::Client =
get_graft_cap(&caps, "runtime").expect("runtime capability should be present");
let _routing: routing_capnp::routing::Client =
get_graft_cap(&caps, "routing").expect("routing capability should be present");
let _finder: routing_capnp::finder::Client =
get_graft_cap(&caps, "routing-finder").expect("Finder capability should be present");
let _announcer: routing_capnp::announcer::Client =
get_graft_cap(&caps, "routing-announcer").expect("Announcer capability should be present");
let _http_client: http_capnp::http_client::Client =
get_graft_cap(&caps, "http-client").expect("http-client capability should be present");

Expand Down
16 changes: 16 additions & 0 deletions crates/authority/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,22 @@ mod wire_type_id_tests {
<crate::membrane_capnp::membrane::Client as HasTypeId>::TYPE_ID,
0xdb52_c251_06bc_2c5e
);
assert_eq!(
<crate::routing_capnp::finder::Client as HasTypeId>::TYPE_ID,
0xebb8_ace9_d47a_e6a8
);
assert_eq!(
<crate::routing_capnp::announcer::Client as HasTypeId>::TYPE_ID,
0xf526_74c7_8f63_1b2f
);
assert_ne!(
<crate::routing_capnp::finder::Client as HasTypeId>::TYPE_ID,
0xa7c3_e8f1_d4b2_9065
);
assert_ne!(
<crate::routing_capnp::announcer::Client as HasTypeId>::TYPE_ID,
0xa7c3_e8f1_d4b2_9065
);
}
}

Expand Down
1 change: 1 addition & 0 deletions crates/cell/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ edition = "2021"
[dependencies]
anyhow = { workspace = true }
async-trait = { workspace = true }
blake3 = { workspace = true }
tokio = { workspace = true, features = ["full"] }
tracing = { workspace = true }
arc-swap = "1"
Expand Down
1 change: 1 addition & 0 deletions crates/cell/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ pub mod image;
pub mod loaders;
pub mod mount;
pub mod proc;
pub mod routing_key;
pub mod sched;
pub mod vfs;

Expand Down
Loading