feat(drivers): tinyflows-drivers — StateStore and Checkpointer over tinystoragedrivers ports

CHANGELOG.md

@@ -9,6 +9,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **`tinyflows-drivers` (new, unpublished crate).** `DriverStateStore`
+  implements the engine's `StateStore` and `DriverCheckpointer` the graph
+  `Checkpointer` over a `tinystoragedrivers` document port (git dependency at
+  `v0.4.0`, so a host that vendors the storage crates once can `[patch]` it).
+  The checkpointer stores one document per checkpoint with a per-thread
+  compare-and-swap sequence, merges pending writes under compare-and-swap,
+  pushes namespace-scoped reads down to one indexed query, reads a whole
+  `state_history` in two queries, length-prefixes every key component and
+  hashes over-long ids with SHA-256. A separate crate rather than an engine
+  feature because the published `tinyflows` package cannot carry a git
+  dependency; the engine is unchanged and keeps its MSRV, this crate needs
+  Rust 1.88.
+
 - **`tinyflows-sqlite` opens `flows.db` and `jobs.db` through the
   tinystoragedrivers SQLite driver's native mode.** The tables, SQL,
   `user_version` gate and column migrations are unchanged, and existing

README.md

@@ -90,6 +90,12 @@ Rust 2024 · MSRV 1.85 · `#![forbid(unsafe_code)]` · GPL-3.0-or-later.
 - Host-injected capability traits: `LlmProvider`, `ToolInvoker`, `HttpClient`,
   `CodeRunner`, `ShellRunner`, and `StateStore`. Deterministic in-memory mocks ship behind the
   `mock` cargo feature (`caps::mock::mock_capabilities()`).
+- Storage-driver backends in the companion `tinyflows-drivers` crate:
+  `DriverStateStore` (a `StateStore`) and `DriverCheckpointer` (a graph
+  `Checkpointer`) over a
+  [tinystoragedrivers](https://github.com/tinyhumansai/tinystoragedrivers)
+  document port, so a host keeps run state and checkpoints in the SQLite,
+  MongoDB or memory backend it already opened.
 - Opaque `connection_ref` credential references — the host resolves them to real
   secrets; the crate never sees them.
 - Versioned wire format: graph `schema_version` and per-node `type_version`, with

crates/tinyflows-drivers/Cargo.toml

@@ -0,0 +1,31 @@
+# `tinyflows` capabilities and checkpoints on `tinystoragedrivers` ports.
+#
+# A crate of its own rather than a feature of the engine: the storage crates
+# are git dependencies (OpenHuman vendors them once and `[patch]`es the URL),
+# and the published `tinyflows` package cannot carry a git dependency, even an
+# optional one. It also keeps the engine's MSRV where it is; this crate needs
+# the storage crates' 1.88.
+[package]
+name = "tinyflows-drivers"
+version = "0.1.0"
+description = "tinyflows StateStore and graph Checkpointer over tinystoragedrivers document ports"
+edition.workspace = true
+rust-version = "1.88"
+license.workspace = true
+repository.workspace = true
+homepage.workspace = true
+readme = "README.md"
+publish = false
+
+[dependencies]
+tinyflows = { workspace = true }
+tinystoragedrivers-core = { git = "https://github.com/tinyhumansai/tinystoragedrivers", tag = "v0.4.0" }
+async-trait = { workspace = true }
+serde = { workspace = true }
+serde_json = { workspace = true }
+sha2 = { workspace = true }
+tokio = { workspace = true }
+tracing = { workspace = true }
+
+[dev-dependencies]
+tokio = { workspace = true }

crates/tinyflows-drivers/README.md

@@ -0,0 +1,46 @@
+# tinyflows-drivers
+
+`tinyflows` run state and graph checkpoints on
+[tinystoragedrivers](https://github.com/tinyhumansai/tinystoragedrivers)
+document ports. A host that has opened a storage backend (SQLite on a desktop,
+MongoDB in the cloud, memory in tests) gets the engine's durable pieces on it,
+scoped per tenant by the document handle it passes in.
+
+| Type | Implements | Layout |
+| --- | --- | --- |
+| `DriverStateStore` | `tinyflows::caps::StateStore` | one document per key in `flows_state` |
+| `DriverCheckpointer<State>` | `tinyflows::graph::Checkpointer<State>` | `flows_graph_checkpoints`, `_threads`, `_writes` |
+
+```rust
+use std::sync::Arc;
+use tinyflows_drivers::{DriverCheckpointer, DriverStateStore};
+
+let docs = Arc::clone(scoped_storage.documents());
+let state = Arc::new(DriverStateStore::new(Arc::clone(&docs)));
+let checkpointer = DriverCheckpointer::<serde_json::Value>::new(docs);
+```
+
+## Checkpointer guarantees
+
+- One document per checkpoint with a per-thread `seq` advanced by
+  compare-and-swap, so listing is insertion order and a re-used checkpoint id
+  resolves to its latest write, as in the SQLite and file backends.
+- `get_scoped` is one indexed `(thread, namespace, seq)` query, so a parent
+  run and its subgraphs never load each other's checkpoints, even when ids
+  repeat.
+- `state_history` reads the namespace's checkpoints and pending writes in two
+  queries and walks the lineage in memory, rather than two round trips per
+  ancestor.
+- Pending writes merge with `merge_writes` under compare-and-swap.
+- Key components are length-prefixed and namespaces encoded injectively; ids
+  over 400 bytes become their SHA-256.
+
+## Why a separate crate
+
+`tinystoragedrivers` is a git dependency: OpenHuman vendors it once (through
+TinyAgents) and points the URL at that copy with
+`[patch."https://github.com/tinyhumansai/tinystoragedrivers"]`, so there is
+exactly one `DocumentStore` trait in its graph. The published `tinyflows`
+package cannot carry a git dependency, so these adapters live here
+(`publish = false`). The engine stays dependency-free and keeps its MSRV; this
+crate needs Rust 1.88, the storage crates' MSRV.