T2.7 fix 1: the memory provider on the default layer too, --layer default

sdm-contract-run --database memory --layer default runs the same non-SQL provider through sdm_host::default_root() in place of the second data layer's root, keyed <case>@memory, so the rewritten swapped-layer test compares two runs that differ in their data layer alone. host::run_memory takes the root. The pending list is empty.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>

Author: adi-kumo

contract/tests/t27-pending.json

@@ -1,8 +1,6 @@
 {
  "about": "T2.7's acceptance tests (the non-SQL test provider and the swapped data layer) that failed when the test author wrote them, before the implementation (sdm-testkit's tests/t27_non_sql_provider.rs as rust:<test>). Each listed test must still fail (strict); the implementer takes a name out of 'pending' once it passes, and changes nothing else: 'pending' stays within 'authored', which is the test author's.",
- "pending": [
-  "rust:the_same_prediction_runs_on_the_swapped_layer"
- ],
+ "pending": [],
  "authored": [
   "rust:the_kit_passes_on_the_non_sql_provider",
   "rust:a_seeded_fault_fails_the_kit_on_the_non_sql_provider",

docs/design/rust-core/phase-2/tracker-T2.7.md

@@ -30,8 +30,8 @@ Both are dev-only, in `sdm-testkit`; nothing shipped reaches them (`l0_deps.py`:
 
 - **The provider passes the kit, including two schema tables over one source**: `t27_non_sql_provider.rs`'s `the_kit_passes_on_the_non_sql_provider` passes (every kit case, `key-counts.one-source-twice` among them), and `a_seeded_fault_fails_the_kit_on_the_non_sql_provider` passes (truncation and a silenced capability each fail the cases `expected.json` names).
   Beyond the kit, `memory_provider.rs`'s `two_schema_tables_over_one_source_predict_as_two_sources`: `shape.same-table-twice.buyers` with both schema tables reading `customers` (and `buyers` dropped) predicts exactly as over the two equal tables.
-- **The same prediction runs on the swapped layer**: not met as the acceptance test words it; `the_same_prediction_runs_on_the_swapped_layer` stays pending (below).
-  What holds (`memory_provider.rs`'s `a_prediction_is_duckdbs_but_for_its_drawn_examples`, on `base.binary` and `shape.same-table-twice.buyers`): every fetch planned as the typed request, no statement run; the description equal to DuckDB's; the prediction's result equal to DuckDB's; the materialized request and the model call body equal to DuckDB's but for `context`, the in-context examples.
+- **The same prediction runs on the swapped layer**: `the_same_prediction_runs_on_the_swapped_layer` passes, as the test author rewrote it (round 1, below): the provider on the swapped layer gives, whole, what it gives on the default layer (`--database memory --layer default`), and DuckDB's but for the drawn example rows.
+  What also holds (`memory_provider.rs`'s `a_prediction_is_duckdbs_but_for_its_drawn_examples`, on `base.binary` and `shape.same-table-twice.buyers`): every fetch planned as the typed request, no statement run; the description equal to DuckDB's; the prediction's result equal to DuckDB's; the materialized request and the model call body equal to DuckDB's but for `context`, the in-context examples.
 - **A guard shows no domain crate changed for it**: `no_domain_crate_changed_for_t2_7` passes on this branch (no `T2.7` commit touches a crate `l0_deps.py` classes as domain); `git diff --stat d339e88c..HEAD` touches `sdm-testkit`, `sdm-host`, `rust/Cargo.lock`, the kit file and docs only.
 
 ### Probes
@@ -46,10 +46,16 @@ Both are dev-only, in `sdm-testkit`; nothing shipped reaches them (`l0_deps.py`:
 - `pytest contract/tests -n 4 --dist loadfile` on that build: 4,543 pass; the 132 failures are this machine's environment, not the change: 118 refuse to run outside the recorded `ref-pandas3` cell (this venv's pyarrow is 26.0.0, the cell's 25.0.1), and the gate-mode tests' check 1 is reported as observed in no cell for the same reason.
   `tests/test_release_contract.py`'s two snapshot tests fail in the shared venv, whose packages are editable installs of another checkout; no Python file changes here.
 
+### Round 1 (the swapped prediction's test)
+
+The test author rewrote `the_same_prediction_runs_on_the_swapped_layer` (`T2.7 tests:`, cherry-picked unedited): the swapped run against the same provider on the default layer, whole, and against DuckDB but for the example rows.
+For it, `sdm-contract-run --database memory --layer default` runs the same provider through `sdm_host::default_root()` (`host::run_memory` takes the root), keyed `<case>@memory` as the swapped run is; the pending list is empty.
+The T2.7 tests, the Rust kit tests (both hosts' `t25_provider_kit`, DuckDB and SQLite) and the gate's rust-host leg pass at the new head.
+
 ### Deviations and questions
 
-- **The swapped prediction's acceptance test cannot pass as written** (`the_same_prediction_runs_on_the_swapped_layer`, pending): it asks the swapped run's `core.materialized` and `core.predicted` to equal DuckDB's whole, including the in-context examples, which are a seeded sample each provider draws its own way (DuckDB's reservoir, SQLite's RS-2 stream, Postgres's `setseed`, the fallback's RS-1).
+- **The swapped prediction's acceptance test, as first written, could not pass** (resolved in round 1): it asked the swapped run's `core.materialized` and `core.predicted` to equal DuckDB's whole, including the in-context examples, which are a seeded sample each provider draws its own way (DuckDB's reservoir, SQLite's RS-2 stream, Postgres's `setseed`, the fallback's RS-1).
   Matching DuckDB's would mean reimplementing DuckDB's reservoir sampler in the test provider.
-  Proposed to the test author: compare everything but `context` (as `memory_provider.rs` does), or compare the swapped run to the same provider on the default root, which is equal whole.
+  The test author took both proposals: everything but the example rows against DuckDB, and the whole run against the same provider on the default root.
 - **Found in the core, not changed here (a domain crate)**: `sdm-core/src/materialize.rs`'s `sampling_capabilities` refuses a provider that does not declare `time_bounds`, `sample_entities`, `neighbors` or `lookup` before it reads anything, even when the provider declares the in-memory fallback's limits, so a provider relying on the fallback cannot `materialize` (design choice 1 says such a request takes the fallback).
   This provider declares `time_bounds` and `sample_entities` itself instead; the core's check is for the orchestrator to route to its own package.

rust/crates/sdm-testkit/src/bin/sdm-contract-run.rs

@@ -9,7 +9,7 @@
 //! sdm-contract-run --providers
 //! sdm-contract-run --replay RECORDING --out FILE CASE_OR_DIRECTORY...
 //! sdm-contract-run --database sqlite --out FILE CASE_OR_DIRECTORY...
-//! sdm-contract-run --database memory --out FILE CASE_OR_DIRECTORY...
+//! sdm-contract-run --database memory [--layer default] --out FILE CASE_OR_DIRECTORY...
 //! sdm-contract-run --kit DATABASE [--fault truncate|silent_capability] --out REPORT
 //! ```
 //!
@@ -29,7 +29,10 @@
 //! DuckDB base copied into the minimal non-SQL test provider
 //! ([`sdm_testkit::memory`]), through the second data layer swapped in at
 //! the composition root ([`sdm_testkit::layer`]), each observation keyed
-//! `<case>@memory`; a case that reads no data is not run there.
+//! `<case>@memory`; a case that reads no data is not run there. With
+//! `--layer default` the same provider runs through
+//! `sdm_host::default_root()`'s layer (`sdm-data`'s) instead, keyed the
+//! same, so the two runs differ in their data layer alone.
 //!
 //! `--replay` runs each case that runs on Postgres there, its fetches
 //! answered from a recording of a Postgres server's answers
@@ -86,6 +89,20 @@ fn run(arguments: &[String]) -> Result<(PathBuf, usize)> {
         _ => (None, arguments),
     };
     let sqlite = database == Some("sqlite");
+    // The data layer a `--database memory` run composes: the second one
+    // (T2.7), or, with `--layer default`, the default root's.
+    let (root, arguments) = match arguments {
+        [flag, layer, rest @ ..] if flag == "--layer" && layer == "default" => {
+            if database != Some("memory") {
+                bail!("--layer default: only with --database memory");
+            }
+            (sdm_host::default_root(), rest)
+        }
+        [flag, layer, ..] if flag == "--layer" => {
+            bail!("--layer {layer}: the one layer to name is `default`")
+        }
+        _ => (sdm_testkit::layer::root(), arguments),
+    };
     let (recording, arguments) = match arguments {
         [flag, path, rest @ ..] if flag == "--replay" => (
             Some(replay::Recording::load(std::path::Path::new(path))?),
@@ -99,7 +116,7 @@ fn run(arguments: &[String]) -> Result<(PathBuf, usize)> {
             cases.iter().map(PathBuf::from).collect::<Vec<_>>(),
         ),
         _ => bail!(
-            "usage: sdm-contract-run [--pre-given-specs] [--database sqlite|memory | --replay RECORDING] --out FILE CASE_OR_DIRECTORY..."
+            "usage: sdm-contract-run [--pre-given-specs] [--database sqlite | --database memory [--layer default] | --replay RECORDING] --out FILE CASE_OR_DIRECTORY..."
         ),
     };
     let mut partial = out.clone().into_os_string();
@@ -147,7 +164,7 @@ fn run(arguments: &[String]) -> Result<(PathBuf, usize)> {
                 }
                 let mut keyed = each.clone();
                 keyed.case.id = format!("{}@memory", each.case.id);
-                (host::run_memory(&each.case, &bases)?, keyed)
+                (host::run_memory(&each.case, &bases, &root)?, keyed)
             }
             None => (host::run(&each.case, &bases)?, each.clone()),
             Some(recording) => {

rust/crates/sdm-testkit/src/host.rs

@@ -610,7 +610,8 @@ enum Source<'a> {
     Duck(&'a duckdb::Connection),
     Sqlite(&'a Sqlite),
     Replay(&'a Replay<'a>),
-    Memory(&'a Memory),
+    /// The non-SQL provider, through the composition root given.
+    Memory(&'a Memory, &'a CompositionRoot),
 }
 
 impl Source<'_> {
@@ -627,18 +628,18 @@ impl Source<'_> {
                 dialect: Dialect::new(Database::Postgres.name()),
             },
             // The non-SQL provider, a host provider, as it declares itself.
-            Source::Memory(_) => ProviderChoice::Host {
+            Source::Memory(..) => ProviderChoice::Host {
                 capabilities: Memory::capabilities(),
             },
         }
     }
 
-    /// The composition root a run over this source starts with: the
-    /// second data layer, for host providers alone, for the non-SQL
-    /// provider ([`crate::layer`], T2.7); `sdm-data`'s for every other.
+    /// The composition root a run over this source starts with: the one
+    /// the non-SQL provider's run was given (T2.7); `sdm-data`'s for every
+    /// other.
     fn root(self) -> CompositionRoot {
         match self {
-            Source::Memory(_) => crate::layer::root(),
+            Source::Memory(_, root) => root.clone(),
             _ => default_root(),
         }
     }
@@ -734,12 +735,13 @@ pub fn run_replayed(case: &Case, recording: &Recording) -> Result<Observed> {
 }
 
 /// Runs `case` as [`run`] runs it on DuckDB, over its DuckDB base copied
-/// into the minimal non-SQL test provider ([`crate::memory`], T2.7), and
-/// through the second data layer swapped in at the composition root
-/// ([`crate::layer`]): every operation the case names, its fetches
-/// answered from in-memory Arrow tables, never by a statement. Its run
-/// records no driver: nothing runs SQL.
-pub fn run_memory(case: &Case, bases: &Bases) -> Result<Observed> {
+/// into the minimal non-SQL test provider ([`crate::memory`], T2.7), every
+/// run started with `root`: the second data layer's
+/// ([`crate::layer::root`]) to run it swapped, or `default_root()`, over
+/// `sdm-data`, to run the same provider on the default layer. Every
+/// operation the case names, its fetches answered from in-memory Arrow
+/// tables, never by a statement; its run records no driver.
+pub fn run_memory(case: &Case, bases: &Bases, root: &CompositionRoot) -> Result<Observed> {
     let Some(spec) = case.core.as_ref().and_then(|c| c.describe_source.as_ref()) else {
         bail!("{}: reads no data (names no describe_source)", case.id);
     };
@@ -756,7 +758,7 @@ pub fn run_memory(case: &Case, bases: &Bases) -> Result<Observed> {
     describe_on(
         case,
         spec,
-        Source::Memory(&memory),
+        Source::Memory(&memory, root),
         &mut DriverLog::default(),
         &edits,
     )
@@ -876,7 +878,7 @@ fn drive(
                             log_replayed(log, &replayed);
                             replayed.answer
                         }
-                        Source::Memory(memory) => {
+                        Source::Memory(memory, _) => {
                             let mut sets = BTreeMap::new();
                             for (id, fetched) in &keys {
                                 sets.insert(

rust/crates/sdm-testkit/tests/memory_provider.rs

@@ -22,7 +22,7 @@ use sdm_contract::data::{DataLayer, Dialect, ProviderChoice};
 use sdm_testkit::bases;
 use sdm_testkit::case::{self, Case};
 use sdm_testkit::host::{self, Answer, Observed};
-use sdm_testkit::layer::HostLayer;
+use sdm_testkit::layer::{self, HostLayer};
 use serde_json::{Value, json};
 
 fn repository() -> PathBuf {
@@ -63,7 +63,7 @@ fn a_prediction_is_duckdbs_but_for_its_drawn_examples() {
     for id in ["base.binary", "shape.same-table-twice.buyers"] {
         let given = case(id);
         let on_duckdb = host::run(&given, &bases).unwrap();
-        let swapped = host::run_memory(&given, &bases).unwrap();
+        let swapped = host::run_memory(&given, &bases, &layer::root()).unwrap();
         assert!(swapped.driver.is_none(), "{id}: the swapped run ran SQL");
         assert!(
             swapped
@@ -96,7 +96,7 @@ fn two_schema_tables_over_one_source_predict_as_two_sources() {
     // the same provider gives over the two tables.
     let bases = bases();
     let given = case("shape.same-table-twice.buyers");
-    let two_sources = host::run_memory(&given, &bases).unwrap();
+    let two_sources = host::run_memory(&given, &bases, &layer::root()).unwrap();
     let mut one_source = given.clone();
     one_source
         .input
@@ -113,7 +113,7 @@ fn two_schema_tables_over_one_source_predict_as_two_sources() {
             table["source"] = json!({"source": ["customers"]});
         }
     }
-    let swapped = host::run_memory(&one_source, &bases).unwrap();
+    let swapped = host::run_memory(&one_source, &bases, &layer::root()).unwrap();
     let read: Vec<&Value> = swapped
         .fetches
         .iter()