Send a relational inference config only when the caller set one (#16)

## Problem

structured-data-nims !189 and !197 make the SDM backends (the Kumo
Relational and Kumo Tabular NIMs, and the Databricks MLflow model that
reuses them) refuse a non-empty `inference.inference_config` with `400
INVALID_REQUEST`. The estimator count is now fixed when the endpoint
starts, through `SDM_NUM_ESTIMATORS`.

This client sent the engine's default config on every KumoRelational
request (`{kind, num_estimators: 1, column_shuffle: false, ...}`),
because `RelationalAdapter._inference` forwarded every key whether or
not the caller had set it. So every relational request failed on those
backends, although none asked for anything beyond the defaults.

## Fix

`RelationalAdapter._inference` now forwards only the inference-config
keys the caller set and omits `inference_config` when none are left.
`run_mode`, `use_prediction_time` and `output.prediction_statistic` are
unchanged, and sessions use the same path. An option the caller does
set, such as `num_estimators: 4`, is still sent on its own; a legacy
backend honours it and an SDM backend refuses it.

This is safe for the legacy kumo-ml backend: it reads `inference_config
or {}` and applies the rest on top of
`InferenceConfig.from_task_type(...)`, whose defaults are identical to
the engine's (one estimator, shuffles off, and `quantile` transforms
with a `median` output for regression). A request without the field gets
exactly what this client used to send.

The two Databricks relational notebooks stop passing `ENSEMBLE =
{'num_estimators': 4, 'column_shuffle': True}`, which SDM backends
refuse; there the ensemble size is set per endpoint through
`SDM_NUM_ESTIMATORS`. `tests/golden/wire_payloads.json` is regenerated
with `scripts/snapshot_wire_payloads.py --write`, and it only loses the
default config from its one relational case.

## Verification

Updated adapter, session and median tests cover a default request
sending no config (through both transports), the engine's full defaults
not being sent, and an explicit option being sent alone. They fail on
main. The client suite passes (1913 passed), along with ruff, mypy and
the golden snapshot check.

End to end, an AdventureWorks `PREDICT SUM(sales_order_details.OrderQty,
0, 30, days) FOR EACH products.ProductID` for product 712 through
`SDMClient.for_databricks_serving` against the SDM staging endpoint
`kumo-rfm-2-1-stg-v23-mr149-review` returns a prediction (120.97) with
no `inference_config` in the request body, while the same request from
main gets `400 INVALID_REQUEST`.

---------

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>

Author: adi-kumo

CHANGELOG.md

@@ -5,6 +5,14 @@ versioned by [semantic versioning](https://semver.org) as
 [`docs/reference/stability.md`](docs/reference/stability.md) defines it for
 these packages. All five distributions share one version and one entry.
 
+## Unreleased
+
+### Fixed
+
+- A `KumoRelational` request sends `inference.inference_config` only with the
+  options the caller set, and omits it otherwise. Sending the engine's defaults
+  on every request made SDM-backed deployments refuse every relational request.
+
 ## 1.0.0 - 2026-10-06
 
 The first release. All five distributions declare `1.0.0` and are published

examples/databricks/adventureworks_relational.ipynb

@@ -299,9 +299,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Four estimators over shuffled columns: one estimator's answer moves with\n",
-    "# column order, and averaging over four permutations settles it.\n",
-    "ENSEMBLE = {'num_estimators': 4, 'column_shuffle': True}\n",
     "DEMAND = (\n",
     "    'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) '\n",
     "    'FOR EACH products.ProductID'\n",
@@ -310,24 +307,16 @@
     "QUESTIONS = {\n",
     "    'demand_30d': (\n",
     "        DEMAND,\n",
-    "        {'indices': [top_product], 'inference_config': ENSEMBLE},\n",
+    "        {'indices': [top_product]},\n",
     "    ),\n",
     "    'demand_30d_historical': (\n",
     "        DEMAND,\n",
-    "        {\n",
-    "            'indices': [top_product],\n",
-    "            'anchor_time': anchor,\n",
-    "            'inference_config': ENSEMBLE,\n",
-    "        },\n",
+    "        {'indices': [top_product], 'anchor_time': anchor},\n",
     "    ),\n",
     "    'demand_3_timeframes': (\n",
     "        'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) '\n",
     "        'FORECAST 3 TIMEFRAMES FOR EACH products.ProductID',\n",
-    "        {\n",
-    "            'indices': [top_product],\n",
-    "            'anchor_time': anchor,\n",
-    "            'inference_config': ENSEMBLE,\n",
-    "        },\n",
+    "        {'indices': [top_product], 'anchor_time': anchor},\n",
     "    ),\n",
     "    'inactivity_90d': (\n",
     "        'PREDICT COUNT(sales_order_headers.*, 0, 90, days)=0 '\n",
@@ -431,7 +420,7 @@
     "    lines = 'adventureworks_sales_order_details'\n",
     "    QUESTIONS['view_demand_30d'] = (\n",
     "        f'PREDICT SUM({lines}.OrderQty, 0, 30, days) FOR EACH products.ProductID',\n",
-    "        {'indices': [top_product], 'inference_config': ENSEMBLE},\n",
+    "        {'indices': [top_product]},\n",
     "    )\n",
     "    QUESTIONS['view_inactivity_90d'] = (\n",
     "        f'PREDICT COUNT({lines}.*, 0, 90, days)=0 FOR EACH customers.CustomerID',\n",

examples/databricks/lakebase_relational.ipynb

@@ -278,9 +278,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Four estimators over shuffled columns: one estimator's answer moves with\n",
-    "# column order, and averaging over four permutations settles it.\n",
-    "ENSEMBLE = {'num_estimators': 4, 'column_shuffle': True}\n",
     "DEMAND = (\n",
     "    'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) '\n",
     "    'FOR EACH products.ProductID'\n",
@@ -289,15 +286,11 @@
     "QUESTIONS = {\n",
     "    'demand_30d': (\n",
     "        DEMAND,\n",
-    "        {'indices': [top_product], 'inference_config': ENSEMBLE},\n",
+    "        {'indices': [top_product]},\n",
     "    ),\n",
     "    'demand_30d_historical': (\n",
     "        DEMAND,\n",
-    "        {\n",
-    "            'indices': [top_product],\n",
-    "            'anchor_time': anchor,\n",
-    "            'inference_config': ENSEMBLE,\n",
-    "        },\n",
+    "        {'indices': [top_product], 'anchor_time': anchor},\n",
     "    ),\n",
     "    'demand_3_timeframes': (\n",
     "        'PREDICT SUM(sales_order_details.OrderQty, 0, 30, days) '\n",

packages/structured-data-models-client/src/sdm_client/adapters/relational.py

@@ -140,7 +140,7 @@ def prediction_plan(
     def _inference(
         self, inference: Any, request: Request
     ) -> dict[str, Any] | None:
-        """Send only the inference options this model reads.
+        """Send only the inference options the caller set and this model reads.
 
         The engine fills in an inference config on every request, including
         keys this client does not forward: `output_type`, whose choice travels
@@ -180,10 +180,15 @@ def _inference(
                 },
             )
 
+        # Only what the caller set: a deployment applies the engine's defaults
+        # to a request that carries none, and the SDM backends refuse any
+        # non-empty `inference_config`, so sending the defaults broke them.
         trimmed = dict(inference)
-        trimmed['inference_config'] = {
-            key: value for key, value in config.items() if key in forwarded
-        }
+        sent = {key: value for key, value in config.items() if key in asked_for}
+        if sent:
+            trimmed['inference_config'] = sent
+        else:
+            del trimmed['inference_config']
         return trimmed
 
     def session_plan(

packages/structured-data-models-client/src/sdm_client/materializers/relational.py

@@ -6,6 +6,7 @@
 
 from __future__ import annotations
 
+import copy
 import warnings
 from collections.abc import Iterator, Mapping, Sequence
 from contextlib import contextmanager
@@ -162,8 +163,13 @@ def materialize(
             query=query.to_string(),
             indices=list(indices),
             entity_table=table_name(query.entity_column),
+            # A copy: the payloads were gathered from the config as it is
+            # now, and a caller's later edit to it must not change what the
+            # adapter believes was asked for.
             requested_inference={
-                'inference_config': options.get('inference_config') or {}
+                'inference_config': copy.deepcopy(
+                    options.get('inference_config') or {}
+                )
             },
         )
 

packages/structured-data-models-client/tests/test_a_regression_reports_the_median.py

@@ -125,12 +125,34 @@ def test_a_regression_asks_for_the_median_by_default(requests_mock):
 
     assert body['task']['kind'] == 'regression'
     assert body['output']['prediction_statistic'] == 'median'
-    assert 'output_type' not in body['inference']['inference_config']
+    assert 'inference_config' not in body['inference']
     # Both transports wrap the same payload, so both ask for the same thing.
     (wrapped,) = sent_to_databricks(request)
     assert without_request_id(wrapped) == without_request_id(body)
 
 
+def test_only_the_inference_option_the_caller_set_is_sent(requests_mock):
+    request = asked(inference_config={'num_estimators': 4})
+    (body,) = sent_to_a_nim(request, requests_mock)
+    assert body['inference']['inference_config'] == {'num_estimators': 4}
+
+
+@pytest.mark.parametrize(
+    'edit', [dict.clear, lambda config: config.update(num_estimators=2)]
+)
+def test_editing_the_config_after_materialize_changes_nothing_sent(
+    edit, requests_mock
+):
+    config = {'num_estimators': 4}
+    request = asked(inference_config=config)
+    edit(config)
+    (body,) = sent_to_a_nim(request, requests_mock)
+    assert body['inference']['inference_config'] == {'num_estimators': 4}
+    assert request.requested_inference == {
+        'inference_config': {'num_estimators': 4}
+    }
+
+
 def test_the_mean_is_sent_when_it_is_asked_for(requests_mock):
     (body,) = sent_to_a_nim(asked(prediction_statistic='mean'), requests_mock)
     assert body['output']['prediction_statistic'] == 'mean'

packages/structured-data-models-client/tests/test_adapter_and_result.py

@@ -325,7 +325,23 @@ def test_a_defaulted_dataclass_field_is_dropped_without_a_refusal():
         regression_config(),
         sent={'num_estimators': 1, 'output_type': 'median'},
     )
-    assert sent_inference(request)['inference_config'] == {'num_estimators': 1}
+    assert 'inference_config' not in sent_inference(request)
+
+
+def test_the_engine_s_defaults_are_not_sent():
+    """A deployment applies them to a request that carries none, and the
+    SDM backends refuse a non-empty inference_config."""
+    defaults = {
+        'kind': 'classification',
+        'num_estimators': 1,
+        'column_shuffle': False,
+        'category_shuffle': False,
+        'hop_shuffle': False,
+        'class_shuffle': False,
+    }
+    assert sent_inference(gathered_with({}, sent=defaults)) == {
+        'run_mode': 'fast'
+    }
 
 
 def test_a_dropped_option_is_refused_with_a_remedy_that_works():

packages/structured-data-models-client/tests/test_decomposition_is_faithful.py

@@ -253,11 +253,12 @@ def test_each_step_moves_the_window_by_its_own_span():
 def test_a_forecast_step_is_the_window_question_asked_on_its_own():
     """Step one of a forecast is the plain question, and so reads against it.
 
-    The AdventureWorks notebook asks a 30 day demand question with an ensemble
-    and the same question forecast three windows out. Its first step can only
-    be read against the plain answer if the two requests are the same, so the
-    ensemble has to reach every step and nothing else may differ. Each later
-    step moves the window along and changes nothing else.
+    The AdventureWorks notebook asks a 30 day demand question and the same
+    question forecast three windows out. Its first step can only be read
+    against the plain answer if the two requests are the same, so an inference
+    config the caller set, here an ensemble, has to reach every step and
+    nothing else may differ. Each later step moves the window along and
+    changes nothing else.
     """
     pytest.importorskip('sdm_relational_engine')
     graph = shop()

packages/structured-data-models-client/tests/test_limits_and_sessions.py

@@ -215,7 +215,7 @@ def test_the_engine_s_statistic_travels_on_the_output_instead():
         }
     }
     (plan,) = RelationalAdapter().prediction_plan(request, ('prediction',))
-    assert plan['inference']['inference_config'] == {'num_estimators': 1}
+    assert 'inference_config' not in plan['inference']
     assert plan['output']['prediction_statistic'] == 'median'
 
 

tests/golden/wire_payloads.json

@@ -7013,13 +7013,6 @@
         }
       },
       "inference": {
-        "inference_config": {
-          "category_shuffle": false,
-          "column_shuffle": false,
-          "hop_shuffle": false,
-          "kind": "regression",
-          "num_estimators": 1
-        },
         "run_mode": "fast",
         "use_prediction_time": false
       },