Skip to content

Commit 3c4d7e7

Browse files
mstrathmanclaude
andcommitted
feat(onnx): derive io_spec from the model — register with just a path
Cuts the ONNX registration UX from ~10 lines of nested JSON to one line. predict_register now reads a model's input/output tensors and fills in the io_spec automatically, so the common case is: SELECT predict_register('clf', '/models/clf.onnx'); SELECT * FROM predict(NULL, 'SELECT id, f1, f2 FROM t', '{"model":"clf"}'); - predict0_onnx_introspect() opens the model and derives layout (1 input -> vector, 3 named x_train/y_train/x_query -> in_context), tensor names, output kind (float [N,K>1] -> probs, [N,1]/[N] -> value), and class count (labels default 0..K-1). It builds the io_spec via json_object, and refuses to guess ambiguous shapes (multi-output, non-float, unnamed 3-input) by asking for an explicit io_spec. - predict_register accepts a bare weights path or a JSON config; runtime ('onnx'), kind ('student'), and license ('unspecified', which the gate allows) all default. in-context adds a top-level target (a SQL column introspection can't infer), spliced into the derived io_spec via json_set. - features[] is now optional everywhere: absent -> positional mapping (apply column order), validated against the model's input width; present -> name-based mapping (the override). The tensor feature dim F is computed per call instead of assumed == named-feature count. - The explicit-io_spec path is unchanged (all prior tests pass), and the ASan/LSan soak now also exercises the introspection + positional path. 5 new tests cover bare-path vector/regressor registration, in-context weights+target, the unspecified-license default, and positional count validation. README/ARCHITECTURE/CHANGELOG updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: mstrathman <matthew.strathman@gmail.com>
1 parent 6f82fbf commit 3c4d7e7

9 files changed

Lines changed: 8480 additions & 133 deletions

‎ARCHITECTURE.md‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,18 @@ serves two `io_spec` layouts. The **vector** layout is a self-contained model
4848
mapping a feature vector to a prediction. The **in_context** layout is a
4949
teacher (TabFM-shaped): it ingests the `train_query` rows as three tensors
5050
(`x_train`, `y_train`, `x_query`) each call and labels the query rows against
51-
that context. Both cache one onnxruntime session per (weights, device,
51+
that context.
52+
53+
The `io_spec` is usually derived, not written. `predict_register` reads the
54+
model's input/output tensors (`predict0_onnx_introspect`) to fill in the
55+
layout, tensor names, output kind, and class count, so a bare weights path is
56+
a complete registration for the vector case (in-context adds only `target`,
57+
which is a SQL column introspection can't see). An explicit `io_spec`
58+
overrides the derivation. Feature columns map positionally by default (apply
59+
column order); a `features` list switches to name-based mapping. All the JSON
60+
handling — reading the `io_spec`, building the derived one — goes through
61+
SQLite's JSON1 (`json_extract`/`json_each`/`json_object`), never a hand-rolled
62+
parser. Both cache one onnxruntime session per (weights, device,
5263
precision), run query rows in batches, and select the execution provider
5364
explicitly, erasing no failure into a silent CPU fallback. Weights are pinned
5465
by content hash, so a receipt records exactly which bytes ran, and the

‎CHANGELOG.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,12 @@ project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
2525
(`make loadable-onnx-gpu`) adds the CUDA and TensorRT execution providers
2626
and fp16/int8 precision, compile-checked in CI and validated on a
2727
dedicated GPU job. The default build stays zero-dependency.
28-
- `predict_register(model_id, config_json)` to register an external model,
28+
- `predict_register(model_id, config)` to register an external model,
2929
pinning its weights by content hash. `_predict_models` gains `weights_uri`
30-
and `io_spec`; receipts record the execution provider and precision.
30+
and `io_spec`; receipts record the execution provider and precision. The
31+
config can be a bare weights path: the io_spec is read off the model
32+
(tensor names, output kind, class count) and feature columns map by
33+
position, so the common case is one line. An explicit io_spec overrides.
3134
- A license gate: a non-permissive model requires `accept_license` to match
3235
before it will run.
3336
- Test suite, AddressSanitizer/UBSan and valgrind soak targets, a libFuzzer

‎README.md‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,14 +81,29 @@ which compresses a teacher into a compact model that serves in
8181
microseconds.
8282

8383
An opt-in ONNX build (`make loadable-onnx`) runs exported models through
84-
onnxruntime: register one with `predict_register()` and call it by name. It
85-
serves two shapes, with a cached session and batched inference:
84+
onnxruntime. Point `predict_register()` at a model file and call it by name;
85+
the io_spec is read off the model (input/output tensors, output kind, class
86+
count), so the common case is one line:
87+
88+
```sql
89+
SELECT predict_register('churn', '/models/churn.onnx');
90+
SELECT * FROM predict(NULL, 'SELECT id, tenure, spend FROM customers',
91+
'{"model":"churn"}');
92+
```
93+
94+
Feature columns map by position (apply-query order); pass an explicit
95+
`io_spec` only to override — real class labels, a named-feature mapping, or a
96+
model whose tensors introspection can't disambiguate. It serves two shapes,
97+
with a cached session and batched inference:
8698

8799
- **vector** — a self-contained model (a distilled student, or any exported
88100
tabular classifier/regressor) mapping a feature vector to a prediction.
101+
A bare weights path is enough.
89102
- **in_context** — a teacher that ingests the `train_query` rows as context
90-
on each call and labels the `apply_query` rows against them, the way
91-
TabFM works.
103+
on each call and labels the `apply_query` rows against them, the way TabFM
104+
works. Register it with the weights path plus a `target` (the training
105+
label column, which introspection can't infer): `predict_register('t',
106+
'{"weights_uri":"/m.onnx","target":"label"}')`.
92107

93108
Both run on CPU today. A GPU build (`make loadable-onnx-gpu`) adds the CUDA
94109
and TensorRT execution providers and fp16/int8 precision; it needs an

‎onnx_export_2026-07-22_17-08-29-313987_conversion.md‎

Lines changed: 7933 additions & 0 deletions
Large diffs are not rendered by default.

‎predict-internal.h‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,16 @@ int predict0_onnx_predict(sqlite3 *db, const char *model_id,
197197
predict0_result **rows, int *n,
198198
char receipt_id_out[PREDICT_ULID_BUFSIZE],
199199
char **errmsg);
200+
201+
/* Derive an io_spec by reading a model's input/output tensors, so a caller
202+
* can register with just a weights path. On success returns SQLITE_OK and a
203+
* sqlite3_malloc'd JSON io_spec in *io_spec_out (caller frees). Covers the
204+
* unambiguous shapes (1 input -> vector; 3 inputs named x_train/y_train/
205+
* x_query -> in_context; a single output whose width picks probs vs value);
206+
* anything it cannot disambiguate returns an error asking for an explicit
207+
* io_spec. */
208+
int predict0_onnx_introspect(sqlite3 *db, const char *weights_uri,
209+
char **io_spec_out, char **errmsg);
200210
#endif
201211

202212
/* Deterministic logical digest of all user tables (schema + rows,

0 commit comments

Comments
 (0)