Skip to content

Commit 539066a

Browse files
mstrathmanclaude
andcommitted
bench(tabular): soft-label distillation results + lock the blob format
Full 51-dataset re-run adding gbt<-tabfm soft (TabFM's predict_proba distilled via proba/classes). Result is modest but exactly what the mechanism predicts: soft vs hard is 16W/11T/8L (mean +0.003) -- most datasets tie because the teacher was already confident -- but soft gbt<-tabfm beats the knn5-taught student on 74% (up from hard's 65%), and on 6 datasets where hard-label distillation had lost to the cheap knn5 teacher, soft caught back up or passed it. Soft labels stop discarding TabFM's calibrated edge; what they can't fix is the representational half of the gap (axis-aligned trees vs a warped boundary), which is the next student, not the next target. tabarena-full.md updated with the soft column, head-to-head, and analysis. ARCHITECTURE.md: point at the now-normative, versioned blob format (RFC 4.1.6, PSTREE01/PSGBT01) -- a stored student stays servable across upgrades, snapshots, and forks, and a serving-only module can execute a blob it never trained. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: mstrathman <matthew.strathman@gmail.com>
1 parent 8b365b8 commit 539066a

3 files changed

Lines changed: 140 additions & 111 deletions

File tree

‎ARCHITECTURE.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,8 +103,11 @@ inherits a foundation model's calibration instead of only its argmax, while
103103
the holdout is still scored against the true `target` labels. `predict()` dispatches
104104
a `tree`-runtime model to the native runtime in the same file, which tells a
105105
single tree (`PSTREE` blob) from a forest (`PSGBT` blob) by magic and needs
106-
no onnxruntime. Both blob formats are little-endian and implementation-defined
107-
(RFC §4.2.4), and rigorously bounds-checked on read, because the registry is
106+
no onnxruntime. Both blob formats are little-endian and normatively specified
107+
and versioned (RFC §4.1.6, `PSTREE01` / `PSGBT01`), so a stored student stays
108+
servable across upgrades, snapshots, and forks, and a serving-only module can
109+
execute a blob it never trained. They are rigorously bounds-checked on read,
110+
because the registry is
108111
writable by any SQL caller (RFC §6.2) — a hand-crafted blob is rejected,
109112
never crashed on. Because the student is native and deterministic, its
110113
predictions carry the same exact-replay receipt as the stat models.

0 commit comments

Comments
 (0)