feat(onnx): load-time warning when the model keys on integration-context fields - #103
Merged
Conversation
…ext fields New context-sensitivity probe alongside the calibration probe: score the same plausible transaction with and without the optional integration-context fields (is_authenticated, device_is_trusted, channel, currency, session length). A gap above 0.5 logs a loud warning and the gap is exposed via getModelInfo() — warning only, never a readiness failure. Against the shipped model the probe measures bare=0.9998 vs full=0.0000 (gap 0.9998): the degeneracy from efficacy-validation finding F3 is now self-detecting on every model load and hot-swap. Found by efficacy-validation (finding F3, PR #99).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to finding F3 from the efficacy validation (#99) — the detection half; the retrain that actually fixes the model is a separate piece of work.
What
A
contextSensitivityProberuns next to the existing calibration probe on every model load and registry hot-swap. It scores the same plausible transaction twice — once with the optional integration-context fields populated (is_authenticated,device_is_trusted,channel_code,currency_code,session_to_txn_seconds), once with them absent, exactly what an integrator sending only the six required fields produces. If the score gap exceeds 0.5 it logs a loud warning with remediation guidance; the measured gap is exposed viagetModelInfo().Warning only — never a readiness failure. The model still functions; the operator needs to know its scores track payload richness rather than behaviour.
Why
#99 measured the shipped model as a trust-context detector: 687/687 bare-payload transactions scored >0.65 while 770/770 full-context transactions scored ≤0.65 — mule networks, trusted-device velocity bursts, and rings all invisible. Nothing in the stack could see this failure mode. Now every load self-checks.
Verification
Run against the shipped
fraud_model.onnx:Lint clean, existing onnx suite green.
🤖 Generated with Claude Code