From 5d8aa041648ff5e5f52fa1a441bdd6dbf455fe9b Mon Sep 17 00:00:00 2001 From: jsboige Date: Thu, 3 Sep 2026 00:11:55 +0200 Subject: [PATCH] fix(training,#14362): DM sanity leg was silently centered, reproducing the verdict leg `run_btc_debiased_recentered` computes two DM legs: a verdict leg on centered errors vs de-biased HAR, and a sanity leg meant to reproduce the #11011 keeper on raw losses vs raw HAR. Both went through `_dm_centered_mse`. The two HAR series differ by exactly a constant (`har_errors_debiased = har_errors - har_bias_oos`), and centering by each series' own mean annihilates precisely that constant: `(e - c) - mean(e - c) = e - mean(e)`. The legs returned the same number -- bit-identical on 12 of 12 rows of the published artefact. A control that cannot go red is not a control. The artefact contradicted its own stated purpose: the sanity leg's p_medians (1.545e-09 / 9.660e-05 / 1.021e-01) reproduced `dm_centered_p_median`, not the keeper (0.00e+00 / 2.25e-10 / 2.39e-09) -- and at h=10 it reported INCONCLUSIVE where the keeper it was meant to retrieve publishes BEATS. Delivered: - `_dm_uncentered_mse`: DM with loss_fn="mse" on untouched errors, same return shape and sentinels as its twin. The sanity leg now calls it on raw series. - Rename `dm_raw_*` -> `dm_uncentered_vs_har_raw_*` (names both properties; zero consumers repo-wide, verified by grep). `dm_centered_*` left untouched -- that one IS consumed by committed notebook cells. - 7 tests, including the sealed control and its falsification (`test_old_composition_is_degenerate` proves the old path collapses, so the main assertion is not incidental). Verified by mutation: reintroducing the regression fires 3 of 3 guards. Full suite: 1089 passed, 1 skipped. - Artefact regenerated at identical config. `dm_centered_stat` returns bit-identical on 12/12 rows (max delta 0.000e+00); the corrected leg now matches the keeper in magnitude and verdict, recovering BEATS at h=10 (p = 3.08e-09 vs keeper 2.39e-09). Legs now coincide on 0/12 rows. - Notebook re-executed end-to-end (C.2: cell 24 prints `elapsed_s`), which also closes a pre-existing machine-path leak in its papermill metadata. - REGISTRY.md correction note; the entry described this leg as reproducing the keeper, and it did not. The verdict is unaffected: `aggregate_verdicts_recentered` reads only `dm_centered_*`. M4 stays confirmed h=1/h=5 + INCONCLUSIVE h=10; M15 stays refuted-de-biased 3/3. The defect cost the ability to check that, not the result itself. Closes #14362 See #14390 Co-Authored-By: Claude-Code --- .../ML-Training-Pipeline/REGISTRY.md | 32 +- .../m4_dlinear_vol_sc_validation.ipynb | 334 +++++++++--------- .../ML-Training-Pipeline/scripts/btc_vol.py | 52 ++- ...linear_vol_btc_sc_debiased_recentered.json | 86 +++-- .../scripts/tests/test_btc_vol.py | 109 +++++- 5 files changed, 407 insertions(+), 206 deletions(-) diff --git a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/REGISTRY.md b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/REGISTRY.md index 0f2520fbff..bbdb10e093 100644 --- a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/REGISTRY.md +++ b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/REGISTRY.md @@ -171,10 +171,40 @@ est **2/3 BEATS, 1/3 INCONCLUSIVE, 0/3 NO BEATS** — mesurable, défendable, et que les 3/3 BEATS de la mse asymétrique. M4 BTC keeper reste défendable : 2 horizons passent la conjonction, le 3ᵉ est statistiquement insuffisant (pas un échec). -- **Run** : `python scripts/btc_vol.py --horizons 1 5 10 --seeds 0 7 42 99 --epochs 50 --out-json scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json` (1786,9 s ≈ 30 min CPU, 12 combos) +- **Run** : `python scripts/btc_vol.py --horizons 1 5 10 --seeds 0 7 42 99 --epochs 50 --out-json scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json` (1776,4 s ≈ 30 min CPU, 12 combos — artefact régénéré par #14362, cf. correction ci-dessous ; le run initial #12734 mesurait 1786,9 s) - **Notebook** : section 9 de `m4_dlinear_vol_sc_validation.ipynb` (recalcul indépendant de la conjonction recentrée, décomposition biais²+variance, outputs C.2) - **Verdict §C recentré** : **2/3 BEATS, 0/3 NO BEATS, 1/3 INCONCLUSIVE** (vs 3/3 BEATS mse asymétrique #11011 — la symétrie du dé-biaisage **réduit** l'edge sur h longs, ne le fabrique pas) +**Correction #14362 (2026-09-02) — la jambe de sanité était silencieusement recentrée.** L'entrée +ci-dessus décrit une seconde jambe DM « RAW » censée reproduire le keeper #11011 sur pertes +brutes. Elle passait par `_dm_centered_mse`, comme la jambe de verdict. Or les deux séries HAR ne +diffèrent que d'une **constante** (`har_errors_debiased = har_errors − har_bias_oos`) et le +recentrage par la moyenne propre l'annule exactement — les deux jambes rendaient le **même +nombre**, bit-identiques sur **12 lignes sur 12** de l'artefact publié. Ses p-médianes +(1,545e-09 / 9,660e-05 / 1,021e-01) reproduisaient `dm_centered_p_median`, pas le keeper ; à h=10 +elle affichait `INCONCLUSIVE` là où le keeper qu'elle devait retrouver publie **BEATS** à +p = 2,39e-09. Un contrôle qui suit la mauvaise cible n'en est pas un. + +Après correction (champ renommé `dm_uncentered_vs_har_raw_*`, DM **non centré** sur erreurs +intactes), artefact régénéré à configuration identique — la jambe **retrouve** le keeper : + +| h | jambe corrigée p_median | keeper #11011 publié | verdicts | +|---|---|---|---| +| 1 | **0,000e+00** | 0,00e+00 | BEATS 4/4 | +| 5 | 1,435e-10 | 2,25e-10 | BEATS 4/4 | +| 10 | 3,083e-09 | 2,39e-09 | **BEATS 4/4** (était INCONCLUSIVE) | + +L'accord est **de rang de grandeur et de verdict, pas bit-à-bit** sur h=5/h=10 (facteur < 2) : le +keeper provient d'un entraînement DLinear distinct, les erreurs du modèle ne sont pas les mêmes +séries. C'est l'accord attendu d'une reproduction indépendante, pas d'un rejeu. + +**Ce que la correction ne change pas** : `dm_centered_stat` revient **bit-identique sur 12/12 +lignes** (Δ max = 0,000e+00) sur le run régénéré — le défaut portait sur un champ de **diagnostic**, +jamais sur la conjonction §C. Le **verdict §C recentré reste 2/3 BEATS, 1/3 INCONCLUSIVE**, et M15 +reste `refuted-de-biased` 3/3. Ce qui était perdu, c'est la capacité de le contrôler. Le champ n'est +agrégé nulle part (`aggregated` ne porte que `dm_centered_p_median`) — c'est ce qui a laissé le +défaut invisible ; suivi en **#14390**. + **Confirmation indépendante #11036 (re-validation ai-01, lane #1454, 2026-08-24)** : re-validation sur les séries du run mse #11011 persistées par combo (M4 déterministe, CPU) — MSE/DM **bit-identiques au keeper sur les 12 combos** (moyennes 0,7518/0,3740/0,3521 = valeurs publiées). diff --git a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/m4_dlinear_vol_sc_validation.ipynb b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/m4_dlinear_vol_sc_validation.ipynb index dc586db1fd..2dc4e6630b 100644 --- a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/m4_dlinear_vol_sc_validation.ipynb +++ b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/m4_dlinear_vol_sc_validation.ipynb @@ -5,10 +5,10 @@ "id": "1260e771", "metadata": { "papermill": { - "duration": 0.002919, - "end_time": "2026-08-14T15:16:33.270612+00:00", + "duration": 0.004166, + "end_time": "2026-09-02T22:07:11.183072+00:00", "exception": false, - "start_time": "2026-08-14T15:16:33.267693+00:00", + "start_time": "2026-09-02T22:07:11.178906+00:00", "status": "completed" }, "tags": [] @@ -36,10 +36,10 @@ "id": "4f61e33f", "metadata": { "papermill": { - "duration": 0.002304, - "end_time": "2026-08-14T15:16:33.276052+00:00", + "duration": 0.002708, + "end_time": "2026-09-02T22:07:11.188967+00:00", "exception": false, - "start_time": "2026-08-14T15:16:33.273748+00:00", + "start_time": "2026-09-02T22:07:11.186259+00:00", "status": "completed" }, "tags": [] @@ -65,16 +65,16 @@ "id": "ae5fe2bd", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:33.282577Z", - "iopub.status.busy": "2026-08-14T15:16:33.282322Z", - "iopub.status.idle": "2026-08-14T15:16:34.064753Z", - "shell.execute_reply": "2026-08-14T15:16:34.063659Z" + "iopub.execute_input": "2026-09-02T22:07:11.195744Z", + "iopub.status.busy": "2026-09-02T22:07:11.195534Z", + "iopub.status.idle": "2026-09-02T22:07:11.591917Z", + "shell.execute_reply": "2026-09-02T22:07:11.591270Z" }, "papermill": { - "duration": 0.787111, - "end_time": "2026-08-14T15:16:34.065473+00:00", + "duration": 0.400701, + "end_time": "2026-09-02T22:07:11.592492+00:00", "exception": false, - "start_time": "2026-08-14T15:16:33.278362+00:00", + "start_time": "2026-09-02T22:07:11.191791+00:00", "status": "completed" }, "tags": [] @@ -136,10 +136,10 @@ "id": "2c88966b", "metadata": { "papermill": { - "duration": 0.002253, - "end_time": "2026-08-14T15:16:34.070107+00:00", + "duration": 0.002797, + "end_time": "2026-09-02T22:07:11.598412+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.067854+00:00", + "start_time": "2026-09-02T22:07:11.595615+00:00", "status": "completed" }, "tags": [] @@ -159,16 +159,16 @@ "id": "6897067c", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:34.076121Z", - "iopub.status.busy": "2026-08-14T15:16:34.075778Z", - "iopub.status.idle": "2026-08-14T15:16:34.094388Z", - "shell.execute_reply": "2026-08-14T15:16:34.093653Z" + "iopub.execute_input": "2026-09-02T22:07:11.607727Z", + "iopub.status.busy": "2026-09-02T22:07:11.607368Z", + "iopub.status.idle": "2026-09-02T22:07:11.616934Z", + "shell.execute_reply": "2026-09-02T22:07:11.616343Z" }, "papermill": { - "duration": 0.022671, - "end_time": "2026-08-14T15:16:34.095118+00:00", + "duration": 0.016073, + "end_time": "2026-09-02T22:07:11.617528+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.072447+00:00", + "start_time": "2026-09-02T22:07:11.601455+00:00", "status": "completed" }, "tags": [] @@ -212,10 +212,10 @@ "id": "b2767d3d", "metadata": { "papermill": { - "duration": 0.002237, - "end_time": "2026-08-14T15:16:34.099901+00:00", + "duration": 0.002841, + "end_time": "2026-09-02T22:07:11.623476+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.097664+00:00", + "start_time": "2026-09-02T22:07:11.620635+00:00", "status": "completed" }, "tags": [] @@ -244,16 +244,16 @@ "id": "3f26cecc", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:34.105148Z", - "iopub.status.busy": "2026-08-14T15:16:34.104961Z", - "iopub.status.idle": "2026-08-14T15:16:34.175734Z", - "shell.execute_reply": "2026-08-14T15:16:34.175054Z" + "iopub.execute_input": "2026-09-02T22:07:11.630385Z", + "iopub.status.busy": "2026-09-02T22:07:11.630199Z", + "iopub.status.idle": "2026-09-02T22:07:11.641472Z", + "shell.execute_reply": "2026-09-02T22:07:11.640882Z" }, "papermill": { - "duration": 0.074533, - "end_time": "2026-08-14T15:16:34.176574+00:00", + "duration": 0.015615, + "end_time": "2026-09-02T22:07:11.642016+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.102041+00:00", + "start_time": "2026-09-02T22:07:11.626401+00:00", "status": "completed" }, "tags": [] @@ -316,10 +316,10 @@ "id": "a13d1e1c", "metadata": { "papermill": { - "duration": 0.002183, - "end_time": "2026-08-14T15:16:34.181086+00:00", + "duration": 0.002785, + "end_time": "2026-09-02T22:07:11.647839+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.178903+00:00", + "start_time": "2026-09-02T22:07:11.645054+00:00", "status": "completed" }, "tags": [] @@ -346,16 +346,16 @@ "id": "ddf0e564", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:34.186360Z", - "iopub.status.busy": "2026-08-14T15:16:34.186178Z", - "iopub.status.idle": "2026-08-14T15:16:39.167287Z", - "shell.execute_reply": "2026-08-14T15:16:39.166555Z" + "iopub.execute_input": "2026-09-02T22:07:11.654216Z", + "iopub.status.busy": "2026-09-02T22:07:11.654002Z", + "iopub.status.idle": "2026-09-02T22:07:14.306015Z", + "shell.execute_reply": "2026-09-02T22:07:14.305179Z" }, "papermill": { - "duration": 4.984814, - "end_time": "2026-08-14T15:16:39.168055+00:00", + "duration": 2.656011, + "end_time": "2026-09-02T22:07:14.306590+00:00", "exception": false, - "start_time": "2026-08-14T15:16:34.183241+00:00", + "start_time": "2026-09-02T22:07:11.650579+00:00", "status": "completed" }, "tags": [] @@ -412,10 +412,10 @@ "id": "ab56f7dc", "metadata": { "papermill": { - "duration": 0.002408, - "end_time": "2026-08-14T15:16:39.173273+00:00", + "duration": 0.003051, + "end_time": "2026-09-02T22:07:14.313227+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.170865+00:00", + "start_time": "2026-09-02T22:07:14.310176+00:00", "status": "completed" }, "tags": [] @@ -433,16 +433,16 @@ "id": "9ecbb28b", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:39.179690Z", - "iopub.status.busy": "2026-08-14T15:16:39.179316Z", - "iopub.status.idle": "2026-08-14T15:16:39.192898Z", - "shell.execute_reply": "2026-08-14T15:16:39.192046Z" + "iopub.execute_input": "2026-09-02T22:07:14.320230Z", + "iopub.status.busy": "2026-09-02T22:07:14.320034Z", + "iopub.status.idle": "2026-09-02T22:07:14.335168Z", + "shell.execute_reply": "2026-09-02T22:07:14.334544Z" }, "papermill": { - "duration": 0.018047, - "end_time": "2026-08-14T15:16:39.193607+00:00", + "duration": 0.019826, + "end_time": "2026-09-02T22:07:14.336057+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.175560+00:00", + "start_time": "2026-09-02T22:07:14.316231+00:00", "status": "completed" }, "tags": [] @@ -480,10 +480,10 @@ "id": "7960064e", "metadata": { "papermill": { - "duration": 0.002342, - "end_time": "2026-08-14T15:16:39.198587+00:00", + "duration": 0.002873, + "end_time": "2026-09-02T22:07:14.341941+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.196245+00:00", + "start_time": "2026-09-02T22:07:14.339068+00:00", "status": "completed" }, "tags": [] @@ -510,10 +510,10 @@ "id": "ce7a26eb", "metadata": { "papermill": { - "duration": 0.002287, - "end_time": "2026-08-14T15:16:39.203373+00:00", + "duration": 0.00274, + "end_time": "2026-09-02T22:07:14.347499+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.201086+00:00", + "start_time": "2026-09-02T22:07:14.344759+00:00", "status": "completed" }, "tags": [] @@ -544,16 +544,16 @@ "id": "087fce6d", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:39.209066Z", - "iopub.status.busy": "2026-08-14T15:16:39.208850Z", - "iopub.status.idle": "2026-08-14T15:16:39.225315Z", - "shell.execute_reply": "2026-08-14T15:16:39.224640Z" + "iopub.execute_input": "2026-09-02T22:07:14.354371Z", + "iopub.status.busy": "2026-09-02T22:07:14.354180Z", + "iopub.status.idle": "2026-09-02T22:07:14.367919Z", + "shell.execute_reply": "2026-09-02T22:07:14.367391Z" }, "papermill": { - "duration": 0.020364, - "end_time": "2026-08-14T15:16:39.225921+00:00", + "duration": 0.018317, + "end_time": "2026-09-02T22:07:14.368637+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.205557+00:00", + "start_time": "2026-09-02T22:07:14.350320+00:00", "status": "completed" }, "tags": [] @@ -621,10 +621,10 @@ "id": "3cd8656e", "metadata": { "papermill": { - "duration": 0.002298, - "end_time": "2026-08-14T15:16:39.230627+00:00", + "duration": 0.002835, + "end_time": "2026-09-02T22:07:14.374434+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.228329+00:00", + "start_time": "2026-09-02T22:07:14.371599+00:00", "status": "completed" }, "tags": [] @@ -644,16 +644,16 @@ "id": "ffab9640", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:39.236662Z", - "iopub.status.busy": "2026-08-14T15:16:39.236397Z", - "iopub.status.idle": "2026-08-14T15:16:39.249088Z", - "shell.execute_reply": "2026-08-14T15:16:39.248325Z" + "iopub.execute_input": "2026-09-02T22:07:14.381421Z", + "iopub.status.busy": "2026-09-02T22:07:14.381205Z", + "iopub.status.idle": "2026-09-02T22:07:14.393147Z", + "shell.execute_reply": "2026-09-02T22:07:14.392473Z" }, "papermill": { - "duration": 0.016808, - "end_time": "2026-08-14T15:16:39.249847+00:00", + "duration": 0.016233, + "end_time": "2026-09-02T22:07:14.393705+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.233039+00:00", + "start_time": "2026-09-02T22:07:14.377472+00:00", "status": "completed" }, "tags": [] @@ -715,10 +715,10 @@ "id": "88c0c29f", "metadata": { "papermill": { - "duration": 0.002491, - "end_time": "2026-08-14T15:16:39.254965+00:00", + "duration": 0.003105, + "end_time": "2026-09-02T22:07:14.400056+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.252474+00:00", + "start_time": "2026-09-02T22:07:14.396951+00:00", "status": "completed" }, "tags": [] @@ -737,16 +737,16 @@ "id": "09b17784", "metadata": { "execution": { - "iopub.execute_input": "2026-08-14T15:16:39.261270Z", - "iopub.status.busy": "2026-08-14T15:16:39.261033Z", - "iopub.status.idle": "2026-08-14T15:16:39.272456Z", - "shell.execute_reply": "2026-08-14T15:16:39.271777Z" + "iopub.execute_input": "2026-09-02T22:07:14.406935Z", + "iopub.status.busy": "2026-09-02T22:07:14.406562Z", + "iopub.status.idle": "2026-09-02T22:07:14.416631Z", + "shell.execute_reply": "2026-09-02T22:07:14.416002Z" }, "papermill": { - "duration": 0.015897, - "end_time": "2026-08-14T15:16:39.273403+00:00", + "duration": 0.014285, + "end_time": "2026-09-02T22:07:14.417188+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.257506+00:00", + "start_time": "2026-09-02T22:07:14.402903+00:00", "status": "completed" }, "tags": [] @@ -809,10 +809,10 @@ "id": "cd0dd693", "metadata": { "papermill": { - "duration": 0.002778, - "end_time": "2026-08-14T15:16:39.278986+00:00", + "duration": 0.003202, + "end_time": "2026-09-02T22:07:14.423737+00:00", "exception": false, - "start_time": "2026-08-14T15:16:39.276208+00:00", + "start_time": "2026-09-02T22:07:14.420535+00:00", "status": "completed" }, "tags": [] @@ -836,10 +836,10 @@ "id": "aeed2f5f", "metadata": { "papermill": { - "duration": 0.001536, - "end_time": "2026-08-15T07:41:42.639006", + "duration": 0.002979, + "end_time": "2026-09-02T22:07:14.429622+00:00", "exception": false, - "start_time": "2026-08-15T07:41:42.637470", + "start_time": "2026-09-02T22:07:14.426643+00:00", "status": "completed" }, "tags": [] @@ -858,16 +858,16 @@ "id": "8010a13e", "metadata": { "execution": { - "iopub.execute_input": "2026-08-15T07:41:42.644030Z", - "iopub.status.busy": "2026-08-15T07:41:42.644030Z", - "iopub.status.idle": "2026-08-15T07:41:42.653862Z", - "shell.execute_reply": "2026-08-15T07:41:42.653294Z" + "iopub.execute_input": "2026-09-02T22:07:14.436653Z", + "iopub.status.busy": "2026-09-02T22:07:14.436473Z", + "iopub.status.idle": "2026-09-02T22:07:14.446058Z", + "shell.execute_reply": "2026-09-02T22:07:14.445422Z" }, "papermill": { - "duration": 0.012813, - "end_time": "2026-08-15T07:41:42.653862", + "duration": 0.014067, + "end_time": "2026-09-02T22:07:14.446606+00:00", "exception": false, - "start_time": "2026-08-15T07:41:42.641049", + "start_time": "2026-09-02T22:07:14.432539+00:00", "status": "completed" }, "tags": [] @@ -925,10 +925,10 @@ "id": "e1a9027d", "metadata": { "papermill": { - "duration": 0.002074, - "end_time": "2026-08-15T07:41:42.658960", + "duration": 0.003158, + "end_time": "2026-09-02T22:07:14.452793+00:00", "exception": false, - "start_time": "2026-08-15T07:41:42.656886", + "start_time": "2026-09-02T22:07:14.449635+00:00", "status": "completed" }, "tags": [] @@ -947,16 +947,16 @@ "id": "d8d7bc61", "metadata": { "execution": { - "iopub.execute_input": "2026-08-15T07:41:42.664837Z", - "iopub.status.busy": "2026-08-15T07:41:42.664837Z", - "iopub.status.idle": "2026-08-15T07:41:42.671679Z", - "shell.execute_reply": "2026-08-15T07:41:42.671126Z" + "iopub.execute_input": "2026-09-02T22:07:14.459672Z", + "iopub.status.busy": "2026-09-02T22:07:14.459483Z", + "iopub.status.idle": "2026-09-02T22:07:14.468912Z", + "shell.execute_reply": "2026-09-02T22:07:14.468094Z" }, "papermill": { - "duration": 0.009964, - "end_time": "2026-08-15T07:41:42.671679", + "duration": 0.013759, + "end_time": "2026-09-02T22:07:14.469486+00:00", "exception": false, - "start_time": "2026-08-15T07:41:42.661715", + "start_time": "2026-09-02T22:07:14.455727+00:00", "status": "completed" }, "tags": [] @@ -1004,10 +1004,10 @@ "id": "sec9-md-header", "metadata": { "papermill": { - "duration": 0.004246, - "end_time": "2026-08-24T05:40:26.506247+00:00", + "duration": 0.003374, + "end_time": "2026-09-02T22:07:14.475853+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.502001+00:00", + "start_time": "2026-09-02T22:07:14.472479+00:00", "status": "completed" }, "tags": [] @@ -1031,16 +1031,16 @@ "id": "sec9-code-decomp", "metadata": { "execution": { - "iopub.execute_input": "2026-08-24T05:40:26.516778Z", - "iopub.status.busy": "2026-08-24T05:40:26.516403Z", - "iopub.status.idle": "2026-08-24T05:40:26.539103Z", - "shell.execute_reply": "2026-08-24T05:40:26.538274Z" + "iopub.execute_input": "2026-09-02T22:07:14.482841Z", + "iopub.status.busy": "2026-09-02T22:07:14.482567Z", + "iopub.status.idle": "2026-09-02T22:07:14.493201Z", + "shell.execute_reply": "2026-09-02T22:07:14.492555Z" }, "papermill": { - "duration": 0.029178, - "end_time": "2026-08-24T05:40:26.539834+00:00", + "duration": 0.015043, + "end_time": "2026-09-02T22:07:14.493854+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.510656+00:00", + "start_time": "2026-09-02T22:07:14.478811+00:00", "status": "completed" }, "tags": [] @@ -1052,10 +1052,16 @@ "text": [ "Run de-biaise recentre : 12 combos BTC-USD, HAR debiased=True, DM centered=True\n", "Loss-fn DM : mse sur erreurs recentrees\n", - "Seeds : [0, 7, 42, 99] | Horizons : [1, 5, 10] | Runtime : 1787s (29.8 min)\n", + "Seeds : [0, 7, 42, 99] | Horizons : [1, 5, 10] | Runtime : 1776s (29.6 min)\n", "\n", "=== HAR bias OOS et decomposition biais²+variance (HAR brut vs HAR debiaise) ===\n", - " h HAR raw MSE HAR deb MSE bias_oos HAR var raw HAR var deb HAR bias²/var\n", + " h HAR raw MSE HAR deb MSE bias_oos HAR var raw HAR var deb HAR bias²/var\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ " 1 0.88768 0.83633 -0.2266 0.83633 0.83633 0.0614\n", " 5 0.52197 0.40420 -0.3432 0.40420 0.40420 0.2914\n", " 10 0.57068 0.36799 -0.4502 0.36799 0.36799 0.5508\n" @@ -1097,10 +1103,10 @@ "id": "sec9-1-md", "metadata": { "papermill": { - "duration": 0.004509, - "end_time": "2026-08-24T05:40:26.548914+00:00", + "duration": 0.003331, + "end_time": "2026-09-02T22:07:14.500639+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.544405+00:00", + "start_time": "2026-09-02T22:07:14.497308+00:00", "status": "completed" }, "tags": [] @@ -1117,16 +1123,16 @@ "id": "sec9-1-code", "metadata": { "execution": { - "iopub.execute_input": "2026-08-24T05:40:26.559012Z", - "iopub.status.busy": "2026-08-24T05:40:26.558708Z", - "iopub.status.idle": "2026-08-24T05:40:26.569079Z", - "shell.execute_reply": "2026-08-24T05:40:26.568093Z" + "iopub.execute_input": "2026-09-02T22:07:14.507841Z", + "iopub.status.busy": "2026-09-02T22:07:14.507672Z", + "iopub.status.idle": "2026-09-02T22:07:14.515442Z", + "shell.execute_reply": "2026-09-02T22:07:14.514690Z" }, "papermill": { - "duration": 0.016636, - "end_time": "2026-08-24T05:40:26.569839+00:00", + "duration": 0.01256, + "end_time": "2026-09-02T22:07:14.516335+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.553203+00:00", + "start_time": "2026-09-02T22:07:14.503775+00:00", "status": "completed" }, "tags": [] @@ -1171,10 +1177,10 @@ "id": "sec9-2-md", "metadata": { "papermill": { - "duration": 0.004448, - "end_time": "2026-08-24T05:40:26.578756+00:00", + "duration": 0.00395, + "end_time": "2026-09-02T22:07:14.526112+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.574308+00:00", + "start_time": "2026-09-02T22:07:14.522162+00:00", "status": "completed" }, "tags": [] @@ -1194,16 +1200,16 @@ "id": "sec9-2-code", "metadata": { "execution": { - "iopub.execute_input": "2026-08-24T05:40:26.589016Z", - "iopub.status.busy": "2026-08-24T05:40:26.588626Z", - "iopub.status.idle": "2026-08-24T05:40:26.597444Z", - "shell.execute_reply": "2026-08-24T05:40:26.596495Z" + "iopub.execute_input": "2026-09-02T22:07:14.534009Z", + "iopub.status.busy": "2026-09-02T22:07:14.533803Z", + "iopub.status.idle": "2026-09-02T22:07:14.540891Z", + "shell.execute_reply": "2026-09-02T22:07:14.540158Z" }, "papermill": { - "duration": 0.015155, - "end_time": "2026-08-24T05:40:26.598278+00:00", + "duration": 0.011992, + "end_time": "2026-09-02T22:07:14.541411+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.583123+00:00", + "start_time": "2026-09-02T22:07:14.529419+00:00", "status": "completed" }, "tags": [] @@ -1244,10 +1250,10 @@ "id": "sec9-3-md", "metadata": { "papermill": { - "duration": 0.004714, - "end_time": "2026-08-24T05:40:26.607630+00:00", + "duration": 0.003457, + "end_time": "2026-09-02T22:07:14.548291+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.602916+00:00", + "start_time": "2026-09-02T22:07:14.544834+00:00", "status": "completed" }, "tags": [] @@ -1266,16 +1272,16 @@ "id": "sec9-3-code", "metadata": { "execution": { - "iopub.execute_input": "2026-08-24T05:40:26.617795Z", - "iopub.status.busy": "2026-08-24T05:40:26.617423Z", - "iopub.status.idle": "2026-08-24T05:40:26.632871Z", - "shell.execute_reply": "2026-08-24T05:40:26.631819Z" + "iopub.execute_input": "2026-09-02T22:07:14.556242Z", + "iopub.status.busy": "2026-09-02T22:07:14.556035Z", + "iopub.status.idle": "2026-09-02T22:07:14.568002Z", + "shell.execute_reply": "2026-09-02T22:07:14.567392Z" }, "papermill": { - "duration": 0.021683, - "end_time": "2026-08-24T05:40:26.633647+00:00", + "duration": 0.017035, + "end_time": "2026-09-02T22:07:14.568751+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.611964+00:00", + "start_time": "2026-09-02T22:07:14.551716+00:00", "status": "completed" }, "tags": [] @@ -1285,7 +1291,13 @@ "name": "stdout", "output_type": "stream", "text": [ - "=== Conjonction §C recentree par horizon ===\n", + "=== Conjonction §C recentree par horizon ===\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ "\n", " h edge (red %) sigma xs edge/2σ dm_p_med var ratio verdict_sc\n", " 1 +10.1% 0.1 33.80x 1.54e-09 0.8990 BEATS <- conjonction §C tenue\n", @@ -1345,10 +1357,10 @@ "id": "sec9-4-md", "metadata": { "papermill": { - "duration": 0.004465, - "end_time": "2026-08-24T05:40:26.642867+00:00", + "duration": 0.003413, + "end_time": "2026-09-02T22:07:14.575692+00:00", "exception": false, - "start_time": "2026-08-24T05:40:26.638402+00:00", + "start_time": "2026-09-02T22:07:14.572279+00:00", "status": "completed" }, "tags": [] @@ -1401,17 +1413,17 @@ }, "papermill": { "default_parameters": {}, - "duration": 7.027581, - "end_time": "2026-08-24T05:40:27.434591+00:00", + "duration": 5.48137, + "end_time": "2026-09-02T22:07:15.250721+00:00", "environment_variables": {}, "exception": null, - "input_path": "C:\\dev\\CoursIA-c1331p441-12734\\MyIA.AI.Notebooks\\QuantConnect\\ML-Training-Pipeline\\m4_dlinear_vol_sc_validation.ipynb", - "output_path": "C:\\dev\\CoursIA-c1331p441-12734\\MyIA.AI.Notebooks\\QuantConnect\\ML-Training-Pipeline\\m4_dlinear_vol_sc_validation_output.ipynb", + "input_path": "m4_dlinear_vol_sc_validation.ipynb", + "output_path": "m4_dlinear_vol_sc_validation.ipynb", "parameters": {}, - "start_time": "2026-08-24T05:40:20.407010+00:00", + "start_time": "2026-09-02T22:07:09.769351+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 -} +} \ No newline at end of file diff --git a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/btc_vol.py b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/btc_vol.py index 12f5eb6676..c7a59696ea 100644 --- a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/btc_vol.py +++ b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/btc_vol.py @@ -94,6 +94,39 @@ def _dm_centered_mse( } +def _dm_uncentered_mse( + errors_a: np.ndarray, errors_b: np.ndarray, horizon: int +) -> dict: + """DM test on RAW (non-centered) errors, with loss_fn='mse'. + + This is the sanity leg reproducing the #11011 keeper measure: the MSE + differential on untouched errors, bias included. It is deliberately NOT + `_dm_centered_mse`: centering subtracts each series' own mean, which + annihilates any constant offset between two error series. Since the + de-biased HAR errors differ from the raw ones by exactly the constant + `har_bias_oos`, routing this leg through the centered helper makes it + return the *same* statistic as the verdict leg -- a control that cannot + go red (#14362). Same return shape as `_dm_centered_mse`. + """ + from dm_test import dm_verdict as dm_verdict_fn + + e_a = np.asarray(errors_a, dtype=float) + e_b = np.asarray(errors_b, dtype=float) + if e_a.shape != e_b.shape: + return {"dm_stat": float("nan"), "dm_pvalue": float("nan"), "dm_verdict": "SHAPE_MISMATCH"} + n = len(e_a) + if n < 10: + return {"dm_stat": float("nan"), "dm_pvalue": float("nan"), "dm_verdict": "INSUFFICIENT_DATA"} + + res = dm_verdict_fn(e_a, e_b, horizon=horizon, loss_fn="mse") + return { + "dm_stat": float(res["dm_statistic"]), + "dm_pvalue": float(res["p_value"]), + "dm_verdict": str(res["verdict"]), + "mean_loss_diff": float(res["mean_loss_diff"]), + } + + def run_btc_debiased_recentered( horizons: list[int], seeds: list[int], @@ -156,12 +189,18 @@ def run_btc_debiased_recentered( dl_decomp = _mse_decomposition(dl_errors) - # DM on centered errors (variance differential) -- HAR DEBIASED. + # Verdict leg: DM on CENTERED errors (variance differential), + # DLinear raw vs HAR DEBIASED. This is the jambe that carries §C. min_len = min(len(dl_errors), len(har_errors_debiased)) dm_centered = _dm_centered_mse( dl_errors[:min_len], har_errors_debiased[:min_len], horizon=h ) - dm_raw = _dm_centered_mse( + # Sanity leg: DM NON centered, DLinear raw vs HAR RAW -- the #11011 + # keeper measure. It must NOT be routed through `_dm_centered_mse`: + # `har_errors_debiased = har_errors - har_bias_oos` differs from + # `har_errors` by a constant, and centering annihilates exactly that + # constant, so both legs would return the same statistic (see #14362). + dm_uncentered = _dm_uncentered_mse( dl_errors[:min_len], har_errors[:min_len], horizon=h ) @@ -190,9 +229,12 @@ def run_btc_debiased_recentered( "dm_centered_pvalue": dm_centered["dm_pvalue"], "dm_centered_verdict": dm_centered["dm_verdict"], "dm_centered_mean_loss_diff": dm_centered.get("mean_loss_diff", float("nan")), - "dm_raw_stat": dm_raw["dm_stat"], - "dm_raw_pvalue": dm_raw["dm_pvalue"], - "dm_raw_verdict": dm_raw["dm_verdict"], + "dm_uncentered_vs_har_raw_stat": dm_uncentered["dm_stat"], + "dm_uncentered_vs_har_raw_pvalue": dm_uncentered["dm_pvalue"], + "dm_uncentered_vs_har_raw_verdict": dm_uncentered["dm_verdict"], + "dm_uncentered_vs_har_raw_mean_loss_diff": dm_uncentered.get( + "mean_loss_diff", float("nan") + ), "n_predictions": int(dl_out["n_total_preds"]), "n_rv_days": int(len(rv)), }) diff --git a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json index d459b7dcad..55cb7978af 100644 --- a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json +++ b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/results/m4_dlinear_vol_btc_sc_debiased_recentered.json @@ -21,9 +21,10 @@ "dm_centered_pvalue": 5.77933256806773e-10, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.08630849855622481, - "dm_raw_stat": -6.228784907758053, - "dm_raw_pvalue": 5.77933256806773e-10, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -9.308815739868166, + "dm_uncentered_vs_har_raw_pvalue": 0.0, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.13763636412397387, "n_predictions": 1890, "n_rv_days": 2278 }, @@ -48,9 +49,10 @@ "dm_centered_pvalue": 4.266757391846454e-09, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.0828739985835896, - "dm_raw_stat": -5.901210747033868, - "dm_raw_pvalue": 4.266757391846454e-09, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -9.150634683670352, + "dm_uncentered_vs_har_raw_pvalue": 0.0, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.13419597911104217, "n_predictions": 1890, "n_rv_days": 2278 }, @@ -75,9 +77,10 @@ "dm_centered_pvalue": 1.2840908336642087e-09, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.0846959474254172, - "dm_raw_stat": -6.099898908625077, - "dm_raw_pvalue": 1.2840908336642087e-09, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -9.262350386794445, + "dm_uncentered_vs_har_raw_pvalue": 0.0, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.13600073281412045, "n_predictions": 1890, "n_rv_days": 2278 }, @@ -102,9 +105,10 @@ "dm_centered_pvalue": 1.8058008421917293e-09, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.08401933075000244, - "dm_raw_stat": -6.044090732273396, - "dm_raw_pvalue": 1.8058008421917293e-09, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -9.333346721250683, + "dm_uncentered_vs_har_raw_pvalue": 0.0, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.13529380087332005, "n_predictions": 1890, "n_rv_days": 2278 }, @@ -129,9 +133,10 @@ "dm_centered_pvalue": 5.352813553383129e-05, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.030535423916036263, - "dm_raw_stat": -4.049040174742201, - "dm_raw_pvalue": 5.352813553383129e-05, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -6.4982458593752215, + "dm_uncentered_vs_har_raw_pvalue": 1.0392997573660523e-10, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.14827922573307575, "n_predictions": 1870, "n_rv_days": 2278 }, @@ -156,9 +161,10 @@ "dm_centered_pvalue": 0.00010662841412600876, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.029458309000076354, - "dm_raw_stat": -3.8833086283328817, - "dm_raw_pvalue": 0.00010662841412600876, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -6.410523125545738, + "dm_uncentered_vs_har_raw_pvalue": 1.8314505467742492e-10, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.14720370139414576, "n_predictions": 1870, "n_rv_days": 2278 }, @@ -183,9 +189,10 @@ "dm_centered_pvalue": 0.00015904809598477065, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.02799095436581433, - "dm_raw_stat": -3.7842228030691065, - "dm_raw_pvalue": 0.00015904809598477065, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -6.394581498492657, + "dm_uncentered_vs_har_raw_pvalue": 2.0285328972136085e-10, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.14572087340869952, "n_predictions": 1870, "n_rv_days": 2278 }, @@ -210,9 +217,10 @@ "dm_centered_pvalue": 8.657436804493379e-05, "dm_centered_verdict": "BEATS baseline", "dm_centered_mean_loss_diff": -0.029952223934137993, - "dm_raw_stat": -3.934064343605695, - "dm_raw_pvalue": 8.657436804493379e-05, - "dm_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_stat": -6.504799754772119, + "dm_uncentered_vs_har_raw_pvalue": 9.959455482544399e-11, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.1476880542748526, "n_predictions": 1870, "n_rv_days": 2278 }, @@ -237,9 +245,10 @@ "dm_centered_pvalue": 0.10344094996441178, "dm_centered_verdict": "INCONCLUSIVE", "dm_centered_mean_loss_diff": -0.013460541598866624, - "dm_raw_stat": -1.6292025580658112, - "dm_raw_pvalue": 0.10344094996441178, - "dm_raw_verdict": "INCONCLUSIVE", + "dm_uncentered_vs_har_raw_stat": -5.94917859880759, + "dm_uncentered_vs_har_raw_pvalue": 3.2169920061164703e-09, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.21614914055752882, "n_predictions": 1845, "n_rv_days": 2278 }, @@ -264,9 +273,10 @@ "dm_centered_pvalue": 0.11757456059303473, "dm_centered_verdict": "INCONCLUSIVE", "dm_centered_mean_loss_diff": -0.012956115065813675, - "dm_raw_stat": -1.5657678314136847, - "dm_raw_pvalue": 0.11757456059303473, - "dm_raw_verdict": "INCONCLUSIVE", + "dm_uncentered_vs_har_raw_stat": -5.921302353064981, + "dm_uncentered_vs_har_raw_pvalue": 3.8006469083029515e-09, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.21565307101237308, "n_predictions": 1845, "n_rv_days": 2278 }, @@ -291,9 +301,10 @@ "dm_centered_pvalue": 0.0717841959895027, "dm_centered_verdict": "INCONCLUSIVE", "dm_centered_mean_loss_diff": -0.014755380212165842, - "dm_raw_stat": -1.8015203320683724, - "dm_raw_pvalue": 0.07178419598950292, - "dm_raw_verdict": "INCONCLUSIVE", + "dm_uncentered_vs_har_raw_stat": -6.014850417141578, + "dm_uncentered_vs_har_raw_pvalue": 2.1659258830908357e-09, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.21745313532776406, "n_predictions": 1845, "n_rv_days": 2278 }, @@ -318,9 +329,10 @@ "dm_centered_pvalue": 0.10075284985975763, "dm_centered_verdict": "INCONCLUSIVE", "dm_centered_mean_loss_diff": -0.013203880305558816, - "dm_raw_stat": -1.6420369908879489, - "dm_raw_pvalue": 0.10075284985975763, - "dm_raw_verdict": "INCONCLUSIVE", + "dm_uncentered_vs_har_raw_stat": -5.963643000538417, + "dm_uncentered_vs_har_raw_pvalue": 2.9495581532756887e-09, + "dm_uncentered_vs_har_raw_verdict": "BEATS baseline", + "dm_uncentered_vs_har_raw_mean_loss_diff": -0.21590240242963946, "n_predictions": 1845, "n_rv_days": 2278 } @@ -375,7 +387,7 @@ "verdict_sc": "INCONCLUSIVE" } ], - "elapsed_s": 1786.9062163829803, + "elapsed_s": 1776.391629934311, "config": { "horizons": [ 1, diff --git a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/tests/test_btc_vol.py b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/tests/test_btc_vol.py index c93cdf44fb..dcf786f2b9 100644 --- a/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/tests/test_btc_vol.py +++ b/MyIA.AI.Notebooks/QuantConnect/ML-Training-Pipeline/scripts/tests/test_btc_vol.py @@ -19,7 +19,11 @@ # Make the parent directory importable. sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) -from btc_vol import _dm_centered_mse, _mse_decomposition # noqa: E402 +from btc_vol import ( # noqa: E402 + _dm_centered_mse, + _dm_uncentered_mse, + _mse_decomposition, +) class TestMseDecomposition: @@ -116,4 +120,105 @@ def test_clearly_more_precise_wins_on_centered(self): out = _dm_centered_mse(e_a, e_b, horizon=1) assert out["dm_verdict"] == "BEATS baseline" assert out["dm_pvalue"] < 0.05 - assert out["dm_stat"] < -2.0 # negative = model wins (smaller MSE) \ No newline at end of file + assert out["dm_stat"] < -2.0 # negative = model wins (smaller MSE) + + +def _har_like_pair(seed: int = 0, n: int = 300, har_bias: float = 0.5): + """A (dl_errors, har_errors) pair whose biases differ, as in btc_vol. + + `har_errors` carries a non-zero mean; the de-biased series that + `run_btc_debiased_recentered` builds is `har_errors - mean(har_errors)`, + i.e. the *same* series shifted by a constant. + """ + rng = np.random.default_rng(seed) + dl_errors = rng.standard_normal(n) * 0.8 + har_errors = rng.standard_normal(n) * 1.0 + har_bias + har_errors_debiased = har_errors - float(np.mean(har_errors)) + return dl_errors, har_errors, har_errors_debiased + + +class TestTwoLegsDiscriminate: + """The two DM legs of `run_btc_debiased_recentered` must not coincide (#14362). + + The verdict leg is centered and runs against **de-biased** HAR; the sanity + leg is un-centered and runs against **raw** HAR, reproducing the #11011 + keeper. Routing the sanity leg through `_dm_centered_mse` -- as the code + did before #14362 -- makes the two bit-identical, because centering + subtracts each series' own mean and the two HAR series differ by exactly + a constant. A control that cannot go red is not a control. + """ + + def test_biases_actually_differ_in_the_fixture(self): + """Guard the guard: the fixture must be a pair whose biases differ.""" + _, har_raw, har_deb = _har_like_pair() + assert abs(float(np.mean(har_raw)) - float(np.mean(har_deb))) > 0.1 + assert float(np.mean(har_deb)) == pytest.approx(0.0, abs=1e-12) + + def test_legs_are_not_the_same_statistic(self): + """THE sealed control: the two legs must return different statistics.""" + dl, har_raw, har_deb = _har_like_pair() + verdict_leg = _dm_centered_mse(dl, har_deb, horizon=1) + sanity_leg = _dm_uncentered_mse(dl, har_raw, horizon=1) + assert abs(verdict_leg["dm_stat"] - sanity_leg["dm_stat"]) > 1e-6, ( + "the sanity leg reproduces the verdict leg -- it is centered " + "somewhere it must not be (regression of #14362)" + ) + + def test_old_composition_is_degenerate(self): + """Falsification: the pre-#14362 composition IS bit-identical. + + Without this, `test_legs_are_not_the_same_statistic` could pass for a + reason unrelated to centering. Here we reproduce the old code path + explicitly and show it collapses -- which is what makes the assertion + above meaningful rather than incidental. + """ + dl, har_raw, har_deb = _har_like_pair() + old_verdict_leg = _dm_centered_mse(dl, har_deb, horizon=1) + old_sanity_leg = _dm_centered_mse(dl, har_raw, horizon=1) # the bug + assert old_sanity_leg["dm_stat"] == pytest.approx( + old_verdict_leg["dm_stat"], abs=1e-12 + ) + + def test_centered_leg_is_invariant_under_constant_shift(self): + """Why the collapse happens, stated as a property.""" + dl, har_raw, _ = _har_like_pair() + base = _dm_centered_mse(dl, har_raw, horizon=1) + shifted = _dm_centered_mse(dl, har_raw - 3.14159, horizon=1) + assert shifted["dm_stat"] == pytest.approx(base["dm_stat"], abs=1e-12) + + def test_uncentered_leg_is_NOT_invariant_under_constant_shift(self): + """The negative control of the property above, in the other direction.""" + dl, har_raw, _ = _har_like_pair() + base = _dm_uncentered_mse(dl, har_raw, horizon=1) + shifted = _dm_uncentered_mse(dl, har_raw - 3.14159, horizon=1) + assert abs(shifted["dm_stat"] - base["dm_stat"]) > 1e-6 + + def test_uncentered_leg_sees_a_pure_bias_gap(self): + """A baseline that is only *biased* loses the un-centered leg... + + ...and does not lose the centered one. This is the substantive reason + the sanity leg exists: it is the only one of the two that can tell + `#11011`'s story (MSE inflated by the HAR bias). + """ + rng = np.random.default_rng(11) + n = 800 + dl = rng.standard_normal(n) * 0.7 + har = dl + 1.5 # SAME dispersion realisation, differing only by a constant + uncentered = _dm_uncentered_mse(dl, har, horizon=1) + centered = _dm_centered_mse(dl, har, horizon=1) + # The un-centered leg sees the bias and calls it decisively. + assert uncentered["dm_verdict"] == "BEATS baseline" + assert uncentered["dm_pvalue"] < 1e-6 + # The centered leg is blind to it -- the differential is exactly zero. + assert centered["dm_stat"] == pytest.approx(0.0, abs=1e-9) + assert centered["dm_verdict"] == "INCONCLUSIVE" + + def test_uncentered_sentinels(self): + assert ( + _dm_uncentered_mse(np.zeros(10), np.zeros(11), horizon=1)["dm_verdict"] + == "SHAPE_MISMATCH" + ) + assert ( + _dm_uncentered_mse(np.zeros(5), np.zeros(5), horizon=1)["dm_verdict"] + == "INSUFFICIENT_DATA" + ) \ No newline at end of file