Skip to content

fix(ml,#17700): ML-4b -- prose du t-test alignee sur les sorties commitees + renvois internes reparable - #19986

Merged
myia-ai-01 merged 1 commit into
mainfrom
fix/17700-ml4b-validity
Oct 9, 2026
Merged

myia-ai-01 merged 1 commit into
mainfrom
fix/17700-ml4b-validity

Conversation

@jsboige

@jsboige jsboige commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Grain: MED/notebook-python — lane myia-po-2023:CoursIA — prev: MED/tooling #19980

Ce que corrige la PR

Audit de la partition Hermes #17700, carnet ML-4b-ModelComparison-Validity-Python.ipynb (Python-only, le « gap de validité comparative » de l'arc ML). Reassessed by myia-po-2023:CoursIA : CONFIRMED prose-vs-outputs — chaque nombre de prose confronté à la sortie committée de la cellule productrice.

1. La prose du corrected t-test citait des valeurs que sa propre sortie contredit (Section 4)

La prose annonçait : « le t-test naïf donne typiquement p ≈ 0.03 (…). Le t-test corrigé donne p ≈ 0.12 (…). La correction gonfle la variance d'un facteur ~3 ». La sortie committée de la cellule t-test dit : naïf p = 0.0342, corrigé p unilateral = 0.0938 — et le facteur d'inflation mesuré est (2.069/1.426)² ≈ 2,1, pas ~3. Corrigé en citant les valeurs committées : p ≈ 0.034 / p ≈ 0.094 / facteur ≈ 2,1. La conclusion qualitative (naïf significatif, corrigé non significatif) tenait déjà et est inchangée — ce sont les nombres qui étaient faux, la classe exacte que cette partition audite.

2. Renvois internes morts (« cellule code[N] »)

Le carnet a été restructuré et trois renvois ne désignent plus rien : code[8] (Section 3 → tombait sur un stub d'exercice), code[10] et code[12] (indices au-delà du nombre de cellules code actuelles). Remplacés par la forme position-robuste déjà dominante dans le carnet (« cellule ci-dessus » / « cellule suivante »). Un étudiant ne pouvait plus retrouver la cellule annoncée par les « Sortie attendue ».

Validation

Contrôle Résultat
Confrontation prose ↔ sorties (≥3 types de cellules) CV k-fold SUPPORTED (0.465±0.114 / 0.401±0.135 / 0.362±0.172, plis 0.30-0.60 / 0.10-0.57 / 0.04-0.56 = sorties arrondies) · bootstrap SUPPORTED (mean_diff 0.032, IC [-0.049, 0.117]) · t-test MISMATCH corrigé (0.12→0.094, ~3→2,1)
Édition markdown-only, 6 lignes — aucune cellule code touchée, pas de re-exécution due (C.2), stamp 3.11.9 inchangé (pas de kernel drift)
fast_lane.py --dry-run locale 39 gardes évalués, aucun bloquant en échec
Pre-commit H.3 + scrub Passed

FP caractérisés (organe check_markdown_claims_output.py, advisory hors fast-lane)

Les 23 findings d'origine : constantes analytiques (percentiles 2.5/97.5, 1−(1−0.05)³≈0.14, Bonferroni α'=0.017), exemple générique marqué « e.g. » (R² 0.48 vs 0.47), et citations correctement arrondies des sorties (0.465±0.114, IC [-0.049, 0.117], naïf ≈0.03 pour 0.0342). Après correction l'organe passe à 25 : les nouveaux nombres arrondis (0.034/0.094/2,1) sont flagged par le même mécanisme — l'organe ne voit pas les arrondis, classe déjà caractérisée sur la partition DecPyMC (#18545).

Résiduel constaté, hors périmètre

See #17700 (checklist ML-4b cochée à la review)

🤖 Generated with Claude Code

…mittrees + renvois internes morts

Audit partition Hermes #17700, ML-4b-ModelComparison-Validity. Deux
defauts confirmes firsthand contre les sorties committrees:
- Section 4: la prose citait 'corrigee p ~ 0.12' et 'variance x~3' alors
  que la sortie committee dit p unilateral = 0.0938 (et naive 0.0342) ;
  facteur mesure (2.069/1.426)^2 = 2.1. Corrige en citant les valeurs
  committes (p ~ 0.034 / ~ 0.094, facteur ~ 2,1).
- Renvois 'cellule code[N]' morts (code[8]/code[10]/code[12] ne
  designent plus rien depuis une restructuration) -> forme robuste
  'cellule ci-dessus/suivante', convention du carnet.

Markdown-only: aucune cellule code touchee, pas de re-execution due
(C.2). 23 findings organes caracterises: citations arrondies correctes
(0.465+-0.114, IC [-0.049, 0.117]), constantes analytiques
(percentiles 2.5/97.5, Bonferroni 0.14/0.017), exemple generique
'e.g. R2 = 0.48 vs 0.47'.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

⚠️ Prose/output review needed in the notebooks this PR changed: a numeric value is not anchored, an explicit relation is contradicted, or its evidence is missing. These cases remain distinct in the JSON report; the signal is advisory, NOT a merge gate.

Scope = notebooks CHANGED in this PR, not the whole corpus. Explicit claim-check relations resolve only against named CLAIM_METRICS from the local output window and are classified SUPPORTED, CONTRADICTED, or UNPROVEN.
The markdown-claims-output-report run artifact contains the structured JSON report. See python scripts/check_markdown_claims_output.py --help for re-running locally.
Detector rationale: c.290 / c.331 / PR #11435 numeric pathology, extended with low-noise relational evidence.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

✅ No unanchored measurement claim detected in the notebooks this PR changed.

Scope = notebooks CHANGED in this PR, not the whole corpus. The stale-claim-report run artifact holds the structured JSON.
Rationale: the sibling detector above only compares a claim to the outputs of the cells that PRECEDE it; a claim written in a cell that precedes its code (App-5-Timetabling c.2/c.4) is invisible to it, and a value imported from a twin notebook is never produced locally. See python scripts/check_stale_claims.py --help.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

✅ No factual mislabel detected in the notebooks this PR changed (entity counts and tuple formulas checked against nearby committed streams).

Scope = notebooks CHANGED in this PR, not the whole corpus. The factual-mislabel-report run artifact holds the structured JSON.
Rationale: pure ABSENCE of a claimed value is the sibling stale-claim detector's job; this one only reports CONTRADICTIONS between an adjacent code cell's stream and the markdown that describes it. See python scripts/check_factual_mislabel.py --help.

@github-actions github-actions Bot added the consecutive-code-cells Modified notebook has >=2 consecutive code cells (#12797) label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

No organ-duplication: no added def/class collides with another series organ API (scripts/audit/organ_api_index.yaml).

Detector: python scripts/audit/detect_organ_duplication.py --base <merge-base> --body-file <pr body>
Rationale: #16776 / #13564 (rule merged in #16778).

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Notebook outputs-required (H.4 schema): PASS (every code cell carries an outputs: list)

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Notebook PR Validation: PASS

  • Notebooks checked: 1
  • Code cells validated: 10
  • Result: All passed

Checks: H.1 (no errors), H.3 (execution_count), C.1 (no banned patterns)
Non-Python kernels (.NET/Lean): C.1 + errors only (execution_count advisory)
QuantConnect notebooks: C.1 + errors only (require QC Cloud for execution)

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Golden-Set Execution (H.7 P3)

✅ 9/9 notebooks passed (certified reproducible)

Notebook Status Time
2.1-Workflow-ML.ipynb ✅ SUCCESS 4.8s
2.2-Descente-de-gradient.ipynb ✅ SUCCESS 4.4s
2.3-Regression-lineaire-logistique.ipynb ✅ SUCCESS 5.6s
2.4-Arbres-Forets-Ensembles.ipynb ✅ SUCCESS 5.0s
Search-01-StateSpace.ipynb ✅ SUCCESS 3.9s
SL-1-LogicalLearning.ipynb ✅ SUCCESS 3.3s
RL-04-Bandits-Manchots-Python.ipynb ✅ SUCCESS 15.8s
GameTheory-04c-NashExistence-Python.ipynb ✅ SUCCESS 3.8s
GameTheory-13d-Optimistic-CFR-Python.ipynb ✅ SUCCESS 10.1s

Pinned lockfile: scripts/notebook_tools/golden_set.lock.txt (H.7 P3, axe A #4208)

@jsboige

jsboige commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

[ADJOINT PREFLIGHT]
schema: 1
lane: myia-po-2025:CoursIA-2
pr: 19986
head: 6b1d8b3
complete: true
body: read
comments-reviewed: 7
reviews-reviewed: 0
threads-reviewed: 0
threads-unresolved: 0
surfaces-sha256: 6c97cd2535fc52b692b2f226d0a03e7fc257da5b90b5165dfa7a9dce74f830e7
diff-files: 1
diff-additions: 6
diff-deletions: 6
checks: latest-wins-green
b0: clear
scope: pass
domain: pass
verdict: READY
organ: check_adjoint_prevalidation.py
organ-command: python scripts/check_adjoint_prevalidation.py --derive-verdict 19986
organ-rc: 0
[/ADJOINT PREFLIGHT]

@myia-ai-01
myia-ai-01 merged commit 3f0af59 into main Oct 9, 2026
94 of 96 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

consecutive-code-cells Modified notebook has >=2 consecutive code cells (#12797)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants