Skip to content

fix(notebook,#18124): research_asset_class_momentum -- local research path + real 12y top-3 backtest replace placeholder - #18162

Merged
myia-ai-01 merged 4 commits into
mainfrom
feature/18124-researchexec-acm
Sep 29, 2026
Merged

myia-ai-01 merged 4 commits into
mainfrom
feature/18124-researchexec-acm

Conversation

@jsboige

@jsboige jsboige commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Grain: MED/notebook-python -- lane myia-po-2026:CoursIA -- prev: #18160

Summary

SOTA repair of Research-Executor/research_asset_class_momentum.ipynb (1st of
the placeholder set in the #18124 audit).

  • Was: QuantBook-only cells (unrunnable locally) ending in a fake
    "BACKTEST RESULTS" placeholder printing unverified "expected
    characteristics" (CAGR ~8-12%, MaxDD ~15-25%, Sharpe ~0.6-0.9).
  • Now: honest local research path (same 5 asset-class ETFs
    SPY/EFA/BND/VNQ/GSG via yfinance, tz-normalized per env-gotchas, unified
    close frame on both QC/local paths) and a real vectorized backtest of
    the strategy itself
    : at each month-end rank by 252d momentum, hold top 3
    equal-weight next month, 132 months measured.

Honest measured findings (SOTA-OK)

  • Strategy CAGR 10.25% | Vol 13.45% | Sharpe 0.76 | MaxDD -30.05% vs on
    the same window: SPY buy&hold 15.38%, equal-weight-5 8.51% (final
    equity 2.92x / 4.81x / 2.45x).
  • Momentum top-3 beats the naive equal-weight but trails SPY buy&hold
    over this 12y window -- stated as measured, no spin.
  • The measured MaxDD (-30%) exceeds the old placeholder's "expected
    15-25%"
    : the previous claims were design expectations, not results.
  • Current selection measured live: GSG (+54.97% 252d momentum), SPY, EFA.
  • No costs/slippage modeled; production engine on QC Cloud remains the
    reference -- stated in the closing note.

Validation

  • Papermill: 9/9 cells, 0 errors, all execution_count set (C.2)
  • C.1: no raise NotImplementedError / assert False / 1/0 (verified)
  • Catalogue byte-identical to main (single .ipynb changed)
  • SOTA verdict: SOTA-OK -- real ETF data, real strategy backtest,
    committed outputs are the real outputs

See #18124 (partial: 4 of 10; remaining from this lane:
defensive_etf_rotation, long_short_harvest).

🤖 Generated with Claude Code

… path + real 12y top-3 backtest replace placeholder

SOTA repair (1st of the placeholder set in the #18124 audit):
- The notebook was QuantBook-only (unrunnable locally) and ended in a fake
  "BACKTEST RESULTS" placeholder with unverified "expected characteristics".
- Added the honest local research path used across the family: the same 5
  asset-class ETFs (SPY/EFA/BND/VNQ/GSG) via yfinance, tz-normalized, same
  `close` frame downstream on both paths.
- Replaced the placeholder with a REAL vectorized backtest of the strategy
  itself: monthly top-3 by 252d momentum, equal weight, next-month holding,
  132 months measured -- CAGR 10.25%, Sharpe 0.76, MaxDD -30.05% vs SPY
  15.38% and equal-weight-5 8.51% on the same window.
- Honest findings written in: momentum beats the naive equal-weight but
  trails SPY buy&hold over this window, and the measured MaxDD (-30%)
  exceeds the placeholder's optimistic "expected 15-25%" -- the old claims
  were design expectations, not results.

Executed 9/9 cells, 0 errors, all execution_count set (C.1/C.2).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Notebook PR Validation: PASS

  • Notebooks checked: 1
  • Code cells validated: 5
  • Result: All passed

Checks: H.1 (no errors), H.3 (execution_count), C.1 (no banned patterns)
Non-Python kernels (.NET/Lean): C.1 + errors only (execution_count advisory)
QuantConnect notebooks: C.1 + errors only (require QC Cloud for execution)

@github-actions

Copy link
Copy Markdown
Contributor

No organ-duplication: no added def/class collides with another series organ API (scripts/audit/organ_api_index.yaml).

Detector: python scripts/audit/detect_organ_duplication.py --base <merge-base> --body-file <pr body>
Rationale: #16776 / #13564 (rule merged in #16778).

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Golden-Set Execution (H.7 P3)

✅ 8/8 notebooks passed (certified reproducible)

Notebook Status Time
2.1-Workflow-ML.ipynb ✅ SUCCESS 3.3s
2.2-Descente-de-gradient.ipynb ✅ SUCCESS 3.4s
2.3-Regression-lineaire-logistique.ipynb ✅ SUCCESS 4.1s
2.4-Arbres-Forets-Ensembles.ipynb ✅ SUCCESS 4.2s
Search-01-StateSpace.ipynb ✅ SUCCESS 3.2s
SL-1-LogicalLearning.ipynb ✅ SUCCESS 2.1s
rl_4_multi_armed_bandits.ipynb ✅ SUCCESS 16.2s
GameTheory-04c-NashExistence-Python.ipynb ✅ SUCCESS 2.8s

Pinned lockfile: scripts/notebook_tools/golden_set.lock.txt (H.7 P3, axe A #4208)

@github-actions github-actions Bot added the variation-tag-prev-absent Tag Grain sans 'prev: <TIER>/<GENRE> #<PR>' (adjacence G-VAR-3 inevaluable) label Sep 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Notebook outputs-required (H.4 schema): PASS (every code cell carries an outputs: list)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Prose/output review needed in the notebooks this PR changed: a numeric value is not anchored, an explicit relation is contradicted, or its evidence is missing. These cases remain distinct in the JSON report; the signal is advisory, NOT a merge gate.

Scope = notebooks CHANGED in this PR, not the whole corpus. Explicit claim-check relations resolve only against named CLAIM_METRICS from the local output window and are classified SUPPORTED, CONTRADICTED, or UNPROVEN.
The markdown-claims-output-report run artifact contains the structured JSON report. See python scripts/check_markdown_claims_output.py --help for re-running locally.
Detector rationale: c.290 / c.331 / PR #11435 numeric pathology, extended with low-noise relational evidence.

@github-actions github-actions Bot added the large-pr-no-review PR > seuil sans review (ni bot ni humaine) -- retire quand une review arrive (#11232) label Sep 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Cette PR depasse le seuil de couverture review (par defaut 300 additions) et n'a recu aucune review -- ni bot, ni humaine.

Le label large-pr-no-review est pose par l'organe scripts/review_coverage.py porte par l'issue #11232. Aucun remede automatique : il faut obtenir une review (Hermes, ai-01, ou review humaine).

Le label est retire au balayage suivant (quotidien) des qu'une review arrive -- dans reviews[] ou en commentaire de verdict -- ou que le diff passe sous le seuil. Fermer/rouvrir la PR ne suffit pas -- la mesure porte sur le diff, pas sur l'etat de la PR.

Seuil, historique et exceptions : cf. docs/reference/review-coverage-threshold.md.

@jsboige

jsboige commented Sep 29, 2026

Copy link
Copy Markdown
Owner Author

[ADJOINT PREFLIGHT]
schema: 1
lane: myia-po-2023:CoursIA
pr: 18162
head: c18632d
complete: true
body: read
comments-reviewed: 6
reviews-reviewed: 0
threads-reviewed: 0
threads-unresolved: 0
surfaces-sha256: f23188c857003f4b9a052fdb2923099c0ada19ad4603a8e60e722d6321f6d302
diff-files: 1
diff-additions: 303
diff-deletions: 143
checks: latest-wins-green
b0: clear
scope: pass
domain: not-applicable
verdict: READY
[/ADJOINT PREFLIGHT]

Note de lecture (informatif, ne change pas le verdict) : le commentaire sticky markdown-claims-output-advisory signale une valeur numerique / relation non ancree dans le notebook change -- advisory par design, le detail vit dans l'artifact markdown-claims-output-report du run. A tracer dans la lecture finale (meme classe de prose que le realignement livre sur #18314 ce jour).

@myia-ai-01
myia-ai-01 merged commit c68be11 into main Sep 29, 2026
90 of 93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

large-pr-no-review PR > seuil sans review (ni bot ni humaine) -- retire quand une review arrive (#11232) variation-tag-prev-absent Tag Grain sans 'prev: <TIER>/<GENRE> #<PR>' (adjacence G-VAR-3 inevaluable)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants