diff --git a/.github/workflows/catalog-drift.yml b/.github/workflows/catalog-drift.yml index bbb2c25b9c..ec856a98c5 100644 --- a/.github/workflows/catalog-drift.yml +++ b/.github/workflows/catalog-drift.yml @@ -44,7 +44,13 @@ concurrency: jobs: catalog-drift: - name: "Notebook catalog drift (read-only)" + # Le marqueur `advisory` dans le NOM du job EST le contrat (#15998) : pr_gate.py + # classe par nom (`ADVISORY_MARKER`, regle 6) et ne lit PAS fast_lane_registry.py. + # Sans ce mot, la promesse "non-bloquant" de l'en-tete restait sans effet sur le + # gate : un echec d'infrastructure (runner, pip, generate_catalog) comptait comme + # un check requis et bloquait toute PR notebook/README -- observe sur #15996. + # Ne pas retirer le mot sans relire scripts/pr_gate.py (rule 6) et #15998. + name: "Notebook catalog drift (read-only, advisory)" # Routage #14283 tranche 3 (decision ai-01 2026-09-02) : jambe Linux # auto-hebergee. Le `if:` ci-dessous est la garde anti-fork exigee par # scripts/ci/check_self_hosted_runner_policy.py ; un `runs-on` STATIQUE la diff --git a/docs/reference/catalog_markers.md b/docs/reference/catalog_markers.md index 187e27a50c..fc27bc703d 100644 --- a/docs/reference/catalog_markers.md +++ b/docs/reference/catalog_markers.md @@ -1,6 +1,6 @@ # Catalog Markers - README Auto-Update System -Source-of-truth counts driven by `COURSE_CATALOG.generated.json`. Markers in README files are expanded by `scripts/notebook_tools/expand_catalog_markers.py` and verified by CI on every PR. +Source-of-truth counts driven by `COURSE_CATALOG.generated.json`. Markers in README files are expanded by `scripts/notebook_tools/expand_catalog_markers.py` and verified by CI on PRs that touch notebooks, series READMEs, or the catalog — the `catalog-drift.yml` workflow is filtered by `paths:`, so it does not run on every PR. ## Overview @@ -75,7 +75,7 @@ python scripts/notebook_tools/expand_catalog_markers.py # Dry-run (show what would change) python scripts/notebook_tools/expand_catalog_markers.py --dry-run -# Check for drift (exit 1 if stale, used by CI) +# Check for drift (exit 1 if stale; local tool -- CI regenerates instead and never calls --check) python scripts/notebook_tools/expand_catalog_markers.py --check # Expand a specific file @@ -100,12 +100,27 @@ on: - 'COURSE_CATALOG.generated.json' ``` -Two checks run in sequence: - -1. **CATALOG-STATUS marker drift** — `expand_catalog_markers.py --check` verifies all markers match the catalog -2. **Notebook catalog drift** — `verify_catalog_readme.py` checks declared counts vs actual notebooks on disk - -If either check fails, the PR is blocked until markers are updated. +Le job régénère le catalogue et les marqueurs **sur le runner** (rien n'est réécrit sur +la branche), puis compare le résultat aux fichiers commités par un unique test +`git diff --cached`. La dérive est remontée en **annotation `notice` uniquement**. + +**Ce check est advisory, non bloquant** (#15998). Le marqueur `advisory` dans le **nom** +du job — `Notebook catalog drift (read-only, advisory)` — **est** le contrat : +`pr_gate.py` classe les checks par nom et ne lit pas `fast_lane_registry.py`. Le job **peut rougir** : seule l'indisponibilité des métadonnées git (`rc=2` de +`generate_catalog.py`) est absorbée en annotation `notice` ; tout autre échec (runner, checkout, +`pip`, `generate_catalog.py` hors `rc=2`) exécute `exit "$rc"` et rend le job rouge. Mais ce +rouge est **exclu des causes bloquantes** : le marqueur `advisory` du nom fait que `PR gate` +le signale sans bloquer — une panne d'infrastructure est remontée, jamais bloquante pour une PR +notebook/README (contrôle positif #16015). Le catalogue est régénéré quotidiennement +sur `main` par `catalog-cron.yml` ; **aucune action manuelle n'est requise sur une branche +de feature** (cf [catalog-pr-hygiene.md](../../.claude/rules/catalog-pr-hygiene.md), #2632). + +> Correction factuelle (2026-09-15) : cette section décrivait deux checks en séquence +> (`expand_catalog_markers.py --check`, `verify_catalog_readme.py`) et concluait qu'un +> échec **bloquait** la PR jusqu'à mise à jour des marqueurs. Les deux affirmations +> étaient fausses : le workflow n'utilise pas `--check` (il régénère), ne fait appel à +> `verify_catalog_readme.py` dans **aucun** workflow, et son job est advisory depuis +> #15998. ## Adding Markers to a New README diff --git a/docs/reference/ci-aggregator-rollout.md b/docs/reference/ci-aggregator-rollout.md index a734683de2..94dad45435 100644 --- a/docs/reference/ci-aggregator-rollout.md +++ b/docs/reference/ci-aggregator-rollout.md @@ -176,7 +176,7 @@ GitHub = code pas rapport), §G.6 (audit avant merge cascade). | `ci / No fabricated text output in changed notebooks` | required | C.4 doc-honesty (#8052) | | `ci / No degenerate figure in changed notebooks` | required | GenAI rendering (cf #6541) | | `ci / No bare cross-dir #load in changed notebooks` | required | coupling cellule cross-répertoire | -| `ci / Notebook catalog drift (read-only)` | advisory | catalog-pr-hygiene R1 — détecteur read-only ; le garde bloquant `catalog-pr-guard.yml` est retiré (#11012) | +| `ci / Notebook catalog drift (read-only, advisory)` | advisory | catalog-pr-hygiene R1 — détecteur read-only ; le garde bloquant `catalog-pr-guard.yml` est retiré (#11012). Le marqueur `advisory` du nom est ce qui rend le gate conforme (#15998 : sans lui, un échec d'infra comptait comme requis) | | `ci / Gitleaks secret scanner` | required | secrets-hygiene | | `hooks-parity gate` | required | #8782 — gate qui ne peut plus rougir | diff --git a/docs/reference/procedures-recurrentes.md b/docs/reference/procedures-recurrentes.md index b60c9b023e..07ec966b10 100644 --- a/docs/reference/procedures-recurrentes.md +++ b/docs/reference/procedures-recurrentes.md @@ -172,7 +172,7 @@ Toute PR touchant `COURSE_CATALOG.generated.json` rend les autres PRs catalog-to - Merger les PRs **non-catalogue d'abord**, puis les catalog-touchers **un-par-un**. - Conflit catalogue = l'auteur **rebase + régénère** : `python scripts/notebook_tools/generate_catalog.py --json --git-tracked-only` (parité CI = N entrées git-tracked ; `--json` nu inclut les `_output.ipynb` locaux = drift), puis `expand_catalog_markers.py`. - **JAMAIS** force-resolve le conflit côté coord ; **JAMAIS** force-push la branche de l'auteur. -- Check CI **rouge "Notebook catalog drift" = NON mergeable** (propagerait le drift) → bounce à l'auteur pour re-régénérer. +- Check CI **"Notebook catalog drift (read-only, advisory)"** : **advisory, jamais bloquant** (#15998) — le catalogue est réécrit par `catalog-cron.yml` sur `main` (#2632/#2744), une PR n'a rien à régénérer. Un drift signalé = notice informative, pas un bounce. - **Cascade-independence** : une PR dont le drift-check = SUCCESS **et** qui ne touche pas le catalogue est **indépendante** — ne pas la hold à tort dans la file cascade. ### Trap "APPROVED" diff --git a/scripts/notebook_tools/fix_catalog_drift.py b/scripts/notebook_tools/fix_catalog_drift.py index 9fabaa478f..056528b9ce 100644 --- a/scripts/notebook_tools/fix_catalog_drift.py +++ b/scripts/notebook_tools/fix_catalog_drift.py @@ -1,8 +1,12 @@ """One-shot fix: swap `breakdown: projects=48, Python=48` → `breakdown: Python=48, projects=48` in MyIA.AI.Notebooks/QuantConnect/README.md, preserving CRLF/LF unchanged. -Used to unblock UNSTABLE PRs whose CI fails on `Notebook catalog drift` due to -non-deterministic Counter.most_common() tie-break (Windows vs Linux sort order). +Historique : ecrit pour debloquer des PRs rendues UNSTABLE par le check +`Notebook catalog drift`, a l'epoque ou `PR gate` le comptait comme un check +requis. Ce check est **advisory** depuis #15998 -- un drift ne bloque plus +aucune PR, puisque le catalogue est regenere par catalog-cron.yml sur main +(#2632/#2744). Le script reste utile comme reparation locale d'un drift de +tie-break Counter.most_common() non deterministe (tri Windows vs Linux). Usage: python scripts/notebook_tools/fix_catalog_drift.py """ diff --git a/scripts/tests/test_pr_gate.py b/scripts/tests/test_pr_gate.py index 1030f54fd9..e3a61c77f6 100644 --- a/scripts/tests/test_pr_gate.py +++ b/scripts/tests/test_pr_gate.py @@ -1085,6 +1085,44 @@ def test_green_advisory_counts_as_a_normal_pass(): assert ok == ["Large blob advisory (>= 10 MiB)"] +def test_catalog_drift_job_name_carries_the_advisory_marker(): + """#15998 -- the catalog-drift job declared itself NON-BLOCKING twice in its + own header (and in catalog-pr-hygiene / ci-aggregator docs), but + `pr_gate.py` classifies advisory by NAME (rule 6, ADVISORY_MARKER) and never + reads `fast_lane_registry.py`. Named "Notebook catalog drift (read-only)", + the job carried no marker, so any infrastructure failure (runner, pip + install, `generate_catalog` exit 2 on missing git metadata, cf #14831) was + counted as a REQUIRED check and reddened every notebook/README PR -- + observed firsthand on #15996: "PR gate: FAIL -- failing checks: Notebook + catalog drift (read-only) (failure)" while every other check passed. + + The repair is the marker in the job name, not a registry entry: the + registry is consumed by the fast lane (which would then also RUN the + catalog generation on every PR), whereas `is_advisory` reads the emitted + check-run name. Asserted over whatever the job is called today -- so a + future rename that keeps the marker passes, and one that drops it fails. + """ + if pr_gate.yaml is None: # pragma: no cover - PyYAML is a CI dependency + pytest.skip("PyYAML unavailable: cannot read the workflow") + wf_path = Path(pr_gate.DEFAULT_WORKFLOWS_DIR) / "catalog-drift.yml" + data = pr_gate.yaml.safe_load(wf_path.read_text(encoding="utf-8")) + job_names = pr_gate._workflow_job_names(data) + assert job_names, "catalog-drift.yml must declare at least one job" + for name in job_names: + assert pr_gate.is_advisory(name), ( + "the catalog-drift job name must carry the `advisory` marker " + "(#15998): pr_gate.py reads the check-run name, not the header " + "comment, so dropping the marker silently re-arms a hard gate " + "against every PR touching a notebook or a series README" + ) + # The defect the marker fixes must stay measurable: with the workflow name + # as it stands, the historical spelling is still classified BLOCKING. If + # this ever flips, the workflow name has absorbed the marker and the + # job-name invariant above stopped being the load-bearing surface. + wf_name = data.get("name") or "" + assert not pr_gate.is_advisory("Notebook catalog drift (read-only)", wf_name) + + def test_non_advisory_failure_still_blocks(): """Guard against the fix becoming a blanket amnesty.""" checks = [