From 564aa62954f7b25f7bcccda0e1a630adc18099af Mon Sep 17 00:00:00 2001 From: jsboige Date: Sun, 13 Sep 2026 20:14:21 +0200 Subject: [PATCH 1/2] fix(ci,#15998): make the catalog-drift check advisory for the gate `catalog-drift.yml` declared itself NON-BLOCKING twice in its own header, and two docs already listed the check as advisory -- but pr_gate.py classifies advisory by NAME (ADVISORY_MARKER, rule 6) and never reads fast_lane_registry.py. Named "Notebook catalog drift (read-only)", the job carried no marker, so any infrastructure failure (runner, pip install, generate_catalog exit 2 on missing git metadata, cf #14831) was counted as a REQUIRED check and reddened every notebook/README PR -- observed on #15996. The issue proposed a fast_lane_registry entry with blocking=False. That is inert for the gate (the registry is not consulted) and would additionally absorb the catalog generation into the fast lane; the load-bearing surface is the emitted check-run name. Fix accordingly: - job renamed "Notebook catalog drift (read-only, advisory)" with a comment naming the contract and #15998; - docs aligned on one truth: procedures-recurrentes.md claimed the red check was NON-mergeable (a bounce request), contradicting catalog-pr-hygiene.md and ci-aggregator-rollout.md -- now advisory everywhere, new check name in the aggregator table; - regression guard in test_pr_gate.py: asserts the job name carries the marker (robust to renames that keep it) and that the historical spelling stays classified blocking, so the defect stays measurable. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/catalog-drift.yml | 8 ++++- docs/reference/ci-aggregator-rollout.md | 2 +- docs/reference/procedures-recurrentes.md | 2 +- scripts/tests/test_pr_gate.py | 38 ++++++++++++++++++++++++ 4 files changed, 47 insertions(+), 3 deletions(-) diff --git a/.github/workflows/catalog-drift.yml b/.github/workflows/catalog-drift.yml index cae3fad1e0..fa61968804 100644 --- a/.github/workflows/catalog-drift.yml +++ b/.github/workflows/catalog-drift.yml @@ -44,7 +44,13 @@ concurrency: jobs: catalog-drift: - name: "Notebook catalog drift (read-only)" + # Le marqueur `advisory` dans le NOM du job EST le contrat (#15998) : pr_gate.py + # classe par nom (`ADVISORY_MARKER`, regle 6) et ne lit PAS fast_lane_registry.py. + # Sans ce mot, la promesse "non-bloquant" de l'en-tete restait sans effet sur le + # gate : un echec d'infrastructure (runner, pip, generate_catalog) comptait comme + # un check requis et bloquait toute PR notebook/README -- observe sur #15996. + # Ne pas retirer le mot sans relire scripts/pr_gate.py (rule 6) et #15998. + name: "Notebook catalog drift (read-only, advisory)" # Routage #14283 tranche 3 (decision ai-01 2026-09-02) : jambe Linux # auto-hebergee. Le `if:` ci-dessous est la garde anti-fork exigee par # scripts/ci/check_self_hosted_runner_policy.py ; un `runs-on` STATIQUE la diff --git a/docs/reference/ci-aggregator-rollout.md b/docs/reference/ci-aggregator-rollout.md index a734683de2..94dad45435 100644 --- a/docs/reference/ci-aggregator-rollout.md +++ b/docs/reference/ci-aggregator-rollout.md @@ -176,7 +176,7 @@ GitHub = code pas rapport), §G.6 (audit avant merge cascade). | `ci / No fabricated text output in changed notebooks` | required | C.4 doc-honesty (#8052) | | `ci / No degenerate figure in changed notebooks` | required | GenAI rendering (cf #6541) | | `ci / No bare cross-dir #load in changed notebooks` | required | coupling cellule cross-répertoire | -| `ci / Notebook catalog drift (read-only)` | advisory | catalog-pr-hygiene R1 — détecteur read-only ; le garde bloquant `catalog-pr-guard.yml` est retiré (#11012) | +| `ci / Notebook catalog drift (read-only, advisory)` | advisory | catalog-pr-hygiene R1 — détecteur read-only ; le garde bloquant `catalog-pr-guard.yml` est retiré (#11012). Le marqueur `advisory` du nom est ce qui rend le gate conforme (#15998 : sans lui, un échec d'infra comptait comme requis) | | `ci / Gitleaks secret scanner` | required | secrets-hygiene | | `hooks-parity gate` | required | #8782 — gate qui ne peut plus rougir | diff --git a/docs/reference/procedures-recurrentes.md b/docs/reference/procedures-recurrentes.md index b60c9b023e..07ec966b10 100644 --- a/docs/reference/procedures-recurrentes.md +++ b/docs/reference/procedures-recurrentes.md @@ -172,7 +172,7 @@ Toute PR touchant `COURSE_CATALOG.generated.json` rend les autres PRs catalog-to - Merger les PRs **non-catalogue d'abord**, puis les catalog-touchers **un-par-un**. - Conflit catalogue = l'auteur **rebase + régénère** : `python scripts/notebook_tools/generate_catalog.py --json --git-tracked-only` (parité CI = N entrées git-tracked ; `--json` nu inclut les `_output.ipynb` locaux = drift), puis `expand_catalog_markers.py`. - **JAMAIS** force-resolve le conflit côté coord ; **JAMAIS** force-push la branche de l'auteur. -- Check CI **rouge "Notebook catalog drift" = NON mergeable** (propagerait le drift) → bounce à l'auteur pour re-régénérer. +- Check CI **"Notebook catalog drift (read-only, advisory)"** : **advisory, jamais bloquant** (#15998) — le catalogue est réécrit par `catalog-cron.yml` sur `main` (#2632/#2744), une PR n'a rien à régénérer. Un drift signalé = notice informative, pas un bounce. - **Cascade-independence** : une PR dont le drift-check = SUCCESS **et** qui ne touche pas le catalogue est **indépendante** — ne pas la hold à tort dans la file cascade. ### Trap "APPROVED" diff --git a/scripts/tests/test_pr_gate.py b/scripts/tests/test_pr_gate.py index 9b7c05a19d..b62dc9dd66 100644 --- a/scripts/tests/test_pr_gate.py +++ b/scripts/tests/test_pr_gate.py @@ -906,6 +906,44 @@ def test_green_advisory_counts_as_a_normal_pass(): assert ok == ["Large blob advisory (>= 10 MiB)"] +def test_catalog_drift_job_name_carries_the_advisory_marker(): + """#15998 -- the catalog-drift job declared itself NON-BLOCKING twice in its + own header (and in catalog-pr-hygiene / ci-aggregator docs), but + `pr_gate.py` classifies advisory by NAME (rule 6, ADVISORY_MARKER) and never + reads `fast_lane_registry.py`. Named "Notebook catalog drift (read-only)", + the job carried no marker, so any infrastructure failure (runner, pip + install, `generate_catalog` exit 2 on missing git metadata, cf #14831) was + counted as a REQUIRED check and reddened every notebook/README PR -- + observed firsthand on #15996: "PR gate: FAIL -- failing checks: Notebook + catalog drift (read-only) (failure)" while every other check passed. + + The repair is the marker in the job name, not a registry entry: the + registry is consumed by the fast lane (which would then also RUN the + catalog generation on every PR), whereas `is_advisory` reads the emitted + check-run name. Asserted over whatever the job is called today -- so a + future rename that keeps the marker passes, and one that drops it fails. + """ + if pr_gate.yaml is None: # pragma: no cover - PyYAML is a CI dependency + pytest.skip("PyYAML unavailable: cannot read the workflow") + wf_path = Path(pr_gate.DEFAULT_WORKFLOWS_DIR) / "catalog-drift.yml" + data = pr_gate.yaml.safe_load(wf_path.read_text(encoding="utf-8")) + job_names = pr_gate._workflow_job_names(data) + assert job_names, "catalog-drift.yml must declare at least one job" + for name in job_names: + assert pr_gate.is_advisory(name), ( + "the catalog-drift job name must carry the `advisory` marker " + "(#15998): pr_gate.py reads the check-run name, not the header " + "comment, so dropping the marker silently re-arms a hard gate " + "against every PR touching a notebook or a series README" + ) + # The defect the marker fixes must stay measurable: with the workflow name + # as it stands, the historical spelling is still classified BLOCKING. If + # this ever flips, the workflow name has absorbed the marker and the + # job-name invariant above stopped being the load-bearing surface. + wf_name = data.get("name") or "" + assert not pr_gate.is_advisory("Notebook catalog drift (read-only)", wf_name) + + def test_non_advisory_failure_still_blocks(): """Guard against the fix becoming a blanket amnesty.""" checks = [ From 187102f4b11f026bafd3e88d3d27a174bc1c5447 Mon Sep 17 00:00:00 2001 From: jsboige Date: Sun, 13 Sep 2026 20:14:37 +0200 Subject: [PATCH 2/2] chore(control,#15998): throwaway control PR -- forced catalog-drift failure Injected forced failure + README trigger to observe how PR gate classifies the catalog-drift check on the fixed name. Throwaway: PR closed after observation. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/catalog-drift.yml | 231 +++++++++--------- .../Probas/DecisionTheory/README.md | 2 + 2 files changed, 120 insertions(+), 113 deletions(-) diff --git a/.github/workflows/catalog-drift.yml b/.github/workflows/catalog-drift.yml index fa61968804..8815a00cdd 100644 --- a/.github/workflows/catalog-drift.yml +++ b/.github/workflows/catalog-drift.yml @@ -1,113 +1,118 @@ -name: Catalog Drift Check - -# Purpose: DETECT (non-blocking, read-only) when COURSE_CATALOG.generated.json -# and the README CATALOG-STATUS markers drift out of sync with the notebooks. -# -# IMPORTANT — read-only since #2744 (2026-06-09): -# This workflow DETECTS drift and reports it as a notice annotation, but it -# NO LONGER COMMITS anything back to PR branches. The earlier design -# auto-committed a regenerated catalog onto every PR matching **/*.ipynb or -# **/README.md. Because those commits were bot-authored, catalog-pr-guard.yml -# (retiré depuis, #11012) exempted them — so they landed even when they mass-reverted curated git -# fields (issue_pr_associee, last_validation) via the branch git-history -# heuristic. That is the stale-catalog-silent-revert pattern (513 poisoned -# entries observed on branches, 0 on main — source-verified by po-2025, -# see #2744). It poisoned EVERY notebook/README PR and created a worker -# treadmill (strip -> bot re-commits on the next synchronize). -# -# Per decision #2632 (catalog cron-only, off-branch), the ONLY legitimate -# catalog writer is catalog-cron.yml on main (daily regen + push HEAD:main). -# The blocking guard catalog-pr-guard.yml was removed (#11012: phantom entity, -# 0 run on pull_request). This workflow stays as the read-only detector so -# reviewers see drift, without ever writing to a branch. -# -# This job is intentionally NON-BLOCKING and READ-ONLY. The REQUIRED notebook -# gate remains "Static validation (H.1/H.3/C.1)" in notebook-execution-required.yml. - -on: - pull_request: - branches: [ main ] - paths: - - 'MyIA.AI.Notebooks/**/*.ipynb' - - 'MyIA.AI.Notebooks/**/README.md' - - 'scripts/notebook_tools/generate_catalog.py' - - 'scripts/notebook_tools/expand_catalog_markers.py' - - 'COURSE_CATALOG.generated.json' - workflow_dispatch: - -permissions: - contents: read - -concurrency: - group: catalog-drift-${{ github.ref }} - cancel-in-progress: true - -jobs: - catalog-drift: - # Le marqueur `advisory` dans le NOM du job EST le contrat (#15998) : pr_gate.py - # classe par nom (`ADVISORY_MARKER`, regle 6) et ne lit PAS fast_lane_registry.py. - # Sans ce mot, la promesse "non-bloquant" de l'en-tete restait sans effet sur le - # gate : un echec d'infrastructure (runner, pip, generate_catalog) comptait comme - # un check requis et bloquait toute PR notebook/README -- observe sur #15996. - # Ne pas retirer le mot sans relire scripts/pr_gate.py (rule 6) et #15998. - name: "Notebook catalog drift (read-only, advisory)" - # Routage #14283 tranche 3 (decision ai-01 2026-09-02) : jambe Linux - # auto-hebergee. Le `if:` ci-dessous est la garde anti-fork exigee par - # scripts/ci/check_self_hosted_runner_policy.py ; un `runs-on` STATIQUE la - # rend auditable (une expression dynamique leve DYNAMIC_RUNS_ON). - # Les PRs de fork sautent le job -- pr_gate.py compte `skipped` comme OK. - # Retour arriere = remettre `runs-on: ubuntu-latest` + retrait de l'allowlist. - runs-on: [self-hosted, coursia-ephemeral, coursia-linux] - if: github.event.pull_request.head.repo.full_name == null || github.event.pull_request.head.repo.full_name == github.repository - - steps: - # Read-only checkout of the PR merge ref. We only generate the catalog and - # diff it against the committed copy to detect drift; we never commit or - # push back to a branch (see #2744). The default merge-ref checkout works - # for both same-repo and fork PRs. - - name: Checkout PR - uses: actions/checkout@v4 - with: - fetch-depth: 0 - filter: blob:none - - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: '3.11' - - # PyYAML is required by generate_catalog.py's editorial-review registry - # loader (_load_editorial_registry, added in #8345). Without it the - # generator crashes: ModuleNotFoundError: No module named 'yaml'. - - name: Install PyYAML - run: pip install pyyaml - - # Canonical regeneration (local to the runner only — nothing is written - # back to the branch). git-tracked notebooks only, for CI determinism. - - name: Regenerate catalog + README markers - run: | - git fetch origin main --depth=1 || true - python scripts/notebook_tools/generate_catalog.py --json-only --git-tracked-only - python scripts/notebook_tools/expand_catalog_markers.py - - - name: Detect drift - id: diff - run: | - # Only the catalog JSON and the README marker files are in scope. - git add COURSE_CATALOG.generated.json MyIA.AI.Notebooks/README.md 'MyIA.AI.Notebooks/*/README.md' 2>/dev/null || true - if git diff --cached --quiet; then - echo "drift=false" >> "$GITHUB_OUTPUT" - echo "Catalog and README markers are already in sync. No drift." - else - echo "drift=true" >> "$GITHUB_OUTPUT" - echo "Catalog drift detected in:" - git diff --cached --name-only | sed 's/^/ - /' - echo "::notice title=Catalog drift (informational)::The catalog/README markers are out of sync with the notebooks. This does NOT block the PR. The catalog is regenerated daily on main by catalog-cron.yml; no manual action is required on this branch." - fi - - # This job is always green. Drift is surfaced as a notice annotation only. - # catalog-cron.yml (main) is the sole writer; the blocking guard - # catalog-pr-guard.yml was removed (#11012) — this read-only detector and - # the catalog-cron.yml regen PR are the sole organs now (see #2632, #2744). - - name: Done (non-blocking, read-only) - run: echo "Catalog drift check complete (non-blocking, read-only)." +name: Catalog Drift Check + +# Purpose: DETECT (non-blocking, read-only) when COURSE_CATALOG.generated.json +# and the README CATALOG-STATUS markers drift out of sync with the notebooks. +# +# IMPORTANT — read-only since #2744 (2026-06-09): +# This workflow DETECTS drift and reports it as a notice annotation, but it +# NO LONGER COMMITS anything back to PR branches. The earlier design +# auto-committed a regenerated catalog onto every PR matching **/*.ipynb or +# **/README.md. Because those commits were bot-authored, catalog-pr-guard.yml +# (retiré depuis, #11012) exempted them — so they landed even when they mass-reverted curated git +# fields (issue_pr_associee, last_validation) via the branch git-history +# heuristic. That is the stale-catalog-silent-revert pattern (513 poisoned +# entries observed on branches, 0 on main — source-verified by po-2025, +# see #2744). It poisoned EVERY notebook/README PR and created a worker +# treadmill (strip -> bot re-commits on the next synchronize). +# +# Per decision #2632 (catalog cron-only, off-branch), the ONLY legitimate +# catalog writer is catalog-cron.yml on main (daily regen + push HEAD:main). +# The blocking guard catalog-pr-guard.yml was removed (#11012: phantom entity, +# 0 run on pull_request). This workflow stays as the read-only detector so +# reviewers see drift, without ever writing to a branch. +# +# This job is intentionally NON-BLOCKING and READ-ONLY. The REQUIRED notebook +# gate remains "Static validation (H.1/H.3/C.1)" in notebook-execution-required.yml. + +on: + pull_request: + branches: [ main ] + paths: + - 'MyIA.AI.Notebooks/**/*.ipynb' + - 'MyIA.AI.Notebooks/**/README.md' + - 'scripts/notebook_tools/generate_catalog.py' + - 'scripts/notebook_tools/expand_catalog_markers.py' + - 'COURSE_CATALOG.generated.json' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: catalog-drift-${{ github.ref }} + cancel-in-progress: true + +jobs: + catalog-drift: + # Le marqueur `advisory` dans le NOM du job EST le contrat (#15998) : pr_gate.py + # classe par nom (`ADVISORY_MARKER`, regle 6) et ne lit PAS fast_lane_registry.py. + # Sans ce mot, la promesse "non-bloquant" de l'en-tete restait sans effet sur le + # gate : un echec d'infrastructure (runner, pip, generate_catalog) comptait comme + # un check requis et bloquait toute PR notebook/README -- observe sur #15996. + # Ne pas retirer le mot sans relire scripts/pr_gate.py (rule 6) et #15998. + name: "Notebook catalog drift (read-only, advisory)" + # Routage #14283 tranche 3 (decision ai-01 2026-09-02) : jambe Linux + # auto-hebergee. Le `if:` ci-dessous est la garde anti-fork exigee par + # scripts/ci/check_self_hosted_runner_policy.py ; un `runs-on` STATIQUE la + # rend auditable (une expression dynamique leve DYNAMIC_RUNS_ON). + # Les PRs de fork sautent le job -- pr_gate.py compte `skipped` comme OK. + # Retour arriere = remettre `runs-on: ubuntu-latest` + retrait de l'allowlist. + runs-on: [self-hosted, coursia-ephemeral, coursia-linux] + if: github.event.pull_request.head.repo.full_name == null || github.event.pull_request.head.repo.full_name == github.repository + + steps: + # Read-only checkout of the PR merge ref. We only generate the catalog and + # diff it against the committed copy to detect drift; we never commit or + # push back to a branch (see #2744). The default merge-ref checkout works + # for both same-repo and fork PRs. + - name: Checkout PR + uses: actions/checkout@v4 + with: + fetch-depth: 0 + filter: blob:none + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + # PyYAML is required by generate_catalog.py's editorial-review registry + # loader (_load_editorial_registry, added in #8345). Without it the + # generator crashes: ModuleNotFoundError: No module named 'yaml'. + - name: Install PyYAML + run: pip install pyyaml + + # Canonical regeneration (local to the runner only — nothing is written + # back to the branch). git-tracked notebooks only, for CI determinism. + - name: Regenerate catalog + README markers + run: | + git fetch origin main --depth=1 || true + python scripts/notebook_tools/generate_catalog.py --json-only --git-tracked-only + python scripts/notebook_tools/expand_catalog_markers.py + + - name: Detect drift + id: diff + run: | + # Only the catalog JSON and the README marker files are in scope. + git add COURSE_CATALOG.generated.json MyIA.AI.Notebooks/README.md 'MyIA.AI.Notebooks/*/README.md' 2>/dev/null || true + if git diff --cached --quiet; then + echo "drift=false" >> "$GITHUB_OUTPUT" + echo "Catalog and README markers are already in sync. No drift." + else + echo "drift=true" >> "$GITHUB_OUTPUT" + echo "Catalog drift detected in:" + git diff --cached --name-only | sed 's/^/ - /' + echo "::notice title=Catalog drift (informational)::The catalog/README markers are out of sync with the notebooks. This does NOT block the PR. The catalog is regenerated daily on main by catalog-cron.yml; no manual action is required on this branch." + fi + + # This job is always green. Drift is surfaced as a notice annotation only. + # catalog-cron.yml (main) is the sole writer; the blocking guard + # catalog-pr-guard.yml was removed (#11012) — this read-only detector and + # the catalog-cron.yml regen PR are the sole organs now (see #2632, #2744). + # CONTROL #15998 -- etape d'echec forcee, branche jetable (PR de controle). + - name: "CONTROL #15998 -- forced failure" + run: | + echo "Echec force pour observer la classification du check par PR gate (#15998)." + exit 1 + - name: Done (non-blocking, read-only) + run: echo "Catalog drift check complete (non-blocking, read-only)." diff --git a/MyIA.AI.Notebooks/Probas/DecisionTheory/README.md b/MyIA.AI.Notebooks/Probas/DecisionTheory/README.md index dc1d89864c..b113e6e861 100644 --- a/MyIA.AI.Notebooks/Probas/DecisionTheory/README.md +++ b/MyIA.AI.Notebooks/Probas/DecisionTheory/README.md @@ -108,3 +108,5 @@ Chaque sous-série démarre par ses fondations (`DecInfer-01` / `DecPyMC-1`) et --- *Cet arc fait partie de la série [Probas](../README.md) du dépôt [CoursIA](https://github.com/jsboige/CoursIA). Documentation détaillée de chaque notebook dans les README des sous-séries.* + +