Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 130 additions & 14 deletions .github/workflows/markdown-table-guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,39 @@ name: Markdown table syntax advisory
# Tranche 2 #12817: the per-PR fork guard died with the pull_request trigger
# (false under schedule -> job SKIPPED, organ extinguished). The nocturne
# scans the last-24h window of main; labels are PR-only and noop'd at night.
#
# #16207 -- the PR-time trigger is BACK, without the cost that killed it.
# #12817 tranche 1 removed pull_request because each run cloned the full
# 2.22 Go working tree (stratification user 2026-08-23: "les jobs lourds
# devraient être payés une fois par fournée"). The founding incident that
# reopened the question: #16177 -- a CODE_SPAN_PIPE in the intro table
# merged with NO signal; the first pair of eyes on it was a post-merge
# human comment. The re-introduction keeps the stratification honest:
# - `paths` fires ONLY for the file trees this guard scans (*.ipynb /
# *.md / *README*), plus its own wiring and the two scripts it runs;
# - the checkout materializes the scan toolbox ONLY (blob:none partial
# clone), then the changed files' blobs on demand via
# `git sparse-checkout add --no-cone` -- never the full working tree.
# Marginal cost per PR: one metadata clone + the changed files' blobs + a
# pure-Python scan, on the self-hosted ephemeral leg. The nocturne is
# unchanged (it also scans only the last-24h window's changed files).
# Rejected alternative (arbitrage consigné): re-housing the scan as a leg
# of always-on-guards.yml (mutualized checkout) -- rejected for blast
# radius: coupling an advisory labeler into the critical-path gate organs
# changes their failure surface for a signal that must stay advisory.

on:
pull_request:
# #16207 : le signal PR-time. Filtre paths = les seuls arbres que la
# garde scanne (elle ne scanne jamais le reste -- voir "Per-PR scope"
# ci-dessus) + son propre câblage + les deux scripts qu'elle exécute.
paths:
- '**/*.ipynb'
- '**/*.md'
- '**/*README*'
- '.github/workflows/markdown-table-guard.yml'
- 'scripts/notebook_tools/scan_md_table_syntax.py'
- 'scripts/notebook_tools/md_table_sweep_comment.py'
schedule:
# Nocturnal sweep -- post-merge main verification.
# Tranche 1 #12817 : sortir les advisory lourds de pull_request
Expand Down Expand Up @@ -65,11 +96,20 @@ jobs:
if: github.event.pull_request.head.repo.full_name == null || github.event.pull_request.head.repo.full_name == github.repository

steps:
- name: Checkout PR
- name: Checkout PR (metadata + scan toolbox, never the full tree)
uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
# #16207 : le cout qui a tue le trigger etait l'arbre de travail
# complet (2.22 Go). Ici on ne materialise QUE la boite a outils du
# scan ; les blobs des fichiers modifies arrivent plus bas, a la
# demande, via `git sparse-checkout add --no-cone` sur changed.txt.
# fetch-depth: 0 reste requis : le diff 3-points BASE...HEAD lit le
# merge-base, et le nocturne resout la fenetre 24 h par rev-list.
sparse-checkout: |
/scripts/notebook_tools/
sparse-checkout-cone-mode: false

- name: Set up Python
uses: actions/setup-python@v5
Expand Down Expand Up @@ -106,16 +146,25 @@ jobs:
# excluded). Skip vendored / archived trees where we do not enforce
# this (.lake/ Mathlib, docs/_archives/, node_modules).
# 3-point diff (#10403): merge-base...HEAD -- strictly this PR's apport.
git diff --name-only --diff-filter=d "$BASE...$HEAD" \
# CR #16207 : -z (enregistrements NUL-separes) + grep -z partout. Un
# split whitespace casserait chaque nom a espaces du depot ('Créateur
# de mail personnalisé.ipynb', 'Conférence Tech 2025', 'Correction
# Activités GenAI.md', ...) en argv orphelins -> 0 fichier scannable
# -> exit 2 avale -> faux "Clean." (voir controle positif plus bas).
git diff -z --name-only --diff-filter=d "$BASE...$HEAD" \
-- '*.ipynb' '*.md' '*README*' \
| grep -v -E '^(\.lake/|docs/_archives/|node_modules/)' \
| grep -v $'\r$' > changed.txt || true
COUNT=$(wc -l < changed.txt | tr -d ' ')
| grep -z -v -E '^(\.lake/|docs/_archives/|node_modules/)' \
| grep -z -v $'\r$' > changed.txt || true
COUNT=$(tr -cd '\0' < changed.txt | wc -c | tr -d ' ')
echo "Modified *.ipynb/*.md/README* in scope: $COUNT"

# Label helpers (idempotent).
ensure_label() {
gh label create "$1" --description "$2" --color "$3" --force 2>/dev/null || true
# #16207 : pas de 2>/dev/null -- le create a echoue en 422
# (description > 100 car.) en silence pendant des semaines parce
# que l'echec etait enterre ; stderr reste visible dans le log,
# seul l'exit reste non bloquant (|| true, advisory).
gh label create "$1" --description "$2" --color "$3" --force || true
}
set_label() { gh pr edit "$PR_NUMBER" --add-label "$1" || true; }
unset_label() { gh pr edit "$PR_NUMBER" --remove-label "$1" 2>/dev/null || true; }
Expand All @@ -126,23 +175,89 @@ jobs:
unset_label() { true; }
fi

ensure_label "$LABEL_DEFECT" "PR introduces a markdown table syntax defect (COL_MISMATCH / CODE_SPAN_PIPE / MATH_SPAN_PIPE / NO_SEP / NO_BLANK_BEFORE/AFTER / ORPHAN_TABLE_ROW). Advisory, non-blocking. See #10097." "d93f0b"
# #16207 : description <= 100 caracteres (limite API GitHub -- le
# texte long d'origine rendait le create en 422 a chaque run,
# silencieusement, et le label n'a jamais existe).
ensure_label "$LABEL_DEFECT" "Table syntax defect in changed files (CODE_SPAN_PIPE, NO_SEP, ...). Advisory. See #10097." "d93f0b"

# ---- Controle positif CR #16207 : nom in-scope avec espaces ----
# Tourne a CHAQUE run (meme sans fichier in-scope dans la PR, meme
# en nocturne) : il gate la FIABILITE de la mesure, pas le contenu.
# Le fichier "md-table controle.md" porte trois lignes pipe sans
# separateur (NO_SEP) et doit rendre total >= 1. Un split
# whitespace a l'ancienne casserait ce path en argv orphelins ->
# exit 2 -> controle rouge. Il ne nourrit jamais le label.
SCAN_RC=0
CONTROL="$RUNNER_TEMP/md-table controle.md"
printf '| a | b |\n| c | d |\n| e | f |\n' > "$CONTROL" || true
python scripts/notebook_tools/scan_md_table_syntax.py --json "$CONTROL" > control.json 2>/dev/null
CTRL_RC=$?
CTRL_TOTAL=0
if [ "$CTRL_RC" -eq 0 ]; then
CTRL_TOTAL=$(python -c "import json,sys; print(json.load(open('control.json'))['total_findings'])" 2>/dev/null) || true
fi
if [ "${CTRL_TOTAL:-0}" -lt 1 ]; then
echo "::error::positive control failed (rc=$CTRL_RC, total=$CTRL_TOTAL) -- whitespace argv is lossy, measurement unreliable"
SCAN_RC=1
else
echo "Positive control passed (whitespace filename fed to argv: $CTRL_TOTAL NO_SEP finding)"
fi
rm -f "$CONTROL" control.json

if [ "$COUNT" -eq 0 ]; then
echo "No in-scope files to scan."
unset_label "$LABEL_DEFECT"
# CR #16207 : meme sans fichier in-scope, un controle de mesure
# echoue (SCAN_RC != 0) interdit l'unset -- le rouge reste.
if [ "$SCAN_RC" -eq 0 ]; then
unset_label "$LABEL_DEFECT"
else
echo "::warning::measurement control failed -- leaving '$LABEL_DEFECT' label untouched"
fi
exit 0
fi

# Advisory: the job ALWAYS exits 0. The actionable signal is the
# LABEL decided from the JSON total below.
python scripts/notebook_tools/scan_md_table_syntax.py $(cat changed.txt) --json > payload.json || true
cat payload.json
# #16207 : materialiser les cibles du scan -- et elles seules. Le
# checkout d'entree est sparse (boite a outils uniquement) ; le diff
# --name-only ci-dessus n'a lu que les arbres (blob:none suffit) ;
# les blobs des fichiers modifies arrivent ici via le promissor du
# clone partiel. add (pas set) : le pattern outils pose par
# actions/checkout doit survivre a l'ajout. Les PATHS sont les argv
# OCTETS-EXACTS du fichier NUL-separe (mapfile -d ''), jamais une
# resubstitution shell (CR #16207).
mapfile -d '' PATHS < changed.txt
git sparse-checkout add --no-cone "${PATHS[@]}"

TOTAL=$(python -c "import json,sys; print(json.load(open('payload.json'))['total_findings'])" 2>/dev/null || echo 0)
echo "Total table-syntax defects in changed files: $TOTAL"
# CR #16207 : le scanner rend exit 2 ("rien a scanner") quand aucun
# path argv ne nomme un fichier existant -- ce qui arrivait en
# silence apres le split whitespace. RC explicite : un echec du
# scanner = PANNE DE MESURE (total illisible), jamais un "Clean.".
python scripts/notebook_tools/scan_md_table_syntax.py --json "${PATHS[@]}" > payload.json 2> scan.err
SRC=$?
if [ "$SRC" -ne 0 ]; then
echo "::error::scan_md_table_syntax.py failed (rc=$SRC) -- $LABEL_DEFECT NOT touched"
if [ -s scan.err ]; then cat scan.err; fi
SCAN_RC=1
elif [ "$SCAN_RC" -ne 0 ]; then
echo "::error::positive control failed -- total unreliable, $LABEL_DEFECT NOT touched"
else
cat payload.json
TOTAL=$(python -c "import json,sys; print(json.load(open('payload.json'))['total_findings'])" 2>/dev/null) || true
echo "Total table-syntax defects in changed files: $TOTAL"
# Payload lisible en rc mais total non numerique = panne de
# mesure, jamais un "Clean.".
case "$TOTAL" in
''|*[!0-9]*) echo "::error::payload unreadable (total='$TOTAL') -- $LABEL_DEFECT NOT touched"; SCAN_RC=1 ;;
esac
fi

if [ "$TOTAL" -gt 0 ]; then
# Label decision. CR #16207 : jamais d'unset sur un etat non mesure
# (SCAN_RC != 0) -- le label pose par un run precedent reste en
# place, le rouge vaut mieux qu'un faux "Clean.".
if [ "$SCAN_RC" -ne 0 ]; then
echo "::warning::table-syntax scan unreliable -- leaving '$LABEL_DEFECT' label untouched"
elif [ "$TOTAL" -gt 0 ]; then
set_label "$LABEL_DEFECT"
if [ "$NOCTURNE" -eq 1 ]; then
echo "::warning::Found $TOTAL markdown table syntax defect(s) in the last-24h window on main. See scan_md_table_syntax.py --check locally."
Expand All @@ -162,7 +277,8 @@ jobs:
# de tag, pas de fermeture, pas de blocage de merge -- on route, ai-01
# tranche. apply : schedule toujours ; workflow_dispatch seulement si
# apply=true (defaut dry-run, le corps s'imprime pour controle).
if [ "$NOCTURNE" -eq 1 ]; then
# CR #16207 : pas de sweep sur un payload illisible (SCAN_RC != 0).
if [ "$NOCTURNE" -eq 1 ] && [ "$SCAN_RC" -eq 0 ]; then
WINDOW_DESC="$(git rev-parse --short "$BASE")..$(git rev-parse --short "$HEAD") (24 h de main)"
EXTRA=""
if [ "${{ github.event_name }}" = "schedule" ] || [ "${{ inputs.apply }}" = "true" ]; then
Expand Down
Loading