Skip to content

tooling(#15405): detecteur paragraphes markdown trop longs + workflow advisory #15455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
myia-ai-01 merged 2 commits into from
Sep 11, 2026
Merged

tooling(#15405): detecteur paragraphes markdown trop longs + workflow advisory #15455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b8bd1df
tooling(#15405): detecteur paragraphes markdown trop longs + workflow…
jsboige Sep 10, 2026
43b6f5b
fix(ci,#15405): admettre paragraph-length-advisory.yml a l'allowlist …
jsboige Sep 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions .github/workflows/paragraph-length-advisory.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
name: Paragraph Length Advisory

# The organ for "wall-of-text markdown paragraphs" (incident 2026-09-10 :
# the README Probas shipped a 3336-char single-line paragraph in PR #15405
# commit 76d7a5bc, no guard caught it). The signal: any markdown prose
# paragraph > 2000 chars on a *single* run of non-empty lines, after
# stripping code fences / tables / headings / HTML comments (CATALOG-STATUS).
#
# ADVISORY, never blocking at this stage (decision user 2026-09-10). The
# job ALWAYS exits 0. The actionable payload is the `paragraph-length` LABEL,
# NEVER the green conclusion of the job. Promote to blocking once the
# residual corpus is at 0 and stable (post-sweep issue de suivi).
#
# Per-PR scope: scans ONLY the *.md files modified by the PR, not a
# repo-wide scan on every push. Self-cover (#8822) lists the workflow
# itself in `paths:`.
#
# Calibration 2026-09-10 (rglob, 791 *.md, 23049 paragraphs) :
# p50=79 p75=274 p90=536 p95=760 p99=1437 max=13409
# > 2000 c : 42 fichiers / 84 paragraphes (hors _archives, vendored)
# These 42 pre-existing files are NOT corrected here -- sweep is its own
# follow-up issue, tracked separately. This PR only adds the organ that
# SHOULD HAVE CAUGHT the Probas paragraph in #15405.

on:
pull_request:
types: [opened, synchronize, reopened]
branches: [main]
paths:
- '**/*.md'
# Self-cover (#8822): a paths-filtered label-poser must list its own file,
# else it cannot re-run (so cannot remove its own label) once the matching
# paths leave the PR diff.
- '.github/workflows/paragraph-length-advisory.yml'
workflow_dispatch:

permissions:
contents: read
pull-requests: write

concurrency:
group: paragraph-length-advisory-${{ github.ref }}
cancel-in-progress: true

jobs:
paragraph-length-advisory:
name: "Paragraph length > 2000 chars advisory (label, non-blocking)"
runs-on: [self-hosted, coursia-ephemeral, coursia-linux]
if: github.event.pull_request.head.repo.full_name == github.repository

steps:
- name: Checkout PR
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'

- name: Advisory check on modified markdown files
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
LABEL_OVERSIZED: paragraph-length
LABEL_UNMEASURED: paragraph-length-unmeasured
run: |
set -uo pipefail

# 3-point diff (#10403): merge-base...HEAD -- strictly this PR's apport.
git diff --name-only --diff-filter=d "$BASE...$HEAD" -- '*.md' \
| grep -v $'\r$' > changed_md.txt || true
COUNT=$(wc -l < changed_md.txt | tr -d ' ')
echo "Modified *.md in this PR: $COUNT"

ensure_label() {
gh label create "$1" --description "$2" --color "$3" --force 2>/dev/null || true
}
set_label() { gh pr edit "$PR_NUMBER" --add-label "$1" || true; }
unset_label() { gh pr edit "$PR_NUMBER" --remove-label "$1" 2>/dev/null || true; }

if [ "$COUNT" -eq 0 ]; then
echo "No modified markdown to check."
unset_label "$LABEL_OVERSIZED"; unset_label "$LABEL_UNMEASURED"
exit 0
fi

# Advisory: ALWAYS exit 0. Job never fails; label is the signal.
# Detector returns rc 0/1/2 per file, but we don't propagate exit
# code here -- the JSON payload is what matters.
python scripts/notebook_tools/detect_paragraph_length.py \
--json $(cat changed_md.txt) > payload.json 2>&1 || true
cat payload.json

# Payload shape: {files: [{file, findings, counts, error?}], summary}
OVERSZ_COUNT=$(python -c "
import json
try:
d = json.load(open('payload.json'))
print(d['summary']['total_findings'])
except Exception:
print('UNREADABLE')
" 2>/dev/null || echo "UNREADABLE")
FLAG_COUNT=$(python -c "
import json
try:
d = json.load(open('payload.json'))
print(d['summary']['flagged_count'])
except Exception:
print('?')
" 2>/dev/null || echo "?")
echo "Oversized paragraphs across modified md: $OVERSZ_COUNT"
echo "Flagged files: $FLAG_COUNT"

ensure_label "$LABEL_OVERSIZED" \
"A modified markdown file carries a paragraph > 2000 chars (wall-of-text, see #15405 incident). Resorb before merge." "FBCA04"
ensure_label "$LABEL_UNMEASURED" \
"A modified markdown file could not be measured by detect_paragraph_length.py -- NOT verified (#8819)." "BFD4F2"

# #8819 guard: if the payload is unreadable, never claim green.
if [ "${OVERSZ_COUNT:-0}" = "UNREADABLE" ] 2>/dev/null; then
echo "::error::paragraph-length payload illisible -- gate measured NOTHING; conformity neither claimed nor denied."
set_label "$LABEL_UNMEASURED"
exit 0
fi

if [ "${OVERSZ_COUNT:-0}" -gt 0 ] 2>/dev/null; then
echo "::notice::$OVERSZ_COUNT oversized paragraph(s) across $FLAG_COUNT modified markdown file(s). See the '$LABEL_OVERSIZED' label."
set_label "$LABEL_OVERSIZED"
else
unset_label "$LABEL_OVERSIZED"
fi

echo "RESULT: $OVERSZ_COUNT oversized paragraph(s) across $FLAG_COUNT modified markdown file(s)."
exit 0
13 changes: 13 additions & 0 deletions scripts/ci/check_self_hosted_runner_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,19 @@
# advisory.yml). Rollback = revert de la PR (l'entree disparait de
# l'allowlist).
"translation-hot-drift-advisory.yml",
# #15405 (decision ai-01 2026-09-11, owner myia-ai-01:CoursIA) : garde
# advisory pur-Python sur les paragraphes markdown depassant 2000
# caracteres (detect_paragraph_length.py, stdlib-only). Declencheur
# pull_request filtrant `**/*.md` + auto-couverture du workflow (#8822,
# sans quoi le poseur de label ne peut plus retirer son propre label une
# fois les chemins sortis du diff). Pose un label signe, jamais exit != 0.
# `pull-requests: write` + GITHUB_TOKEN limites a l'API labels -- meme
# profil que markdown-deaccent-advisory.yml / repeated-prose-advisory.yml,
# deja admis a ce titre. Garde same-repo au niveau job (#13874) : les PRs
# de fork se font skipper proprement par pr_gate. Runner = jambe Linux
# containerisee (LINUX_RUNNER_LABELS). Rollback = revert de la PR
# (l'entree disparait de l'allowlist).
"paragraph-length-advisory.yml",
}
GITHUB_HOSTED_LABELS = {
"ubuntu-latest",
Expand Down
36 changes: 35 additions & 1 deletion scripts/notebook_tools/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ complement. L'inventaire ci-dessous remplace la lecture en aveugle de

| Categorie | Scripts | Role |
|-----------|---------|------|
| **Detecteurs anti-regression** | `detect_blank_figures.py`, `detect_fabricated_outputs.py`, `detect_svg_decimal_commas.py`, `detect_svg_empty_display.py`, `detect_ascii_workaround.py`, `detect_accent_stripping.py`, `detect_link_target_regression.py`, `detect_solution_leaks.py`, `detect_cjk_residue.py` | Flags deterministes par regle C.1 / H.1 / SOTA / #2876 (axe-1 texte + **axe-3 link-targets** triade) / #3801 / #4970 / **#6927** (SVG inline rollout) / **#6891 axe-2 fabrication textuelle** (sibling detector) / **#8428** (CJK LLM-translation residue, regression-guard post-fleet-sweep) |
| **Detecteurs anti-regression** | `detect_blank_figures.py`, `detect_fabricated_outputs.py`, `detect_svg_decimal_commas.py`, `detect_svg_empty_display.py`, `detect_ascii_workaround.py`, `detect_accent_stripping.py`, `detect_link_target_regression.py`, `detect_solution_leaks.py`, `detect_cjk_residue.py`, `detect_paragraph_length.py` | Flags deterministes par regle C.1 / H.1 / SOTA / #2876 (axe-1 texte + **axe-3 link-targets** triade) / #3801 / #4970 / **#6927** (SVG inline rollout) / **#6891 axe-2 fabrication textuelle** (sibling detector) / **#8428** (CJK LLM-translation residue, regression-guard post-fleet-sweep) / **#15405** (paragraphes markdown > 2000 c, wall-of-text guard) |
| **Validateurs CI** | `validate_pr_notebooks.py`, `check_c2_compliance.py`, `check_notebook_navlinks.py`, `check_plotly_static_risk.py` | Gates pre-merge, `--check` exit-code CI-ready |
| **Scanners structurels** | `scan_cell_ordering.py`, `scan_md_hierarchy.py`, `scan_figure_visual_signature.py` | Audit hierarchie markdown + ordre cellules pedagogiques + **signature visuelle des figures PNG (consolidation L777-L1/L778-L1/L2/L779-L1/L2/L780-L1/L2/L3/L781-L1/L2/L3 du rollout MANIFEST c.754-c.781, EPIC #5780)** |
| **Execution kernels** | `dotnet_executor.py`, `exec_dotnet_persist.py`, `exec_single_cell.py`, `batch_reexecute.py`, `wsl_papermill.py` | .NET Interactive + Python Papermill via WSL |
Expand Down Expand Up @@ -253,6 +253,40 @@ Baseline c.884 : 937 notebooks, **0 residu inattendu**, 2 allowed (fleet clean
post-sweep). Le guard n'empeche que la recidive ; la correction d'un nouveau
residu reste byte-surgical par notebook (cf #8428 fix pattern).

### `detect_paragraph_length.py` (#15405, organe anti « wall-of-text »)

Detecteur de paragraphes markdown trop longs : un seul bloc contigu de
lignes non-vides > **2000 caracteres** est signale avec sa position et
son extrait. Ignore les fences code (``` / ~~~), les lignes de tableau
(`|`), les titres (`#` ... `######`), les commentaires HTML (`<!--
... -->` dont le marqueur CATALOG-STATUS) et les directives Sphinx.
Listes et blockquotes comptent (un item de liste de 10k c est un mur).

Incident fondateur : le README Probas (PR #15405, commit `76d7a5bc`,
remarque user 2026-09-10) livrait un paragraphe unique de **3336 c / 24
phrases sur une seule ligne physique**. Aucun garde CI ne le detectait.

Calibration 2026-09-10 (rglob sur 791 `*.md` / 23049 paragraphes) :
p50=79, p75=274, p90=536, p95=760, p99=1437, max=13409. Le seuil 2000
capture l'incident avec marge et signale 42 fichiers / 84 paragraphes
(les autres sont de la prose technique legitime, sweeps a venir).

```bash
python scripts/notebook_tools/detect_paragraph_length.py README.md # human-readable
python scripts/notebook_tools/detect_paragraph_length.py --json README.md # dict agrege CI-ready
python scripts/notebook_tools/detect_paragraph_length.py --self-test # temoin fondateur tire
python scripts/notebook_tools/detect_paragraph_length.py README.md --fail-on-findings
# exit 2 si au moins un paragraphe > 2000 c
```

Câblé dans `.github/workflows/paragraph-length-advisory.yml` (advisory,
label `paragraph-length`, jamais bloquant a ce stade). Bascule
bloquante = PR dediee apres que le sweep de resorption sur les 42
fichiers pre-existants soit a zero.

**Owner** : partition-mienne pour les PRs docs (relecture fichier-entier
README series), cluster-manager pour la bascule bloquante.

---

## Triade accent #2876 — défense outillée (3 axes complémentaires)
Expand Down
Loading
Loading