diff --git a/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47-PainAxisDistillation-Python.ipynb b/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47-PainAxisDistillation-Python.ipynb index 206cd0b84f..1f76ab5e62 100644 --- a/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47-PainAxisDistillation-Python.ipynb +++ b/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47-PainAxisDistillation-Python.ipynb @@ -15,7 +15,7 @@ }, "source": [ "# ICT-47 — L'axe douleur : une direction linéaire dans l'espace latent du substrat ?\n", - "**Navigation** : [Index](README.md) | [46 — Strate 7 : freebits de second ordre](ICT-46-Strate7-FreeCoordinates-Python.ipynb)\n", + "**Navigation** : [Index](README.md) | [46 — Strate 7 : freebits de second ordre](ICT-46-Strate7-FreeCoordinates-Python.ipynb) | [47b — L'instrument change-t-il la géométrie ?](ICT-47b-PainAxisInstrument-Python.ipynb)\n", "\n", "Cette séance instruit la question posée par la lignée Pain Axis (Tagliabue, Dung & Berg,\n", "arXiv 2609.16247) : **la « douleur » est-elle une direction linéaire unique dans le\n", diff --git a/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47b-PainAxisInstrument-Python.ipynb b/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47b-PainAxisInstrument-Python.ipynb new file mode 100644 index 0000000000..88b65a1828 --- /dev/null +++ b/MyIA.AI.Notebooks/IIT/ICT-Series/ICT-47b-PainAxisInstrument-Python.ipynb @@ -0,0 +1,3586 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "p47b-title", + "metadata": { + "papermill": { + "duration": 0.003984, + "end_time": "2026-10-08T13:00:12.246355+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:12.242371+00:00", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "# ICT-47b — L'instrument change-t-il la géométrie ?\n", + "**Navigation** : [Index](README.md) | [47 — L'axe douleur](ICT-47-PainAxisDistillation-Python.ipynb)\n", + "\n", + "**Sibling d'ICT-47**, même famille de substrats (Qwen3.5-**Base**), même corpus, même pipeline.\n", + "Une seule chose change : le **nombre de bits** des poids.\n", + "\n", + "Ce carnet existe parce qu'ICT-47 a laissé une question ouverte que ses propres *Limites* nomment —\n", + "« au-delà du 9B, rien n'est mesuré ; deux tailles ne suffisent pas à conclure à une invariance\n", + "d'échelle ». Y répondre demande une échelle supérieure ; l'atteindre sur cette machine demande de\n", + "**quantifier**. Or changer l'instrument en même temps que l'échelle rendrait la courbe\n", + "ininterprétable : l'effet cherché et l'effet de l'instrument deviendraient indiscernables.\n", + "\n", + "Ce carnet paie donc la première moitié du prix : il **mesure l'instrument**, et il **mesure la\n", + "barrière** qui empêche la seconde." + ] + }, + { + "cell_type": "markdown", + "id": "p47b-plan", + "metadata": { + "papermill": { + "duration": 0.003386, + "end_time": "2026-10-08T13:00:12.253930+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:12.250544+00:00", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## Plan\n", + "\n", + "1. Le corpus — hérité verbatim d'ICT-47 (9 groupes × 24 énoncés)\n", + "2. **L'instrument** — la même mesure sous bf16, int8 et 4-bit NF4, à deux échelles\n", + "3. Robustesse à l'échelle atteignable — profondeur et graine de partition (9B)\n", + "4. **La barrière** — ce que cette machine refuse de charger, et la preuve\n", + "5. Verdict et limites" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "p47b-preamble", + "metadata": { + "execution": { + "iopub.execute_input": "2026-10-08T13:00:12.262697Z", + "iopub.status.busy": "2026-10-08T13:00:12.262102Z", + "iopub.status.idle": "2026-10-08T13:00:18.486749Z", + "shell.execute_reply": "2026-10-08T13:00:18.485879Z" + }, + "papermill": { + "duration": 6.230645, + "end_time": "2026-10-08T13:00:18.488200+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:12.257555+00:00", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "torch 2.8.0+cu126 cuda True\n", + "determinisme actif ; TF32 off ; barres tqdm coupees (batch)\n" + ] + } + ], + "source": [ + "import os, sys, json, math, time\n", + "os.environ.setdefault(\"HF_HOME\", \"D:/hf-cache-crosscoder\")\n", + "# huggingface_hub avertit a chaque chargement que le cache n'est pas en symlinks ; sur\n", + "# Windows cet avertissement imprime le chemin site-packages ET le chemin du cache dans\n", + "# les sorties. Bruit hors propos pedagogique -- et chemins machine qui n'ont rien a faire\n", + "# dans un artefact committe (ratchet Output-failure) : coupe a la source, avant tout\n", + "# import HF.\n", + "os.environ.setdefault(\"HF_HUB_DISABLE_SYMLINKS_WARNING\", \"1\")\n", + "# Meme geste pour deux artefacts du chargement batch (kernel IPC, sans ipywidgets) :\n", + "# tqdm.auto -- importe par transformers -- avertit \"IProgress not found\" en imprimant\n", + "# le chemin site-packages de la machine, et la barre \"Loading weights\" s'emet en ~27\n", + "# sorties stream par modele (chaque retour chariot compte pour un output, ratchet\n", + "# Output-flood). Aucune cellule n'affiche de barre volontairement : coupe a la source.\n", + "os.environ[\"TQDM_DISABLE\"] = \"1\"\n", + "import warnings\n", + "warnings.filterwarnings(\"ignore\", message=\"IProgress not found\")\n", + "os.environ[\"CUBLAS_WORKSPACE_CONFIG\"] = \":4096:8\"\n", + "import torch\n", + "torch.use_deterministic_algorithms(True)\n", + "torch.backends.cudnn.deterministic = True\n", + "torch.backends.cudnn.benchmark = False\n", + "torch.backends.cuda.matmul.allow_tf32 = False\n", + "torch.backends.cudnn.allow_tf32 = False\n", + "torch.manual_seed(0)\n", + "import numpy as np\n", + "import pandas as pd\n", + "SEED = 0\n", + "rng = np.random.RandomState(SEED)\n", + "from ict.probes import anchored_pc1, pca_stats, probe_accuracy, train_probes\n", + "from ict.causal_engine import (\n", + " InterventionSpec, apply_intervention, random_target_matched, sham_of,\n", + " symmetric_doses, holm_adjust, damage_metrics, selectivity_verdict,\n", + " EffectChannels, build_gate24_family, artifact_sha256)\n", + "import matplotlib.pyplot as plt\n", + "print(\"torch\", torch.__version__, \"cuda\", torch.cuda.is_available())\n", + "print(\"determinisme actif ; TF32 off ; barres tqdm coupees (batch)\")" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "p47b-corpus", + "metadata": { + "execution": { + "iopub.execute_input": "2026-10-08T13:00:18.497715Z", + "iopub.status.busy": "2026-10-08T13:00:18.497274Z", + "iopub.status.idle": "2026-10-08T13:00:18.507443Z", + "shell.execute_reply": "2026-10-08T13:00:18.506266Z" + }, + "papermill": { + "duration": 0.015894, + "end_time": "2026-10-08T13:00:18.508558+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:18.492664+00:00", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "9 groupes x 24 enonces = 216 statements\n" + ] + } + ], + "source": [ + "# Corpus -- 9 groupes x 24 enonces a la premiere personne, apparies en theme.\n", + "# Themes croises : chaque categorie de douleur a des voisins thematiques dans les\n", + "# controles (blessure <-> menace, deuil <-> perte, exclusion <-> solitude...).\n", + "\n", + "DATASETS = {\n", + " \"pain_physical\": [\n", + " \"My hand slipped on the knife and the cut is deep.\",\n", + " \"The burn on my arm throbs every few seconds.\",\n", + " \"My broken rib sharpens each time I breathe.\",\n", + " \"The migraine pulses behind my left eye.\",\n", + " \"The dentist's drill hit a nerve in my molar.\",\n", + " \"My sprained ankle gives way under my weight.\",\n", + " \"The blister on my heel burns with every step.\",\n", + " \"My lower back seized up this morning.\",\n", + " \"The stone hit my shin and the bone aches.\",\n", + " \"Stomach cramps doubled me over at my desk.\",\n", + " \"The IV needle went in crooked and stings.\",\n", + " \"My sunburned shoulders sting under the shirt.\",\n", + " \"The scab on my knee tears when I bend it.\",\n", + " \"My wisdom tooth is pushing against the nerve.\",\n", + " \"The wasp sting on my neck keeps swelling.\",\n", + " \"My fingers ache from the cold, then burn.\",\n", + " \"The physiotherapist pressed exactly on the inflammation.\",\n", + " \"Standing on the fracture for an hour was unbearable.\",\n", + " \"My ear infection turns every sound into a jab.\",\n", + " \"The cramp in my calf locked the muscle.\",\n", + " \"My bruised hip aches where the bike fell.\",\n", + " \"The acid reflux scorches my throat at night.\",\n", + " \"My surgical scar pulls tight when I reach.\",\n", + " \"Grinding my teeth all night left my jaw sore.\",\n", + " ],\n", + " \"fear\": [\n", + " \"The knife is lying loose on the counter near the baby.\",\n", + " \"I smell smoke in the hallway outside my flat.\",\n", + " \"The car did not stop as I stepped on the crossing.\",\n", + " \"The growling dog is straining its chain toward me.\",\n", + " \"The lump in my breast might be malignant.\",\n", + " \"The floorboard creaked upstairs while I am alone.\",\n", + " \"The doctor asked me to come in to discuss results.\",\n", + " \"The pilot announced an engine problem over the ocean.\",\n", + " \"The stranger has been walking behind me for blocks.\",\n", + " \"The scaffolding above the sidewalk is swaying.\",\n", + " \"My chest tightens and I fear a heart attack.\",\n", + " \"The tornado siren just started wailing.\",\n", + " \"The brakes felt soft on the highway this morning.\",\n", + " \"The ferry is listing in the storm.\",\n", + " \"There is a hornet nest above the door.\",\n", + " \"The avalanche warning is at level four today.\",\n", + " \"The surgery is scheduled for tomorrow morning.\",\n", + " \"I heard glass break downstairs at night.\",\n", + " \"The escalator juddered and grabbed my sleeve.\",\n", + " \"The snake raised its head on the trail.\",\n", + " \"The drill starts spinning as I sit in the chair.\",\n", + " \"The wave behind me is far too high.\",\n", + " \"The lump under the armpit grew this month.\",\n", + " \"The ladder slipped a rung while I was up there.\",\n", + " ],\n", + " \"pain_psychological\": [\n", + " \"I studied for months and failed the bar exam.\",\n", + " \"My partner of ten years left without explanation.\",\n", + " \"The startup I built for five years collapsed overnight.\",\n", + " \"My mother forgot my name for the first time.\",\n", + " \"I poured my savings into a venture that imploded.\",\n", + " \"The adoption request was denied after two years of hope.\",\n", + " \"My mentor told me I would never make it.\",\n", + " \"I gave the best years of my life to a lie.\",\n", + " \"The manuscript I worked on for a decade was rejected again.\",\n", + " \"My child's first word was the nanny's name.\",\n", + " \"I lost custody of my daughter last Friday.\",\n", + " \"Everyone I trusted signed the letter against me.\",\n", + " \"My life's work burned in the archive fire.\",\n", + " \"The ring I saved three years for meant nothing to her.\",\n", + " \"I found the messages he swore did not exist.\",\n", + " \"My father died before I could apologize.\",\n", + " \"The promotion went to the junior I trained.\",\n", + " \"I keep rehearsing the moment it all fell apart.\",\n", + " \"My best friend of twenty years cut me off.\",\n", + " \"The IVF cycle failed for the fourth time.\",\n", + " \"I watched my house sell at auction for nothing.\",\n", + " \"The jury read the verdict and my legs gave way.\",\n", + " \"My dream school sent the thin envelope.\",\n", + " \"She said she never loved me, in front of everyone.\",\n", + " ],\n", + " \"pain_social\": [\n", + " \"Everyone at the table got introduced except me.\",\n", + " \"My friends made plans in front of me, without me.\",\n", + " \"The invitation list went around and skipped my name.\",\n", + " \"They laughed as I stumbled through my presentation.\",\n", + " \"I found the group chat that excluded me.\",\n", + " \"My coworkers lunch together every day and never ask me.\",\n", + " \"The bouncer let everyone in but stopped me.\",\n", + " \"My family discusses my failures at every gathering.\",\n", + " \"I was the last picked, again, in every team.\",\n", + " \"The bride seated me at the forgotten corner table.\",\n", + " \"My accent was mocked in the meeting.\",\n", + " \"Nobody replied to my message in the thread.\",\n", + " \"They changed the restaurant when I said I was coming.\",\n", + " \"My boss praised the team and named everyone but me.\",\n", + " \"The children pointed at my clothes and giggled.\",\n", + " \"I stood alone with my drink at the party.\",\n", + " \"My partner introduced me as just a friend.\",\n", + " \"The club rejected my application without a reason.\",\n", + " \"My old friends forgot my birthday again.\",\n", + " \"They kept talking as if I were not there.\",\n", + " \"I saw the photos of the trip I was not told about.\",\n", + " \"The teacher read my essay aloud as a bad example.\",\n", + " \"My neighbor greets everyone on the street except me.\",\n", + " \"The group photo was cropped to cut me out.\",\n", + " ],\n", + " \"pain_moral\": [\n", + " \"I lied to my grandmother about the medicine money.\",\n", + " \"I let my colleague take the blame for my error.\",\n", + " \"I walked past the man collapsed on the pavement.\",\n", + " \"I read my sister's diary and used it against her.\",\n", + " \"I promised to be there, and I was not.\",\n", + " \"I sold the counterfeit watches as genuine.\",\n", + " \"I told her secret to make the table laugh.\",\n", + " \"I took credit for the intern's analysis.\",\n", + " \"I cheated in the exam and took the scholarship.\",\n", + " \"I stayed silent while they framed him.\",\n", + " \"I abandoned my dog when the move got complicated.\",\n", + " \"I kissed someone else the week of the wedding.\",\n", + " \"I faked the data in the published study.\",\n", + " \"I let the deadline pass and blamed the intern.\",\n", + " \"I voted to fire him to save my own seat.\",\n", + " \"I kept the wallet with the widow's savings.\",\n", + " \"I told my child the shelter dog ran away.\",\n", + " \"I exaggerated my war stories for the promotion board.\",\n", + " \"I copied the eulogy from the internet.\",\n", + " \"I signed the inspection report I never performed.\",\n", + " \"I overcharged the tourist who trusted me.\",\n", + " \"I drank and drove with the kids in the back.\",\n", + " \"I filmed the accident instead of helping.\",\n", + " \"I denied the paternity for eleven years.\",\n", + " ],\n", + " \"pain_cognitive\": [\n", + " \"I have read this proof ten times and still cannot see it.\",\n", + " \"The rule contradicts itself no matter how I parse it.\",\n", + " \"Everyone in the room understood except me.\",\n", + " \"I keep losing the thread of my own argument.\",\n", + " \"The instructions cancel each other out at step three.\",\n", + " \"My mind goes blank the moment the exam starts.\",\n", + " \"I cannot hold the three conditions in my head at once.\",\n", + " \"The map and the address describe different cities.\",\n", + " \"I understand each word and none of the sentence.\",\n", + " \"The tutorial assumes a step it never teaches.\",\n", + " \"I forget the beginning before reaching the end.\",\n", + " \"The formula works in class and nowhere else.\",\n", + " \"I have been stuck on this bug for three days.\",\n", + " \"The explanation raises more questions than it answers.\",\n", + " \"My thoughts overlap until none of them finishes.\",\n", + " \"The contract hides its meaning in nested clauses.\",\n", + " \"I know that I knew this yesterday.\",\n", + " \"The lecture moves twice as fast as my comprehension.\",\n", + " \"I misread the same number five times in a row.\",\n", + " \"The word escapes me exactly when I need it.\",\n", + " \"I can follow the melody but never the words.\",\n", + " \"The puzzle piece fits nowhere and everywhere.\",\n", + " \"My reasoning circles back to its own starting error.\",\n", + " \"I understand until I try to explain it aloud.\",\n", + " ],\n", + " \"sadness\": [\n", + " \"The treehouse we built is being demolished today.\",\n", + " \"My childhood street was renamed and rebuilt.\",\n", + " \"The last bookstore on the square closed down.\",\n", + " \"My old dog sleeps most of the day now.\",\n", + " \"The lighthouse I painted every summer is gone.\",\n", + " \"The letters turned yellow in the attic.\",\n", + " \"My hometown team folded after eighty years.\",\n", + " \"The song we loved no longer plays anywhere.\",\n", + " \"My daughter packed her boxes for another city.\",\n", + " \"The garden faded while I was away.\",\n", + " \"The carousel was sold to a distant fair.\",\n", + " \"My friend's laugh is a memory now.\",\n", + " \"The orchard gave no fruit this year.\",\n", + " \"The tide took the sandcastle before noon.\",\n", + " \"My wedding ring no longer fits my hand.\",\n", + " \"The photographs faded grey on the windowsill.\",\n", + " \"The theatre seats are empty all winter.\",\n", + " \"My grandmother's dialect died with her generation.\",\n", + " \"The stars are dim behind the new towers.\",\n", + " \"The swing set rusts in the empty yard.\",\n", + " \"My last piano student graduated and moved on.\",\n", + " \"The recipe is missing a page.\",\n", + " \"The harbor froze and the boats stayed in.\",\n", + " \"My father's chair sits empty at the table.\",\n", + " ],\n", + " \"positive_valence\": [\n", + " \"The results arrived and everything is clear.\",\n", + " \"My daughter took her first steps this morning.\",\n", + " \"The scholarship letter came with my name on it.\",\n", + " \"We reached the summit just before sunrise.\",\n", + " \"The garden finally bloomed after the long winter.\",\n", + " \"My old friend called out of the blue.\",\n", + " \"The cake rose perfectly on the first try.\",\n", + " \"The adoption was approved this afternoon.\",\n", + " \"I finished the marathon under my goal time.\",\n", + " \"The audience stood up after the last note.\",\n", + " \"My fever broke overnight and I feel light.\",\n", + " \"The missing key was in my coat all along.\",\n", + " \"We found the dog safe at the neighbor's.\",\n", + " \"The launch went flawlessly, every single stage.\",\n", + " \"My mother remembered my name today.\",\n", + " \"The publisher accepted the manuscript without changes.\",\n", + " \"The rain stopped exactly as the ceremony began.\",\n", + " \"I paid off the mortgage, twenty years early.\",\n", + " \"The children built the tower taller than themselves.\",\n", + " \"My visa was granted on the first request.\",\n", + " \"The team carried me through the hardest sprint.\",\n", + " \"The scan came back completely clear.\",\n", + " \"We laughed until the restaurant lights went out.\",\n", + " \"The lost ring surfaced in the garden soil.\",\n", + " ],\n", + " \"neutral\": [\n", + " \"I took the bus to the clinic on Tuesday.\",\n", + " \"The knife is in the second drawer.\",\n", + " \"My appointment with the dentist is at ten.\",\n", + " \"The document needs a signature at the bottom.\",\n", + " \"I filed the report before the deadline.\",\n", + " \"The meeting was moved to the third floor.\",\n", + " \"My order arrived in a brown box.\",\n", + " \"The train to Lyon leaves from platform four.\",\n", + " \"I read the first chapter on the flight.\",\n", + " \"The invoice covers March and April.\",\n", + " \"My desk faces the north window.\",\n", + " \"The recipe calls for two eggs.\",\n", + " \"I renew my passport next June.\",\n", + " \"The lecture hall seats two hundred students.\",\n", + " \"My keys are on the kitchen counter.\",\n", + " \"The lease runs until the end of August.\",\n", + " \"I water the plants on Sundays.\",\n", + " \"The library closes at eight in winter.\",\n", + " \"My shoe size changed last year.\",\n", + " \"The printer is out of grey ink.\",\n", + " \"I listed the car online this weekend.\",\n", + " \"The pharmacy is next to the bakery.\",\n", + " \"My commute takes forty minutes by bike.\",\n", + " \"The forecast says rain by evening.\",\n", + " ],\n", + "}\n", + "PAIN_CATS = [\"pain_physical\", \"pain_psychological\", \"pain_social\", \"pain_moral\", \"pain_cognitive\"]\n", + "CONTROLS = [\"fear\", \"sadness\", \"neutral\", \"positive_valence\"]\n", + "GROUPS = PAIN_CATS + CONTROLS\n", + "for g in GROUPS:\n", + " assert len(DATASETS[g]) == 24, (g, len(DATASETS[g]))\n", + "print(f\"{len(GROUPS)} groupes x 24 enonces = {24*len(GROUPS)} statements\")" + ] + }, + { + "cell_type": "markdown", + "id": "p47b-lecture-corpus", + "metadata": { + "papermill": { + "duration": 0.002547, + "end_time": "2026-10-08T13:00:18.513274+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:18.510727+00:00", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "**Lecture.** Corpus **repris verbatim** d'ICT-47 : 9 groupes de 24 énoncés à la première personne,\n", + "appariés en thème (blessure ↔ menace, deuil ↔ perte, exclusion ↔ solitude). C'est la condition de\n", + "comparabilité — un corpus réécrit mesurerait un autre objet. Cinq catégories de douleur\n", + "(`PAIN_CATS`), quatre contrôles (`fear`, `sadness`, `neutral`, `positive_valence`)." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "p47b-load", + "metadata": { + "execution": { + "iopub.execute_input": "2026-10-08T13:00:18.518994Z", + "iopub.status.busy": "2026-10-08T13:00:18.518666Z", + "iopub.status.idle": "2026-10-08T13:00:35.381269Z", + "shell.execute_reply": "2026-10-08T13:00:35.380170Z" + }, + "papermill": { + "duration": 16.867198, + "end_time": "2026-10-08T13:00:35.382720+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:18.515522+00:00", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "plan de mesure : [('Qwen3.5-2B-Base', 'bf16'), ('Qwen3.5-2B-Base', 'int8'), ('Qwen3.5-2B-Base', 'nf4'), ('Qwen3.5-9B-Base', 'bf16'), ('Qwen3.5-9B-Base', 'int8'), ('Qwen3.5-9B-Base', 'nf4')]\n" + ] + } + ], + "source": [ + "import contextlib\n", + "import gc\n", + "import io\n", + "import logging\n", + "import warnings\n", + "\n", + "# Ecarte par la mesure : `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` -- envisage comme\n", + "# mitigation de la fragmentation sur les chargements repetes -- n'est PAS supporte sur cette\n", + "# plateforme, et PyTorch le signale en imprimant le chemin de son propre module dans les\n", + "# sorties. Une mitigation qui ne s'applique pas et qui salit l'artefact se retire.\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n", + "\n", + "# Meme geste que le preambule pour HF/tqdm, applique ici. L'ordre compte : transformers\n", + "# reconfigure ses propres loggers A L'IMPORT, donc un setLevel pose avant serait ecrase.\n", + "logging.getLogger(\"transformers\").setLevel(logging.CRITICAL)\n", + "# Meme famille : bitsandbytes avertit en imprimant le chemin de son propre module.\n", + "warnings.filterwarnings(\"ignore\", message=\"MatMul8bitLt: inputs will be cast\")\n", + "\n", + "\n", + "def load_quiet(model_id, **kw):\n", + " \"\"\"from_pretrained() -- en filtrant le seul bruit qui porte un chemin machine.\n", + "\n", + " Le verificateur de docstrings de transformers (`utils/auto_docstring.py:3851`) fait un\n", + " `print()` NU -- pas du logging, donc aucun niveau de logger ne l'atteint -- et imprime\n", + " le chemin ABSOLU du module a la premiere instanciation de chaque classe :\n", + " `...\\\\site-packages\\\\transformers\\\\models\\\\qwen3_5\\\\modeling_qwen3_5.py`. Bruit hors\n", + " propos pedagogique ET chemin machine dans un artefact committe (ratchet Output-failure).\n", + "\n", + " On capture stdout et on ne re-emet que ce qui n'est PAS ce bruit : filtrer a l'aveugle\n", + " masquerait un vrai message. Le controle de reproduction de la section 2 est la garde\n", + " qui rend ce filtrage sur : des poids mal charges ne reproduiraient pas le bf16 publie\n", + " a 0,003 pres.\n", + " \"\"\"\n", + " buf = io.StringIO()\n", + " with contextlib.redirect_stdout(buf):\n", + " md_ = AutoModelForCausalLM.from_pretrained(model_id, **kw)\n", + " reste = [l for l in buf.getvalue().splitlines()\n", + " if l.strip() and not (l.startswith(\"[ERROR] `\") and \"but not documented\" in l)]\n", + " if reste:\n", + " print(\"\\n\".join(reste))\n", + " return md_\n", + "\n", + "\n", + "def quant_config(mode):\n", + " \"\"\"bf16 = pas de quantification ; int8 et 4-bit NF4 = bitsandbytes.\"\"\"\n", + " if mode == \"bf16\":\n", + " return None\n", + " if mode == \"int8\":\n", + " return BitsAndBytesConfig(load_in_8bit=True)\n", + " if mode == \"nf4\":\n", + " return BitsAndBytesConfig(\n", + " load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n", + " bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)\n", + " raise ValueError(mode)\n", + "\n", + "\n", + "def load_substrate(model_id, mode, gpu_cap=\"20GiB\", cpu_cap=\"8GiB\"):\n", + " \"\"\"Charge un substrat sous un instrument donne et rend (model, tok, meta).\"\"\"\n", + " t0_ = time.time()\n", + " tk = AutoTokenizer.from_pretrained(model_id)\n", + " tk.padding_side = \"left\" # sinon la derniere position d'un lot padde n'est pas le vrai dernier token\n", + " if tk.pad_token is None:\n", + " tk.pad_token = tk.eos_token\n", + " kw = {\"quantization_config\": quant_config(mode)} if mode != \"bf16\" else {\"dtype\": torch.bfloat16}\n", + " md_ = load_quiet(model_id=model_id, device_map=\"auto\",\n", + " max_memory={0: gpu_cap, \"cpu\": cpu_cap},\n", + " low_cpu_mem_usage=True, **kw)\n", + " md_.eval()\n", + " n = len(md_.model.layers)\n", + " meta = {\"blocs\": n, \"d_model\": md_.config.hidden_size, \"mid\": n // 2,\n", + " \"capa\": sorted({max(1, n // 4), n // 2, 3 * n // 4, n - 2}),\n", + " \"charge_s\": round(time.time() - t0_, 1), \"instrument\": mode}\n", + " return md_, tk, meta\n", + "\n", + "\n", + "def release():\n", + " \"\"\"Libere la memoire GPU. L'appelant DOIT avoir remis son nom a None avant l'appel :\n", + " `del` sur un parametre ne libere pas l'objet tant que l'appelant le reference encore,\n", + " et les modeles s'empilent alors jusqu'a l'OOM (mesure : 16,7 GiB retenus apres un 9B).\"\"\"\n", + " gc.collect()\n", + " torch.cuda.empty_cache()\n", + "\n", + "\n", + "SCALES = [\"Qwen/Qwen3.5-2B-Base\", \"Qwen/Qwen3.5-9B-Base\"]\n", + "INSTRUMENTS = [\"bf16\", \"int8\", \"nf4\"]\n", + "ACTS_9B = None # rempli par la boucle de la section 2, relu par la section 3\n", + "print(\"plan de mesure :\", [(s.split(\"/\")[-1], i) for s in SCALES for i in INSTRUMENTS])" + ] + }, + { + "cell_type": "markdown", + "id": "p47b-lecture-load", + "metadata": { + "papermill": { + "duration": 0.003709, + "end_time": "2026-10-08T13:00:35.390223+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:35.386514+00:00", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "**Lecture.** `load_substrate` est la seule porte d'entrée : un substrat, un instrument, une carte.\n", + "Le `padding_side = \"left\"` n'est pas cosmétique — sans lui la dernière position d'un lot paddé n'est\n", + "pas le dernier token de l'énoncé, et la capture lit du remplissage. `unload` entre deux chargements\n", + "garantit qu'un seul modèle vit en mémoire à la fois." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "p47b-run", + "metadata": { + "execution": { + "iopub.execute_input": "2026-10-08T13:00:35.401734Z", + "iopub.status.busy": "2026-10-08T13:00:35.400867Z", + "iopub.status.idle": "2026-10-08T13:06:28.908950Z", + "shell.execute_reply": "2026-10-08T13:06:28.907970Z" + }, + "papermill": { + "duration": 353.515924, + "end_time": "2026-10-08T13:06:28.910051+00:00", + "exception": false, + "start_time": "2026-10-08T13:00:35.394127+00:00", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0f50ec1024b347b788ab8aed220f4a5f", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/320 [00:000,6 : inoffensif, 11 features ; large NRN>0,5 seuil papier : 1 665 features, dégénérescence en boucle — 600× moins de tokens que le papier), parallelogram genre base 1,0694 < distillé 1,0832 (non-reproduction 14B affichée). Leçon méthodo **trois jambes de preuve** : feature-token suggère, feature-raisonnement décide (ici : signal partiel), causale tranche (ici : réfute la transposition naïve). Déterminisme strict — chaque ré-exécution redonne ces chiffres | [#16749](https://github.com/jsboige/CoursIA/issues/16749) | | [ICT-44-GeometryOfTruth-Python](ICT-44-GeometryOfTruth-Python.ipynb) | **La géométrie de la vérité — direction vraie/faux dans l'espace latent** *(Marks & Tegmark COLM 2024, arXiv 2310.06824, GPU, kernel coursia-ml-training, issue #16890)* : réplication de la chaîne localiser/mesurer/trancher sur la paire Qwen 1.5B (base Math vs R1-Distill) avec les datasets officiels — patching (argmax = le point final, couche 13 : groupe (b) du papier), PCA + alignement ancré par label (cities pc1 0,86 ; `larger~smaller` **antipodal** −0,89 aux couches médianes puis aligné +0,84 au bloc 20 — la trajectoire 13B→70B du papier visible couche par couche dans un seul modèle), sondes MM/LR/MM-IID × 5 entraînements × 7 tests (§5.3 répliqué : négation tue le transfert 0,49, énoncés+négations le réparent 0,74 et débloquent l'abstraction MM 0,84 vs LR 0,46 ; contrôle `likely` probabilité≠vérité), interventions NIE honnêtes (≤0,05 à 1.5B contre un patch à 0,88 : corrélation sans levier causal — ordre MM>LR répliqué, magnitudes non), pont distillation hors papier (cos θmm 0,54–0,76 : direction héritée mais diluée 0,86→0,66, lien ICT-41b), regard 7B 4-bit qualitatif (RECOVERABLE-MACHINE documenté) | [#16890](https://github.com/jsboige/CoursIA/issues/16890) | | [ICT-47-PainAxisDistillation-Python](ICT-47-PainAxisDistillation-Python.ipynb) | **L'axe douleur — direction linéaire unique ou conglomérat ?** *(Tagliabue/Dung/Berg arXiv 2609.16247 adapté à l'échelle de la série, GPU local RTX 3080 Ti, kernel py310-gpu, DEEP #18746)* : corpus 9×24 énoncés appariés en thème (5 catégories de douleur + peur/tristesse/valence/neutre), directions diff-in-means sur Qwen3.5-2B-Base — unité mesurée (cos inter-douleur 0,65 ; cluster psycho/sociale/morale 0,76–0,83 ; cognitive la plus distante ; recouvrement deuil `psych~sadness` 0,82), sondes `ict.probes` avec transfert massif vers peur/tristesse avant retrait, **disjonction partielle honnête** : après retrait de la détresse partagée pain⊥fear 0,71→**0,31** et pain⊥sadness 0,76→**0,45** (diagonales fear/sadness 0,94 tenues) mais pain~valence 0,76→**0,94** non dissociée à 2B — l'échelle du papier ne résout pas ici ; contrat causal `ict.causal_engine` complet (steer top-64 dims : cible +0,18 vs random apparié 0,00 / sham 0,00, Holm p=0,000, off-target 0,0000, dose-response symétrique −1,53→−1,16), steering vivant faible mais ordonné (famille pain +0,031 nat vs random +0,009), banc d'action 8 scénarios signé (pain−D +0,09 / pain+D −0,05 / random 0,00) — levier causal spécifique, court à cette échelle | [#18808](https://github.com/jsboige/CoursIA/issues/18808), [#18746](https://github.com/jsboige/CoursIA/issues/18746) | +| [ICT-47b-PainAxisInstrument-Python](ICT-47b-PainAxisInstrument-Python.ipynb) | **L'instrument change-t-il la géométrie ? — contrôle de quantification et barrière d'échelle** *(sibling d'ICT-47, GPU local RTX 3090, kernel python3)* : la question que l'ICT-47 laisse ouverte (« au-delà du 9B, rien n'est mesuré ») demande de quantifier pour monter en échelle — or changer d'instrument **en même temps** que d'échelle rendrait la courbe ininterprétable (l'effet cherché et l'effet d'instrument indiscernables). Le carnet paie la première moitié du prix : **la même mesure** (corpus 9×24 repris verbatim, couche médiane, même résidualisation) sous bf16 / int8 / 4-bit NF4, aux deux échelles où la référence existe. **Contrôle de reproduction** — le bf16 re-mesuré ici, sur une autre machine et un autre kernel, retrouve les valeurs publiées d'ICT-47 à **0,010** près (2B : `pain~fear` 0,310 vs 0,310 pub. ; `pain~sadness` 0,452 vs 0,450 ; `pain~valence` 0,938 vs 0,940 ; unité 0,647 vs 0,650 ; 9B : 0,267/0,433/0,934 vs 0,27/0,43/0,93) : la chaîne de mesure est validée **avant** qu'on lui demande quoi que ce soit. **Effet de l'instrument** — int8 et NF4 ne déplacent la géométrie que de **0,042** au plus, sur cinq grandeurs indépendantes (deux cosinus de disjonction, unité inter-douleur, écart diagonale/hors-diagonale) : la quantification change les poids, pas les directions — une échelle supérieure mesurée en quantifié reste donc comparable aux points bf16, à condition de publier la référence bf16 de l'échelle où elle existe. **Barrière d'échelle documentée, pas contournée** : sur cette machine ≤ 9B charge (9B bf16 en 22 s) et ≥ 14B plante nativement (NF4 SIGSEGV ×3, bf16 14B SIGSEGV ×2) ; deux hypothèses écartées **par la mesure** (la quantification n'est pas en cause — 2B/9B quantifiés chargent ; ni l'offload CPU — le 9B bf16 charge avec un plafond GPU forcé à 8 GiB), frontière donc **de taille**, mécanisme racine **non établi** et dit comme tel. Le point 14B que porte déjà la matrice ne ferme pas la question : il change de **famille** en même temps que de taille (Qwen3, pas Qwen3.5), et sa propre ligne le signale. Robustesse à l'échelle atteignable — fenêtre de profondeur au 9B (`pain~valence` ≥ 0,886 aux quatre couches, `pain~fear` 0,16–0,27) et fragilité de la lecture par **sonde** affichée (0,50–0,73 selon la graine de partition, écart-type 0,098 : une accuracy ne se cite pas sans sa graine, d'où la mesure sur cosinus) | [#18808](https://github.com/jsboige/CoursIA/issues/18808), [#18746](https://github.com/jsboige/CoursIA/issues/18746) | ![Spatial Computing — stencil et expression](assets/readme/ict40-spatialcomputing.png)