Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,100 changes: 2,100 additions & 0 deletions MyIA.AI.Notebooks/IIT/ICT-Series/ICT-45-InoculationBifurcation-9B.ipynb

Large diffs are not rendered by default.

12 changes: 10 additions & 2 deletions MyIA.AI.Notebooks/IIT/ICT-Series/scripts/fit_jlens_local.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
"""Fit local J-lens par taille (sous-grain (c) #8236).

Reproduit la construction du lens publie 9B (neuronpedia/jacobian-lens,
Salesforce-wikitext n=458, min_chars 600) sur Qwen3-1.7B-Base et
Qwen3.5-2B-Base, aux profondeurs de calibration SAE (frac 0.25/0.5) :
Salesforce-wikitext n=458, min_chars 600) sur Qwen3-1.7B-Base,
Qwen3.5-2B-Base et Qwen3.5-9B-Base, aux profondeurs de calibration SAE
(frac 0.25/0.5) :

1.7B (28 couches) : source_layers [7, 14]
2B (24 couches) : source_layers [6, 12]
9B (32 couches) : source_layers [8, 16]

target = logits finaux (defaut). Le checkpoint de fit est resumable
(checkpoint_path + resume=True) ; le lens final est sauvegarde via
Expand All @@ -20,6 +22,7 @@
import argparse
import hashlib
import json
import logging
import time
from pathlib import Path

Expand All @@ -28,6 +31,7 @@
MODELS = {
"Qwen/Qwen3-1.7B-Base": [7, 14], # 28 couches : frac 0.25 / 0.5
"Qwen/Qwen3.5-2B-Base": [6, 12], # 24 couches : frac 0.25 / 0.5
"Qwen/Qwen3.5-9B-Base": [8, 16], # 32 couches : frac 0.25 / 0.5
}
FITS_DIR = Path("C:/dev/jlens_fits")

Expand All @@ -45,6 +49,10 @@ def main() -> None:
)
args = ap.parse_args()

# Progression per-prompt du fitter jlens (sinon WARNING seul = silence
# pendant des heures sur un fit 9B ~15 h).
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")

import jlens
from jlens.examples import load_wikitext_prompts

Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading