update titles

Author: AlexandreAndr

docs/curriculum_roadmap.md

@@ -0,0 +1,187 @@
+# ENM 3800 "Learning from Data" — Curriculum Roadmap
+
+Living design doc for course structure and planned changes. Committed so it
+travels with the repo rather than living in chat history. Edit as decisions land.
+
+**Status legend**
+
+- ✅ **Decided** — agreed; safe to implement.
+- 🔧 **Proposed** — drafted, not yet agreed or implemented.
+- ❓ **Open** — needs a decision before work starts.
+
+> **Note:** The handoff references `docs/ENM3800_Revised_Schedule.md` (fixed,
+> date-bound slots; rule for changes = reallocate/re-theme, don't add). That file
+> is **not yet in the repo.** Lecture slots below are referenced by number (L2–L15).
+> Add the schedule doc so date-bound slots have a source of truth.
+
+---
+
+## Current module structure
+
+| Module | Title | Notebooks | Lectures |
+|--------|-------|-----------|----------|
+| M1 | Learning from Data | 1a, 1b | L2–3 |
+| M2 | Building & Selecting Models | 2a, 2b | L5–6 |
+| M3 | Probability & Inference | 3a, 3b, 3c | L8–10 |
+| M4 | Supervised | 4a, 4b, 4c | L7, 11–13 |
+| M5 | Unsupervised | 5a, 5b | L14–15 |
+
+---
+
+## M1 split — Learning from Data  ✅ IMPLEMENTED
+
+The Module 1 split was already in the repo when audited:
+
+| File | Title | Lecture |
+|---|---|---|
+| `Notebook_1a_Overview_and_Data_Representation` | Overview & Data Representation | L2 |
+| `Notebook_1b_Noise_and_Modeling_Data` | Noise, Variability & Modeling Data | L3 |
+
+Both are registered as separate M1 chapters in `_quarto-book.yml`. The split is
+clean by theme — **1a = representation** (Python, linear algebra, matrices as
+transformations, images, tables); **1b = variability** (noise, covariance,
+distributions) — and 1a hands off explicitly ("covariance… comes in the next
+notebook").
+
+**Self-containment audit** surfaced two issues, both since fixed:
+
+1. **Dangling prerequisite.** 1b claimed its covariance work built on "eigenvector
+   ideas from the previous notebook," but 1a never introduces eigenvectors. Fixed by
+   removing the claim.
+2. **Cross-module duplication with the M3/M5 rework:**
+   - *Covariance eigenvectors* — 1b (L3) duplicated the geometric-covariance content
+     staged for M5 PCA. **Decision: cut** 1b's "Eigenvectors of the Covariance
+     Matrix" section (too heavy for L3); keep a light covariance-as-shape teaser with
+     a forward pointer to Module 5. `_staging_covariance_geometry.py` is now the
+     single source of the eigenvector treatment.
+   - *Probability distributions* — 1b (L3, intuitive) overlaps 3a's zoo (L8,
+     rigorous). **Decision: keep as a deliberate preview** — 1b now points forward to
+     Module 3, and 3a's zoo points back to 1b. Coordinated spiral, no cuts.
+
+---
+
+## M3 rework — Probability & Inference  ✅ IMPLEMENTED
+
+**Goal:** re-theme the three M3 notebooks along a cleaner narrative spine —
+**L8 Probability → L9 Statistics 1 → L10 Statistics 2** — culminating L8 in the CLT
+as the bridge into inference. Fits existing slots (no additions).
+
+> **Implemented** (this pass). Titles, content, **and filenames** now match the new
+> themes. Files were renamed (`git mv` on both the `.py` and `.ipynb` of each pair):
+>
+> | New file | Was |
+> |---|---|
+> | `Notebook_3a_Probability` | `Notebook_3a_Probability_and_Covariance` |
+> | `Notebook_3b_Statistics_1` | `Notebook_3b_Hypothesis_Testing` |
+> | `Notebook_3c_Statistics_2` | `Notebook_3c_Bootstrapping_and_Permutation_Tests` |
+>
+> `_quarto-book.yml` chapter paths and every Colab badge URL (title + `#scrollTo=`
+> exercise deep-links) were updated to match; exercise badges are regenerated by
+> `scripts/add_exercise_badges.py`, which derives the URL from the file path.
+>
+> A new generating-process exercise (`id="ex-generating-processes"`, "Name That
+> Process") was added to 3a's zoo.
+>
+> Geometric-covariance cells were extracted to
+> `notebooks/Notebook_5/_staging_covariance_geometry.py` (leading underscore → not
+> rendered, not in the book config). Fold them into **5a** when PCA is worked, and
+> update that file's badge URLs to 5a's path.
+
+### Key findings from the current notebooks
+
+Two facts reshaped this from "write a lot of new content" into "mostly migration":
+
+1. **No Bayes content exists** in 3a or 3b. "Bayes deprioritized" is a no-op —
+   nothing to remove.
+2. **The CLT/LLN capstone already exists — in 3b.** The "Sampling Distributions:
+   Where t-tests Come From" section (`Notebook_3b:467`) *is* a CLT/LLN demo:
+   repeated samples → distribution of the mean → it narrows → `SE = σ/√n`.
+   Already written, tested, with embedded outputs. Migrate it into 3a rather than
+   author from scratch.
+
+### Target: 3a → **Probability** (L8) ✅
+
+Narrative spine: build distributions → expectation/variance → LLN (averages
+converge) → CLT (averages go Gaussian, `SE = σ/√n`).
+
+- **Keep:** What is probability; Random variables; Expectation & variance;
+  Coin-flip simulation (already Bernoulli/Binomial *by generating process* — the seed).
+- **New (the only net-new writing):** the **distributions zoo** — Uniform,
+  Poisson, Exponential, Gaussian, each taught *by its generating process*, with
+  E/Var formalized per distribution.
+- **Migrate in from 3b:** "Sampling Distributions" + "SE shrinks with √n" →
+  reframed as the **LLN → CLT capstone**.
+- **Move out:** all covariance/correlation (see split below).
+
+### Target: 3b → **Statistics 1** (L9) ✅
+
+- **Loses** the sampling-distribution material to 3a (~95 lines freed).
+- **Gains** the *statistical* covariance/correlation from 3a.
+- **Keeps:** framing a question; test statistics & p-values; confidence intervals;
+  t-test deep dive; two-sample t-test types; ANOVA; paired t-test.
+- **Opens** by cashing in 3a's CLT: "because the sample mean is ~Gaussian, the
+  t-statistic has a known distribution" — clean handoff.
+- **Type I/II introduced here** (α = Type I error; power = 1−β). The existing α
+  and "fail-to-reject ≠ accept" callouts already imply it — this is a labeling
+  pass, not new content.
+
+### Target: 3c → **Statistics 2** (L10) ✅
+
+- **Keeps:** bootstrap; permutation tests.
+- **Type I/II quantified** — the "Noise, Sample Size, and Power" demo needed for
+  this already exists at `Notebook_3c:347`. Reframing, not writing.
+
+### Covariance split ✅ (resolves Open Decision: "covariance's home")
+
+The 3a covariance content is mostly *geometric* (shape-of-a-data-cloud, tilted
+clouds, interactive shape explorer) — that's PCA prep, not t-test prep. Split it:
+
+- **Statistical** part (correlation coefficient, correlation matrix, correlated-
+  variables sim, correlation in a real dataset) → **L9 Statistics (3b).**
+- **Geometric** part (covariance as shape of a data cloud, tilted cloud, three-
+  shapes comparison, interactive shape explorer) → **earmark for M5 PCA prep.**
+  Re-invoke the covariance matrix there: "its eigenvectors are the principal
+  components." Same spiral pattern as Type I/II.
+
+### Work summary
+
+1. **Write** the distributions zoo (3a) — the only net-new authoring.
+2. **Migrate** sampling-distribution/CLT section 3b → 3a; statistical covariance 3a → 3b.
+3. **Relabel** for the Type I/II spiral (3b introduce, 3c quantify).
+4. **Earmark** geometric covariance for M5 (leave a pointer; move when PCA is touched).
+
+Per repo convention: edit the `.py`, then `jupytext --sync`; re-run
+`scripts/execute_notebooks.sh` for fresh embedded outputs.
+
+---
+
+## Other proposals (from handoff — not yet worked)
+
+*(M1 split — now ✅ implemented; see the M1 section below.)*
+
+🔧 **M4 deep learning** — add backprop/autograd intuition + transfer learning.
+Recommendation from discussion: **deepen 4c in place (L13)** rather than spin up a
+2-notebook DL module, which would cost an M5 slot and force autoencoders/self-sup
+out of 5a (weakening M5's unsupervised identity).
+
+🔧 **L7 Regularization → "Bias–Variance & Regularization"** — bias–variance is
+missing and lecture-worthy; reframing also fixes timing so L11 regression can apply
+ridge/lasso.
+
+🔧 **L4 "What is ML" → "the map of ML"** — sharpen into taxonomy / workflow /
+generalization. Recommendation: **keep as a structural keystone**, don't cut to a
+buffer or to fund DL (wrong time of year).
+
+🔧 **Type I/II spiral** (cross-module) — introduce in 3b (α, power = 1−β), quantify
+in 3c (power vs sample size/noise), reconnect in **4b** (confusion matrix FP/FN =
+Type I/II; "Error Costs" exercise = cost asymmetry).
+
+---
+
+## Open decisions
+
+- ✅ ~~Covariance's home~~ → **split** (statistical → L9; geometric → M5 PCA prep).
+- ❓ **DL:** deepen 4c in place vs. 2-lecture module (costs an M5 slot).
+  *Leaning deepen-in-place.*
+- ❓ **Timing of the M1 split.** *Leaning: do first — lowest risk.*
+- ❓ **L4:** stays "map of ML" vs. becomes a buffer. *Leaning: keep as keystone.*

notebooks/Notebook_1/Notebook_1a_Overview_and_Data_Representation.ipynb

@@ -7,7 +7,7 @@
     "id": "nb1a-title"
    },
    "source": [
-    "# Learning From Data: Overview & Data Representation\n",
+    "# Overview & Data Representation\n",
     "\n",
     "**Course Title:** ENM 3800: Learning from Data\n",
     "\n",

notebooks/Notebook_1/Notebook_1a_Overview_and_Data_Representation.py

@@ -13,7 +13,7 @@
 # ---
 
 # %% [markdown] id="nb1a-title"
-# # Learning From Data: Overview & Data Representation
+# # Overview & Data Representation
 #
 # **Course Title:** ENM 3800: Learning from Data
 #