Milestone 6: private Academy media storage

README.md

@@ -15,7 +15,7 @@ Transfemme voice-training web app.
 - Frontend: Vite/React static build in `dist/`
 - Backend: Flask/Gunicorn WSGI app in `server/`
 - Auth: first-party FemmeVoice username and passphrase accounts with server-side sessions
-- Database: MongoDB progress storage
+- Database: MongoDB for accounts, progress, governed Academy content, and separate private Academy media storage
 - Service: Gunicorn/Flask backend
 - Admin feedback inbox: server-side administrator allowlist via `FEMMEVOICE_ADMIN_USERNAMES` (comma-separated lowercase usernames); never grant admin access from the browser
 

docs/media-pipeline.md

@@ -35,7 +35,21 @@ The Admin Media Library provides structured fields for identity, source, checksu
 
 The HTML application shell is served with `no-cache` so a deployment immediately discovers the new hashed frontend bundle. This does not disable public API caching: content and media manifests retain their independent cache and stale-while-revalidate policies.
 
-Binary storage remains separate from metadata governance. Teaching files and caption tracks currently require same-origin paths, matching the app's restrictive content-security policy and avoiding third-party learner requests. External HTTPS URLs are accepted only as rights/source attribution. Educational media does not use the encrypted learner-recording vault. A later Milestone 6 slice must choose an upload/CDN strategy and an explicit origin policy before large production audio or video is accepted.
+Binary storage remains separate from metadata governance. Teaching files and caption tracks require same-origin paths, matching the app's restrictive content-security policy and avoiding third-party learner requests. External HTTPS URLs are accepted only as rights/source attribution. Educational media does not use the encrypted learner-recording vault.
+
+## File Storage And Delivery
+
+Academy authors can upload supported raster images, audio, video, WebVTT captions, and PDF resources through the Media Library. The backend streams each upload through a temporary spool, enforces a 100 MB limit, verifies a conservative MIME signature, computes SHA-256, and deduplicates matching content in a separate `academy_media_files` GridFS bucket.
+
+An uploaded file URL returns 404 until a published media revision references it as the teaching source or caption track. Public delivery then provides:
+
+- the reviewed MIME type and a safe filename;
+- a checksum ETag and conditional `304` responses;
+- one-year immutable caching because the file id never changes;
+- byte-range requests for audio/video seeking;
+- attachment delivery for PDFs and inline delivery for lesson media.
+
+The encrypted private-recording bucket is never reused. Abandoned draft uploads require a future cleanup job. GridFS is the current storage adapter, not a permanent mandate: an object-storage/CDN adapter should be reviewed before high-volume or larger video libraries are introduced.
 
 ## Lesson References
 

docs/roadmap.md

@@ -2,19 +2,19 @@
 
 **Status:** living implementation guide  
 **Last updated:** 17 July 2026  
-**Current development phase:** Milestone 6 educational media governance and delivery
+**Current development phase:** Milestone 6 complete; ready for the remaining Foundations curriculum
 
 ## Active Development
 
 | Item | Current state |
 | --- | --- |
-| Current milestone | Milestone 6 - Educational Media Pipeline |
-| Current version | `v0.3.33` |
-| Current working branch | `feat/academy-media-resolution` |
-| Active pull request(s) | PR #21 - Deterministic governed lesson-media resolution; PR #20 is integrated into `main`. |
+| Current milestone | Milestone 6 - Educational Media Pipeline completed |
+| Current version | `v0.3.34` |
+| Current working branch | `main` after the Milestone 6 completion integration |
+| Active pull request(s) | None after PR #22 - Private Academy media storage and immutable delivery - integrates into `main`. |
 | Base branch | `main` is canonical. |
-| Next planned milestone | Continue Milestone 6 with deterministic lesson asset references and public resolution. |
-| Overall completion estimate | About 40% of the long-term Academy vision; engine, history, governance, authoring, and delivery are complete, while media, remaining curriculum, and coaching remain substantial work. |
+| Next planned milestone | Milestone 7 - Remaining Foundations Lessons |
+| Overall completion estimate | About 45% of the long-term Academy vision; engine, history, governance, authoring, and media delivery are complete, while the full curriculum, interactive practice, and coaching remain substantial work. |
 
 ## Vision
 
@@ -57,6 +57,7 @@ The definitive product direction is the [Product Vision](product-vision.md). Voi
 - `v0.3.31` - Governed Academy media library and publication workflow
 - `v0.3.32` - Governed media-library pull-request readiness
 - `v0.3.33` - Deterministic governed lesson-media resolution
+- `v0.3.34` - Private Academy media storage and immutable delivery
 
 Update this list whenever a versioned change is pushed so milestones, pull requests, and releases remain easy to correlate.
 
@@ -417,18 +418,18 @@ Milestone 5 ends when a contributor can comfortably maintain the complete Founda
 
 ### Milestone 6 - Educational Media Pipeline
 
-**Status:** 🚧 In Progress
+**Status:** ✅ Completed
 **Goal:** Produce and maintain reviewed, accessible educational assets without treating media as an afterthought.
 **Complexity:** High
 **Dependencies:** Milestone 5
 
 **Why here:** The engine and authoring workflow must be stable before investing in a larger asset library. Content operations gives every asset a durable review, revision, and publication path.
 
 **Acceptance criteria**
-- [ ] Audio, video, illustration, caption, transcript, and localization assets have clear ownership and version linkage.
-- [ ] Research, content, and accessibility review are recorded before publication.
-- [ ] Assets have replacement, correction, and localization workflows rather than being embedded as one-off files.
-- [ ] No production teaching media is published without its required accessible alternatives.
+- [x] Audio, video, illustration, caption, transcript, and localization assets have clear ownership and version linkage.
+- [x] Research, content, and accessibility review are recorded before publication.
+- [x] Assets have replacement, correction, and localization workflows rather than being embedded as one-off files.
+- [x] No production teaching media is published without its required accessible alternatives.
 
 **In progress**
 - [x] Versioned media contract covers locale, source, rights, accessibility metadata, and three-part review.
@@ -439,7 +440,17 @@ Milestone 5 ends when a contributor can comfortably maintain the complete Founda
 - [x] Lesson media blocks reference deterministic governed asset revisions and resolve them through the public manifest.
 - [x] Public delivery retains every immutable published revision so older lesson pins remain reproducible after replacement.
 - [x] Validate exact-revision resolution and bundled fallback through the staging learner experience.
-- [ ] Validate localization resolution with real Foundations lesson content.
+- [x] Validate replacement and localization revisions with the real Foundations pathway asset.
+- [x] Author uploads are streamed into a separate Academy GridFS bucket with a 100 MB limit, MIME signature checks, SHA-256 deduplication, and draft-private access.
+- [x] Published files support immutable ETags, conditional requests, and byte ranges for audio/video seeking.
+
+### Milestone 6 Retrospective
+
+- **Accomplished:** governed media drafts and review, immutable public manifests, exact lesson revision pins, resilient bundled fallback, author uploads, caption upload support, replacement lineage, localization lineage, and cacheable range delivery.
+- **Key decision:** project teaching assets use a separate Academy GridFS bucket. Learner recordings remain isolated and encrypted; a file becomes public only when an approved published media revision references its exact immutable id.
+- **Learned:** retaining only the newest media revision would break reproducible lesson pins, and accepting arbitrary HTTPS files conflicted with the same-origin content-security policy. The manifest now retains every published revision and delivery stays same-origin.
+- **Technical debt:** abandoned draft uploads need a scheduled orphan-cleanup job. Mongo/GridFS is appropriate for the current free curriculum and 100 MB asset limit, but high-volume or very large video should move behind a reviewed object-storage/CDN adapter later.
+- **Next:** Milestone 7 should use the complete authoring and media workflow to finish Foundations. Interactive Practice remains a separate backlog item and should be designed from real exercise requirements rather than interrupting curriculum work.
 
 ### Milestone 7 - Remaining Foundations Lessons
 
@@ -480,7 +491,8 @@ The existing `interactive_exercise` block remains suitable for simple learner-le
 - Academy history must remain a separate versioned contract. Milestone 4A establishes useful local history; Milestone 4B owns migration, retention, export, deletion, multi-device conflict handling, and explicit sync consent.
 - Existing production course records require the documented one-time version-index migration before a second course revision is saved.
 - Public media needs a storage/CDN strategy; do not put large course video into the existing encrypted-recording vault.
-- Media metadata governance is implemented, but binary upload/storage is intentionally pending a dedicated storage decision. Current assets use reviewed local or HTTPS sources.
+- Abandoned Academy draft uploads need a bounded cleanup job; public files cannot be deleted while a published immutable revision references them.
+- Academy GridFS delivery is capped at 100 MB per asset. Revisit an object-storage/CDN adapter before publishing large or high-traffic video libraries.
 - Lesson authoring must enforce captions, transcripts, evidence, and safety metadata, or content quality will drift.
 - Community features require funded human moderation and safeguarding; they are not a learner MVP dependency.
 - Native iOS work depends on later API-token and audio-provider boundaries described in the iOS readiness plan.
@@ -544,3 +556,4 @@ The existing `interactive_exercise` block remains suitable for simple learner-le
 - **2026-07-17:** Milestone 6 begins with a governed media library. Authors can save incomplete private drafts, but only real checksummed assets with complete rights and kind-specific accessibility metadata may enter role-separated review and publication. Published metadata is exposed through a cacheable read-only manifest; lesson resolution follows in the next focused slice.
 - **2026-07-17:** Governed lesson media uses an exact `{id, version, locale}` reference plus a bundled fallback source. The public manifest retains every published immutable revision, and the learner resolver uses only a kind-matched, fully reviewed exact revision; unavailable delivery never blanks a lesson.
 - **2026-07-17:** Staging published the real “How voice learning works” lesson and a one-lesson Foundations course through the author, reviewer, and publisher identities. The learner player consumed the pinned media revision and its governed alternative text on desktop and 390px mobile. HTML now uses `no-cache` so deployments discover hashed bundles while public manifests remain cacheable.
+- **2026-07-17:** Milestone 6 completed with separate private Academy GridFS storage, strict upload validation, draft-private file access, published immutable range delivery, checksum deduplication, and real pathway illustration v2/replacement plus Finnish localization validation. Remaining large-video/CDN and orphan-cleanup work is recorded as technical debt rather than hidden scope.

docs/staging-academy.md

@@ -15,3 +15,5 @@ The Milestone 5 validation used separate author, reviewer, and publisher session
 Milestone 6 media validation uses the same identities in the **Educational media** section. Save a private asset draft as the author, submit it only after its source, checksum, rights, and accessible alternative are complete, approve all three review areas as the reviewer, and publish as the publisher. `GET /api/academy/media` must remain empty before publication and return the published revision with `Cache-Control: public, max-age=300, stale-while-revalidate=3600` afterward. Published media fields are read-only; replacements and localizations begin as new private revisions.
 
 To validate lesson delivery, pin the asset's exact id, version, and locale in a lesson media block while retaining a bundled fallback URL. Publish that lesson and its course through the same three identities. The learner player must use the governed source and accessibility metadata when the exact revision is present, and preserve the bundled block unchanged when it is absent. The public manifest intentionally retains older published revisions after replacements so existing lesson pins remain reproducible.
+
+To validate storage, upload the pathway illustration as the author and confirm its returned `/api/academy/media/files/...` URL is 404 before publication. Publish a replacement media revision, then confirm the file returns its original bytes, checksum ETag, immutable cache header, and `206 Partial Content` for a byte-range request. Uploading the same file again should return the same source id. A localized asset may reuse that immutable file while supplying separately reviewed localized accessibility text.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "femmevoice",
-  "version": "0.3.33",
+  "version": "0.3.34",
   "private": true,
   "type": "module",
   "scripts": {

server/academy_media_files.py

@@ -0,0 +1,61 @@
+ACADEMY_MEDIA_FILE_LIMIT = 100 * 1024 * 1024
+
+ALLOWED_MEDIA_MIMES = {
+    "image/jpeg", "image/png", "image/webp",
+    "audio/mpeg", "audio/wav", "audio/ogg", "audio/mp4", "audio/webm",
+    "video/mp4", "video/webm",
+    "text/vtt", "application/pdf",
+}
+
+
+def validate_academy_media_file(mime_type, leading_bytes, size):
+    if mime_type not in ALLOWED_MEDIA_MIMES:
+        raise ValueError("Choose a supported image, audio, video, caption, or PDF file.")
+    if not isinstance(size, int) or size < 1 or size > ACADEMY_MEDIA_FILE_LIMIT:
+        raise ValueError("Academy media files must be between 1 byte and 100 MB.")
+    signatures = {
+        "image/jpeg": lambda data: data.startswith(b"\xff\xd8\xff"),
+        "image/png": lambda data: data.startswith(b"\x89PNG\r\n\x1a\n"),
+        "image/webp": lambda data: data.startswith(b"RIFF") and data[8:12] == b"WEBP",
+        "audio/mpeg": lambda data: data.startswith(b"ID3") or (len(data) > 1 and data[0] == 0xFF and data[1] & 0xE0 == 0xE0),
+        "audio/wav": lambda data: data.startswith(b"RIFF") and data[8:12] == b"WAVE",
+        "audio/ogg": lambda data: data.startswith(b"OggS"),
+        "audio/mp4": _is_mp4,
+        "audio/webm": lambda data: data.startswith(b"\x1a\x45\xdf\xa3"),
+        "video/mp4": _is_mp4,
+        "video/webm": lambda data: data.startswith(b"\x1a\x45\xdf\xa3"),
+        "text/vtt": lambda data: data.lstrip(b"\xef\xbb\xbf \t\r\n").startswith(b"WEBVTT"),
+        "application/pdf": lambda data: data.startswith(b"%PDF-"),
+    }
+    if not signatures[mime_type](leading_bytes):
+        raise ValueError("The file contents do not match the selected media type.")
+    return mime_type
+
+
+def parse_byte_range(header, length):
+    if not header:
+        return None
+    if not isinstance(length, int) or length < 1 or not header.startswith("bytes=") or "," in header:
+        raise ValueError("Unsupported byte range.")
+    start_text, separator, end_text = header[6:].partition("-")
+    if not separator or (not start_text and not end_text):
+        raise ValueError("Unsupported byte range.")
+    try:
+        if not start_text:
+            suffix = int(end_text)
+            if suffix < 1:
+                raise ValueError
+            start, end = max(0, length - suffix), length - 1
+        else:
+            start = int(start_text)
+            end = int(end_text) if end_text else length - 1
+            if start < 0 or start >= length or end < start:
+                raise ValueError
+            end = min(end, length - 1)
+    except (TypeError, ValueError):
+        raise ValueError("Unsupported byte range.") from None
+    return start, end
+
+
+def _is_mp4(data):
+    return len(data) >= 12 and data[4:8] == b"ftyp"