docs(fern): finalize preview workflow and nav cleanup (#784)

Author: pimlock

.agents/skills/update-docs/SKILL.md

@@ -10,8 +10,8 @@ Scan recent git history for commits that affect user-facing behavior and draft d
 ## Prerequisites
 
 - You must be in the OpenShell git repository.
-- The `docs/` directory must exist with the current doc set.
-- Read `docs/CONTRIBUTING.md` before writing any content. It contains the style guide and formatting rules.
+- The published docs tree must exist under `fern/pages/`.
+- Read `fern/pages/CONTRIBUTING.mdx` before writing any content. It contains the current style guide and formatting rules.
 
 ## When to Use
 
@@ -48,16 +48,16 @@ For each relevant commit, determine which doc page(s) it affects. Use this mappi
 
 | Code area | Likely doc page(s) |
 |---|---|
-| `crates/openshell-cli/` (gateway commands) | `docs/sandboxes/manage-gateways.md` |
-| `crates/openshell-cli/` (sandbox commands) | `docs/sandboxes/manage-sandboxes.md` |
-| `crates/openshell-cli/` (provider commands) | `docs/sandboxes/manage-providers.md` |
-| `crates/openshell-cli/` (new top-level command) | May need a new page or `docs/reference/` entry |
-| `crates/openshell-proxy/` or policy code | `docs/sandboxes/policies.md`, `docs/reference/policy-schema.md` |
-| `crates/openshell-inference/` | `docs/inference/configure.md` |
-| `python/` (SDK changes) | `docs/reference/` or `docs/get-started/quickstart.md` |
-| `proto/` (API changes) | `docs/reference/` |
-| `deploy/` (Dockerfile, Helm) | `docs/sandboxes/manage-gateways.md`, `docs/about/architecture.md` |
-| Community sandbox definitions | `docs/sandboxes/community-sandboxes.md` |
+| `crates/openshell-cli/` (gateway commands) | `fern/pages/sandboxes/manage-gateways.mdx` |
+| `crates/openshell-cli/` (sandbox commands) | `fern/pages/sandboxes/manage-sandboxes.mdx` |
+| `crates/openshell-cli/` (provider commands) | `fern/pages/sandboxes/manage-providers.mdx` |
+| `crates/openshell-cli/` (new top-level command) | May need a new page or `fern/pages/reference/` entry |
+| Proxy or policy code | `fern/pages/sandboxes/policies.mdx`, `fern/pages/reference/policy-schema.mdx` |
+| Inference code | `fern/pages/inference/configure.mdx` |
+| `python/` (SDK changes) | `fern/pages/reference/` or `fern/pages/get-started/quickstart.mdx` |
+| `proto/` (API changes) | `fern/pages/reference/` |
+| `deploy/` (Dockerfile, Helm) | `fern/pages/sandboxes/manage-gateways.mdx`, `fern/pages/about/architecture.mdx` |
+| Community sandbox definitions | `fern/pages/sandboxes/community-sandboxes.mdx` |
 
 If a commit does not map to any existing page but introduces a user-visible concept, flag it as needing a new page.
 
@@ -88,16 +88,22 @@ Identify where the new content should go. Follow the page's existing structure.
 
 ## Step 5: Draft the Update
 
-Write the doc update following the rules in `docs/CONTRIBUTING.md`. Key reminders:
+Write the doc update following the rules in `fern/pages/CONTRIBUTING.mdx`. Key reminders:
 
 - **Active voice, present tense, second person.**
 - **No unnecessary bold.** Reserve bold for UI labels and parameter names.
 - **No em dashes** unless used sparingly. Prefer commas or separate sentences.
 - **Start sections with an introductory sentence** that orients the reader.
 - **No superlatives.** Say what the feature does, not how great it is.
-- **Code examples use `console` language** with `$` prompt prefix.
+- **Code examples use `shell` language** for copyable commands, with no `$` prompt prefix.
+- **Use `text` fences** for transcripts, logs, or shell sessions that should not be copied verbatim.
 - **Include the SPDX header** if creating a new page.
-- **Match existing frontmatter format** if creating a new page.
+- **Match existing Fern frontmatter format** if creating a new page, including `sidebar-title`, `keywords`, `tags`, and `position` when they are relevant. Use frontmatter `slug` only for folder-discovered pages or absolute URL overrides.
+- **Use `sidebar-title` for short nav labels**. For explicit navbar entries, keep relative `slug` values in `fern/versions/latest.yml` instead of page frontmatter.
+- **Keep explicit `page:` entries in `fern/versions/latest.yml`**. Fern still requires them. If the page defines `sidebar-title`, set `page:` to that value. Otherwise set `page:` to the page frontmatter `title`.
+- **Use `skip-slug: true` in `fern/versions/latest.yml`** when a child page should live at the parent section path.
+- **Use `keywords` as a comma-separated string**. When migrating a page from `docs/`, combine the legacy `topics` and `tags` into `keywords` and preserve the legacy `tags` array when it is still useful.
+- **Do not add a duplicate H1**. Fern renders the page title from frontmatter.
 - **Always write NVIDIA in all caps.** Wrong: Nvidia, nvidia.
 - **Always capitalize OpenShell correctly.** Wrong: openshell, Openshell, openShell.
 - **Do not number section titles.** Wrong: "Section 1: Deploy a Gateway" or "Step 3: Verify." Use plain descriptive titles.
@@ -112,8 +118,8 @@ When updating an existing page:
 
 When creating a new page:
 
-- Follow the frontmatter template from `docs/CONTRIBUTING.md`.
-- Add the page to the appropriate `toctree` in `docs/index.md`.
+- Follow the frontmatter template from `fern/pages/CONTRIBUTING.mdx`.
+- Add the page to the appropriate section in `fern/versions/latest.yml`.
 
 ## Step 6: Present the Results
 
@@ -123,8 +129,8 @@ After drafting all updates, present a summary to the user:
 ## Doc Updates from Commits
 
 ### Updated pages
-- `docs/sandboxes/manage-gateways.md`: Added `--gpu` flag documentation (from commit abc1234).
-- `docs/reference/policy-schema.md`: Updated network policy schema for new `tls_inspect` field (from commit def5678).
+- `fern/pages/sandboxes/manage-gateways.mdx`: Added `--gpu` flag documentation (from commit abc1234).
+- `fern/pages/reference/policy-schema.mdx`: Updated network policy schema for new `tls_inspect` field (from commit def5678).
 
 ### New pages needed
 - None (or list any new pages created).
@@ -136,23 +142,29 @@ After drafting all updates, present a summary to the user:
 
 ## Step 7: Build and Verify
 
-After making changes, build the docs locally:
+After making changes, validate the Fern docs locally:
 
 ```bash
-mise run docs
+fern check
+```
+
+If a human needs to inspect rendering while iterating, they can also run:
+
+```bash
+fern docs dev
 ```
 
 Check for:
 
-- Build warnings or errors.
+- Validation warnings or errors.
 - Broken cross-references.
-- Correct rendering of new content.
+- Correct rendering of new content in the PR preview when available.
 
 ## Tips
 
 - When in doubt about whether a commit needs a doc update, check if the commit message references a CLI flag, config option, or user-visible behavior.
 - Group related commits that touch the same doc page into a single update rather than making multiple small edits.
-- If a commit is a breaking change, add a note at the top of the relevant section using a `:::{warning}` admonition.
+- If a commit is a breaking change, add a note at the top of the relevant section using a Fern `<Warning>` callout.
 - PRs that are purely internal refactors with no behavior change do not need doc updates, even if they touch high-signal directories.
 
 ## Example Usage
@@ -165,4 +177,4 @@ User says: "Catch up the docs for everything merged since v0.2.0."
 4. Read the commit diffs and current doc pages.
 5. Draft updates following the style guide.
 6. Present the summary.
-7. Build with `mise run docs` to verify.
+7. Run `fern check` to verify.

.github/workflows/branch-docs.yml

@@ -0,0 +1,81 @@
+name: Branch Docs Preview
+
+on:
+  pull_request:
+    paths:
+      - "fern/**"
+      - ".github/workflows/branch-docs.yml"
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  preview:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Check Fern preview availability
+        id: fern-preview
+        env:
+          FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
+        run: |
+          if [ -n "$FERN_TOKEN" ]; then
+            echo "enabled=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "enabled=false" >> "$GITHUB_OUTPUT"
+            echo "::notice::Skipping Fern docs preview because FERN_TOKEN is unavailable. This is expected for fork PRs and repos without Fern preview credentials configured."
+          fi
+
+      - name: Setup Node.js
+        if: ${{ steps.fern-preview.outputs.enabled == 'true' }}
+        uses: actions/setup-node@v6
+        with:
+          node-version: "24"
+
+      - name: Install Fern CLI
+        if: ${{ steps.fern-preview.outputs.enabled == 'true' }}
+        run: npm install -g fern-api
+
+      - name: Generate preview URL
+        if: ${{ steps.fern-preview.outputs.enabled == 'true' }}
+        id: generate-docs
+        env:
+          FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
+          PREVIEW_ID: pr-${{ github.event.pull_request.number }}
+        working-directory: ./fern
+        run: |
+          OUTPUT=$(fern generate --docs --preview --id "$PREVIEW_ID" 2>&1)
+          echo "$OUTPUT"
+          URL=$(echo "$OUTPUT" | grep -oP 'Published docs to \K.*(?= \()')
+          if [ -z "$URL" ]; then
+            echo "::error::Failed to generate preview URL. See fern output above."
+            exit 1
+          fi
+          echo "preview_url=$URL" >> "$GITHUB_OUTPUT"
+
+      - name: Post or update PR comment
+        if: ${{ steps.fern-preview.outputs.enabled == 'true' }}
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          PREVIEW_URL: ${{ steps.generate-docs.outputs.preview_url }}
+        run: |
+          BODY=":herb: **Preview your docs:** <${PREVIEW_URL}>"
+          MARKER="<!-- preview-docs -->"
+          BODY="${BODY}
+
+          ${MARKER}"
+
+          COMMENT_ID=$(gh api "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \
+            --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" | head -1)
+
+          if [ -n "$COMMENT_ID" ]; then
+            gh api "repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
+              -X PATCH -f body="$BODY"
+          else
+            gh api "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \
+              -f body="$BODY"
+          fi

.github/workflows/release-tag.yml

@@ -383,6 +383,30 @@ jobs:
             release/*.whl
             release/openshell-checksums-sha256.txt
 
+  publish-fern-docs:
+    name: Publish Fern Docs
+    needs: [release]
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.tag || github.ref }}
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: "24"
+
+      - name: Install Fern CLI
+        run: npm install -g fern-api
+
+      - name: Publish Fern docs
+        env:
+          FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
+        working-directory: ./fern
+        run: fern generate --docs
+
   trigger-wheel-publish:
     name: Trigger Wheel Publish
     needs: [compute-versions, build-python-wheels, release]

AGENTS.md

@@ -43,6 +43,8 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `python/openshell/` | Python SDK | Python bindings and CLI packaging |
 | `proto/` | Protobuf definitions | gRPC service contracts |
 | `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests |
+| `fern/` | Published docs | Fern site config, navigation, components, and MDX pages |
+| `docs/` | Legacy docs source | Sphinx/MyST source retained for migration, comparison, and legacy build tasks |
 | `.agents/skills/` | Agent skills | Workflow automation for development |
 | `.agents/agents/` | Agent personas | Sub-agent definitions (e.g., reviewer, doc writer) |
 | `architecture/` | Architecture docs | Design decisions and component documentation |
@@ -185,8 +187,10 @@ ocsf_emit!(event);
 ## Documentation
 
 - When making changes, update the relevant documentation in the `architecture/` directory.
-- When changes affect user-facing behavior, also update the relevant pages under `docs/`.
-- Follow the style guide in [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md): active voice, no unnecessary bold, no em dash overuse, no filler introductions.
+- When changes affect user-facing behavior, update the relevant published docs pages under `fern/pages/` and navigation in `fern/versions/latest.yml`.
+- `docs/` is retained for legacy Sphinx build tasks and migration/reference work. Do not update it unless the user explicitly asks.
+- Follow the docs style guide in [fern/pages/CONTRIBUTING.mdx](fern/pages/CONTRIBUTING.mdx): active voice, minimal formatting, no filler introductions, `shell` fences for copyable commands, and no duplicate body H1.
+- Fern PR previews run through `.github/workflows/branch-docs.yml`, and production publish runs through the `publish-fern-docs` job in `.github/workflows/release-tag.yml`.
 - Use the `update-docs` skill to scan recent commits and draft doc updates.
 
 ## Security

CONTRIBUTING.md

@@ -173,7 +173,7 @@ These are the primary `mise` tasks for day-to-day development:
 | `mise run test`    | Default test suite                                      |
 | `mise run e2e`     | Default end-to-end test lane                            |
 | `mise run ci`      | Full local CI checks (lint, compile/type checks, tests) |
-| `mise run docs`    | Build and serve documentation locally                   |
+| `mise run docs`    | Build legacy Sphinx docs locally                        |
 | `mise run clean`   | Clean build artifacts                                   |
 
 ## Project Structure
@@ -185,9 +185,10 @@ These are the primary `mise` tasks for day-to-day development:
 | `proto/`        | Protocol buffer definitions                   |
 | `tasks/`        | `mise` task definitions and build scripts     |
 | `deploy/`       | Dockerfiles, Helm chart, Kubernetes manifests |
+| `fern/`         | Published Fern docs site and MDX pages        |
+| `docs/`         | Legacy Sphinx/MyST docs retained during transition |
 | `architecture/` | Architecture docs and plans                   |
 | `rfc/`          | Request for Comments proposals                |
-| `docs/`         | User-facing documentation (Sphinx/MyST)       |
 | `.agents/`      | Agent skills and persona definitions          |
 
 ## RFCs
@@ -196,19 +197,30 @@ For cross-cutting architectural decisions, API contract changes, or process prop
 
 ## Documentation
 
-If your change affects user-facing behavior (new flags, changed defaults, new features, bug fixes that contradict existing docs), update the relevant pages under `docs/` in the same PR.
+If your change affects user-facing behavior (new flags, changed defaults, new features, bug fixes that contradict existing docs), update the relevant pages under `fern/pages/` in the same PR and adjust `fern/versions/latest.yml` if navigation changes. For explicit navbar entries, keep `page:` aligned with `sidebar-title` when present and put relative `slug:` values in `fern/versions/latest.yml`. Reserve frontmatter `slug` for folder-discovered pages or absolute URL overrides.
 
 To ensure your doc changes follow NVIDIA documentation style, use the `update-docs` skill.
-It scans commits, identifies doc pages that need updates, and drafts content that follows the style guide in `docs/CONTRIBUTING.md`.
+It scans commits, identifies doc pages that need updates, and drafts content that follows the style guide in `fern/pages/CONTRIBUTING.mdx`.
 
-To build and preview docs locally:
+To preview Fern docs locally:
 
 ```bash
-mise run docs # to build the docs locally
-mise run docs:serve # to serve locally and automatically rebuild on changes
+fern docs dev
 ```
 
-See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for more details.
+To run non-interactive validation:
+
+```bash
+fern check
+```
+
+PRs that touch `fern/**` also get a preview from `.github/workflows/branch-docs.yml` when `FERN_TOKEN` is available to the workflow.
+
+Fern docs publishing is handled by the `publish-fern-docs` job in `.github/workflows/release-tag.yml` when a release tag is created.
+
+`docs/` and `mise run docs` / `mise run docs:serve` are still kept for the legacy Sphinx build during the transition, but they are no longer the primary published docs workflow.
+
+See [fern/pages/CONTRIBUTING.mdx](fern/pages/CONTRIBUTING.mdx) for the current docs authoring guide.
 
 ## Pull Requests