refactor(v3-languages): collapse fetch + generate into one script

- Folds scripts/generate_language_table.py into
  scripts/fetch_v3_languages.py. One script now fetches every
  /v3/languages response and regenerates the inline RESOURCES block in
  the snippet (or skips the snippet via --no-snippet). The dedicated
  workflow step is gone; the cron just runs the fetcher.
- Renames snippets/language-table.jsx to
  snippets/supported-languages.jsx so the filename matches the only
  component it exports. Updates the two MDX imports and the workflow
  add-paths to follow.
- Discovers per-resource files from data/v3-languages/*.json instead
  of from a hardcoded list, so new resources (and the manual
  translation_memory.json) appear in the snippet automatically.
- Adds retry-with-backoff for transient API failures: 429 and 5xx
  statuses plus network errors retry up to five times, honouring
  Retry-After when the server sets it. Other HTTP errors raise as
  before.
- Validates response schemas before writing them: /resources must be
  a list of objects with non-empty string `name`; each per-resource
  response must be a list of objects with non-empty string `lang` and
  `name`. A schema mismatch raises with the exact path so a workflow
  failure is actionable.
- Hard-fails (instead of warning + continuing) when the snippet file
  or its BEGIN/END GENERATED markers are missing.
- Trims data/v3-languages/README.md down to the data layout; script
  flags and behaviour now live in the script's module docstring and
  --help output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: Daniel Jones

.github/workflows/refresh-v3-languages.yml

@@ -23,14 +23,11 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v5
 
-      - name: Fetch latest /v3/languages responses
+      - name: Fetch latest /v3/languages responses and refresh snippet
         env:
           DEEPL_AUTH_KEY: ${{ secrets.DEEPL_API_KEY }}
         run: python3 scripts/fetch_v3_languages.py
 
-      - name: Regenerate snippets/language-table.jsx from vended data
-        run: python3 scripts/generate_language_table.py
-
       - name: Open or update refresh PR
         uses: peter-evans/create-pull-request@v6
         with:
@@ -40,7 +37,7 @@ jobs:
           title: "chore(v3-languages): refresh vended responses"
           body: |
             Automated refresh of `data/v3-languages/` from `https://api.deepl.com/v3/languages`,
-            with `snippets/language-table.jsx` regenerated from the new responses. Opened by
+            with `snippets/supported-languages.jsx` regenerated from the new responses. Opened by
             the `refresh-v3-languages` workflow.
 
             This PR is rewritten in place on each run: if the vended responses change again
@@ -51,6 +48,6 @@ jobs:
             closed and the branch deleted automatically.
           add-paths: |
             data/v3-languages/**
-            snippets/language-table.jsx
+            snippets/supported-languages.jsx
           delete-branch: true
           labels: automated, v3-languages

api-reference/improve-text.mdx

@@ -4,7 +4,7 @@ public: true
 sidebarTitle: "Overview"
 ---
 
-import { SupportedLanguages } from "/snippets/language-table.jsx"
+import { SupportedLanguages } from "/snippets/supported-languages.jsx"
 
 <Info>
   **Introducing DeepL API for Write**

data/v3-languages/README.md

@@ -7,28 +7,15 @@ This directory holds verbatim JSON responses from the DeepL `/v3/languages` endp
 | File | Endpoint |
 |---|---|
 | `resources.json` | `GET /v3/languages/resources` |
-| `translate_text.json` | `GET /v3/languages?resource=translate_text&include=beta&include=external` |
-| `translate_document.json` | `GET /v3/languages?resource=translate_document&include=beta&include=external` |
-| `voice.json` | `GET /v3/languages?resource=voice&include=beta&include=external` |
-| `write.json` | `GET /v3/languages?resource=write&include=beta&include=external` |
-| `glossary.json` | `GET /v3/languages?resource=glossary&include=beta&include=external` |
-| `style_rules.json` | `GET /v3/languages?resource=style_rules&include=beta&include=external` |
+| `<resource>.json` | `GET /v3/languages?resource=<resource>&include=beta&include=external` |
 
-Each per-resource file requests `include=beta&include=external` so the vended data is the full superset. Consumers filter on the `status` and per-feature `external` fields when they want a narrower view.
-
-## Refreshing
-
-Set `DEEPL_AUTH_KEY` in your environment, then run:
+where <resource> is one of the resources returned by `/resources` (`translate_text`, `translate_document`,
+`voice`, `write`, etc.).
 
-```sh
-python3 scripts/fetch_v3_languages.py
-```
-
-Flags:
+Each per-resource file requests `include=beta&include=external` so the vended data is the full superset. Consumers filter on the `status` and per-feature `external` fields when they want a narrower view.
 
-- `--free` — hit `https://api-free.deepl.com` instead of the Pro endpoint.
-- `--base-url <url>` — point at any other host (staging, mock, local server). Also configurable via the `DEEPL_API_BASE_URL` environment variable.
+These files are refreshed hourly by the [`refresh-v3-languages`](../../.github/workflows/refresh-v3-languages.yml) GitHub Action, which runs [`scripts/fetch_v3_languages.py`](../../scripts/fetch_v3_languages.py) and opens a pull request whenever the API responses change. See that script's module docstring or `--help` for flags and behaviour (auth, alternate endpoints, manual local refresh).
 
-The script overwrites every file in this directory.
+## `translation_memory.json`
 
-A scheduled GitHub Action refreshes these files automatically and opens a pull request when the responses change; manual runs are only needed for local testing.
+Translation Memory is not yet exposed by `/v3/languages`, so `translation_memory.json` is currently maintained by hand in the shape of a `/v3/languages` response. The fetcher skips it; once the API exposes Translation Memory as a resource, the next refresh will overwrite the manual file with the real response and no other code has to change.

docs/getting-started/supported-languages.mdx

@@ -6,7 +6,7 @@ sidebarTitle: "Languages supported"
 mode: "wide"
 ---
 
-import { SupportedLanguages } from "/snippets/language-table.jsx"
+import { SupportedLanguages } from "/snippets/supported-languages.jsx"
 
 The DeepL API supports the following languages. These can also be retrieved programmatically via the [`/v3/languages` endpoint](/api-reference/languages/retrieve-supported-languages-by-resource), which returns language support per resource along with feature availability (e.g. formality, glossary, auto-detection). The legacy [`/v2/languages` endpoint](/api-reference/languages/retrieve-supported-languages) is also available but deprecated.
 

scripts/fetch_v3_languages.py

@@ -1,33 +1,76 @@
 #!/usr/bin/env python3
-"""Fetch v3/languages responses and write them to data/v3-languages/.
+"""Fetch /v3/languages responses and refresh the vended copies in this repo.
 
-Reads DEEPL_AUTH_KEY from the environment. By default uses the Pro
-endpoint; pass --free to hit api-free.deepl.com, or --base-url to point
-at any other host (e.g. a staging environment or local mock for
-testing). The DEEPL_API_BASE_URL environment variable does the same.
+What this script does, in order:
 
-The fetcher requests every resource with ?include=beta&include=external so
-the vended files contain the full superset of languages and features.
-Consumers filter client-side using the status field on languages/features.
+1. Calls `GET /v3/languages/resources` and, for every resource, calls
+   `GET /v3/languages?resource=<name>&include=beta&include=external`.
+2. Writes each response verbatim to `data/v3-languages/<name>.json`
+   (plus `resources.json`). These files are the single source of truth
+   that the rest of the repo reads.
+3. Regenerates the inline `RESOURCES` object in
+   `snippets/supported-languages.jsx` between its `// BEGIN GENERATED` /
+   `// END GENERATED` markers so the `<SupportedLanguages>` snippet
+   stays in sync with the vended JSON. Mintlify's snippet sandbox does
+   not honour ES module imports, so the data has to ship inline.
+
+Auth and endpoint:
+
+  - Reads `DEEPL_AUTH_KEY` from the environment (required).
+  - Defaults to the Pro endpoint (`https://api.deepl.com`).
+  - `--free` switches to `https://api-free.deepl.com`.
+  - `--base-url <url>` (or `DEEPL_API_BASE_URL`) points at any other host,
+    e.g. a staging environment or a local mock for testing. `--base-url`
+    and `--free` are mutually exclusive.
+
+Other flags:
+
+  - `--out <dir>` overrides the JSON output directory (defaults to
+    `data/v3-languages/` relative to this repo).
+  - `--no-snippet` skips step 3 when you only want to refresh the JSON.
+
+The hourly `refresh-v3-languages` GitHub Action runs this script and
+opens a pull request whenever the responses change. Manual runs are
+only needed for local testing or after editing the script itself.
 """
 from __future__ import annotations
 
 import argparse
 import json
 import os
 import sys
+import time
 import urllib.error
 import urllib.parse
 import urllib.request
 from pathlib import Path
 
 INCLUDES = ("beta", "external")
-DATA_DIR = Path(__file__).resolve().parent.parent / "data" / "v3-languages"
+REPO_ROOT = Path(__file__).resolve().parent.parent
+DATA_DIR = REPO_ROOT / "data" / "v3-languages"
+SNIPPET = REPO_ROOT / "snippets" / "supported-languages.jsx"
 PRO_URL = "https://api.deepl.com"
 FREE_URL = "https://api-free.deepl.com"
+BEGIN_MARKER = "    // BEGIN GENERATED: RESOURCES (do not edit; run scripts/fetch_v3_languages.py)"
+END_MARKER = "    // END GENERATED"
+
+# Retry config for transient failures (429s, 5xx, network errors).
+MAX_RETRIES = 5
+INITIAL_BACKOFF_S = 1.0
+RETRYABLE_STATUSES = frozenset({429, 500, 502, 503, 504})
+
+def _retry_delay(attempt: int, retry_after: str | None) -> float:
+    """Honour Retry-After (seconds) when present; otherwise exponential backoff."""
+    if retry_after:
+        try:
+            return max(0.0, float(retry_after))
+        except ValueError:
+            # HTTP-date form is rare; fall through to backoff.
+            pass
+    return INITIAL_BACKOFF_S * (2 ** attempt)
 
 
-def get(url: str, key: str) -> object:
+def http_get_json(url: str, key: str) -> object:
     req = urllib.request.Request(
         url,
         headers={
@@ -36,36 +79,138 @@ def get(url: str, key: str) -> object:
             "User-Agent": "deepl-api-docs-vendor/1.0",
         },
     )
-    with urllib.request.urlopen(req, timeout=30) as resp:
-        return json.loads(resp.read())
+    for attempt in range(MAX_RETRIES + 1):
+        try:
+            with urllib.request.urlopen(req, timeout=30) as resp:
+                return json.loads(resp.read())
+        except urllib.error.HTTPError as e:
+            if e.code in RETRYABLE_STATUSES and attempt < MAX_RETRIES:
+                delay = _retry_delay(attempt, e.headers.get("Retry-After"))
+                print(
+                    f"retry {attempt + 1}/{MAX_RETRIES}: {e.code} {e.reason} from {url} "
+                    f"(sleeping {delay:.1f}s)",
+                    file=sys.stderr,
+                )
+                time.sleep(delay)
+                continue
+            raise
+        except urllib.error.URLError as e:
+            # Network-level failure (connection reset, DNS, timeout, etc.).
+            if attempt < MAX_RETRIES:
+                delay = _retry_delay(attempt, None)
+                print(
+                    f"retry {attempt + 1}/{MAX_RETRIES}: network error {e.reason} "
+                    f"from {url} (sleeping {delay:.1f}s)",
+                    file=sys.stderr,
+                )
+                time.sleep(delay)
+                continue
+            raise
+    # Unreachable: loop either returns or raises.
+    raise RuntimeError("retry loop exited unexpectedly")
 
 
-def fetch(base_url: str, key: str) -> dict[str, object]:
-    resources = get(f"{base_url}/v3/languages/resources", key)
+def _require_str(value: object, path: str) -> str:
+    if not isinstance(value, str) or not value:
+        raise RuntimeError(f"unexpected schema at {path}: want non-empty string, got {value!r}")
+    return value
+
+
+def _validate_resources(resources: object) -> list[dict]:
     if not isinstance(resources, list):
-        raise RuntimeError(f"Unexpected resources payload: {resources!r}")
+        raise RuntimeError(f"unexpected /v3/languages/resources payload: want list, got {type(resources).__name__}")
+    validated: list[dict] = []
+    for i, entry in enumerate(resources):
+        if not isinstance(entry, dict):
+            raise RuntimeError(f"unexpected schema at resources[{i}]: want object, got {type(entry).__name__}")
+        _require_str(entry.get("name"), f"resources[{i}].name")
+        validated.append(entry)
+    return validated
+
+
+def _validate_languages(payload: object, resource: str) -> list[dict]:
+    if not isinstance(payload, list):
+        raise RuntimeError(
+            f"unexpected /v3/languages?resource={resource} payload: "
+            f"want list, got {type(payload).__name__}"
+        )
+    for i, entry in enumerate(payload):
+        if not isinstance(entry, dict):
+            raise RuntimeError(
+                f"unexpected schema at {resource}[{i}]: want object, got {type(entry).__name__}"
+            )
+        _require_str(entry.get("lang"), f"{resource}[{i}].lang")
+        _require_str(entry.get("name"), f"{resource}[{i}].name")
+    return payload
+
 
-    out: dict[str, object] = {"resources.json": resources}
+def fetch(base_url: str, key: str) -> dict[str, object]:
+    resources = _validate_resources(
+        http_get_json(f"{base_url}/v3/languages/resources", key)
+    )
+
+    payloads: dict[str, object] = {"resources.json": resources}
     qs = urllib.parse.urlencode([("include", v) for v in INCLUDES], doseq=True)
     for entry in resources:
         name = entry["name"]
-        langs = get(f"{base_url}/v3/languages?resource={name}&{qs}", key)
-        out[f"{name}.json"] = langs
-    return out
+        langs = _validate_languages(
+            http_get_json(f"{base_url}/v3/languages?resource={name}&{qs}", key),
+            name,
+        )
+        payloads[f"{name}.json"] = langs
+    return payloads
 
 
-def write_files(payloads: dict[str, object], out_dir: Path) -> list[Path]:
+def write_json_files(payloads: dict[str, object], out_dir: Path) -> list[Path]:
     out_dir.mkdir(parents=True, exist_ok=True)
     written: list[Path] = []
     for filename, payload in payloads.items():
         path = out_dir / filename
         with path.open("w", encoding="utf-8") as f:
-            json.dump(payload, f, indent=2, ensure_ascii=False, sort_keys=False)
+            json.dump(payload, f, indent=2, ensure_ascii=False)
             f.write("\n")
         written.append(path)
     return written
 
 
+def render_resources_block(data_dir: Path) -> str:
+    """Build the `const RESOURCES = {...}` literal from every per-resource
+    JSON file in `data_dir`. `resources.json` (the index) is skipped; every
+    other `*.json` file is treated as a resource response, including manual
+    files like `translation_memory.json`.
+    """
+    lines = [BEGIN_MARKER, "    const RESOURCES = {"]
+    for path in sorted(data_dir.glob("*.json")):
+        if path.stem == "resources":
+            continue
+        entries = json.loads(path.read_text(encoding="utf-8"))
+        lines.append(f'        {json.dumps(path.stem)}: [')
+        for entry in entries:
+            lines.append(f"            {json.dumps(entry, ensure_ascii=False)},")
+        lines.append("        ],")
+    lines.append("    }")
+    lines.append(END_MARKER)
+    return "\n".join(lines)
+
+
+def update_snippet(data_dir: Path) -> bool:
+    if not SNIPPET.exists():
+        raise SystemExit(f"error: snippet not found at {SNIPPET}")
+    text = SNIPPET.read_text(encoding="utf-8")
+    if BEGIN_MARKER not in text or END_MARKER not in text:
+        raise SystemExit(
+            f"error: {SNIPPET} is missing the BEGIN/END GENERATED markers"
+        )
+    new_block = render_resources_block(data_dir)
+    start = text.index(BEGIN_MARKER)
+    end = text.index(END_MARKER, start) + len(END_MARKER)
+    updated = text[:start] + new_block + text[end:]
+    if updated == text:
+        return False
+    SNIPPET.write_text(updated, encoding="utf-8")
+    return True
+
+
 def resolve_base_url(cli_base_url: str | None, free: bool) -> str:
     if cli_base_url and free:
         raise SystemExit("error: --base-url and --free are mutually exclusive")
@@ -96,7 +241,12 @@ def main() -> int:
         "--out",
         type=Path,
         default=DATA_DIR,
-        help=f"Output directory (default: {DATA_DIR})",
+        help=f"Output directory for the vended JSON (default: {DATA_DIR})",
+    )
+    parser.add_argument(
+        "--no-snippet",
+        action="store_true",
+        help="Skip regenerating snippets/supported-languages.jsx",
     )
     args = parser.parse_args()
 
@@ -112,9 +262,16 @@ def main() -> int:
         print(f"error: {e.code} {e.reason} from {e.url}", file=sys.stderr)
         return 1
 
-    paths = write_files(payloads, args.out)
+    paths = write_json_files(payloads, args.out)
     for p in paths:
         print(p)
+
+    if not args.no_snippet:
+        if update_snippet(args.out):
+            print(SNIPPET)
+        else:
+            print(f"{SNIPPET}: up to date")
+
     return 0