docs: NVIDIA/Blackwell support notes + standalone test harness

contrib/nvidia-standalone-test.sh

@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+#
+# nvidia-standalone-test.sh — safe, guarded test of gamescope's standalone DRM
+# backend (and HDR) on NVIDIA, with an auto-revert watchdog so a wedged run
+# never strands you at a black screen.
+#
+# HOW TO RUN (important):
+#   1. From your desktop, switch to a free TTY:  Ctrl+Alt+F3
+#   2. Log in, then:  ~/dev/gamescope/contrib/nvidia-standalone-test.sh
+#   3. It runs gamescope standalone for N seconds, logs everything, then exits.
+#   4. Switch back to your desktop:  Ctrl+Alt+F2  (or whichever VT it was on)
+#
+# A background watchdog SIGKILLs gamescope and chvt's you back even if it hangs.
+#
+set -euo pipefail
+
+DURATION="${1:-12}"                       # seconds to run gamescope
+GS="${GS:-$HOME/dev/gamescope/build/src/gamescope}"
+CLIENT="${CLIENT:-vkcube}"
+HDR="${HDR:-0}"                            # HDR=1 to test --hdr-enabled
+LOG="${LOG:-/tmp/gamescope-nvidia-$(date +%s 2>/dev/null || echo run).log}"
+
+die() { echo "FATAL: $*" >&2; exit 1; }
+
+# ── Safety: refuse to run inside a graphical session (it would fight for DRM) ──
+if [ -n "${WAYLAND_DISPLAY:-}" ] || [ -n "${DISPLAY:-}" ]; then
+    die "Detected WAYLAND_DISPLAY/DISPLAY — you are inside a desktop session.
+     Run this from a bare TTY (Ctrl+Alt+F3), not from a terminal in COSMIC."
+fi
+[ -x "$GS" ] || die "gamescope binary not found/executable at: $GS"
+command -v "$CLIENT" >/dev/null || die "test client '$CLIENT' not installed"
+
+ORIG_VT="$(fgconsole 2>/dev/null || echo '')"
+
+echo "=== gamescope NVIDIA standalone test ==="
+echo "binary:   $GS ($("$GS" --help 2>&1 | head -1 || true))"
+echo "client:   $CLIENT   HDR: $HDR   duration: ${DURATION}s"
+echo "log:      $LOG"
+echo
+
+# ── Diagnostics captured regardless of whether gamescope succeeds ─────────────
+{
+    echo "### nvidia driver"; nvidia-smi --query-gpu=driver_version,name --format=csv,noheader 2>/dev/null || true
+    echo "### nvidia_drm.modeset"; cat /sys/module/nvidia_drm/parameters/modeset 2>/dev/null || true
+    echo "### DRM cards"; ls -l /sys/class/drm/ 2>/dev/null | grep -E 'card[0-9]+($|-)' || true
+    echo "### explicit-sync caps (DRM_CAP_SYNCOBJ / TIMELINE)"
+    for c in /dev/dri/card*; do echo "-- $c"; done
+} >> "$LOG" 2>&1
+
+# ── Watchdog: guarantees we return to the desktop even if gamescope wedges ────
+watchdog() {
+    sleep "$((DURATION + 8))"
+    pkill -KILL -x gamescope 2>/dev/null || true
+    [ -n "$ORIG_VT" ] && chvt "$ORIG_VT" 2>/dev/null || true
+}
+watchdog &
+WD_PID=$!
+trap 'kill "$WD_PID" 2>/dev/null || true; [ -n "$ORIG_VT" ] && chvt "$ORIG_VT" 2>/dev/null || true' EXIT
+
+# ── Run gamescope standalone (DRM backend), time-boxed ────────────────────────
+GS_ARGS=( -W 2560 -H 1440 -O '*' )
+[ "$HDR" = "1" ] && GS_ARGS+=( --hdr-enabled --hdr-debug-force-output )
+
+echo "launching: $GS ${GS_ARGS[*]} -- $CLIENT"  | tee -a "$LOG"
+set +e
+timeout --signal=TERM "$DURATION" \
+    "$GS" "${GS_ARGS[@]}" -- "$CLIENT" >> "$LOG" 2>&1
+rc=$?
+set -e
+
+echo
+echo "=== gamescope exited rc=$rc (124 = ran full ${DURATION}s = good) ==="
+echo "--- last 30 log lines ---"
+tail -n 30 "$LOG"
+echo
+echo "Full log: $LOG"
+echo "Now switch back to your desktop: Ctrl+Alt+F${ORIG_VT:-2}"

docs/NVIDIA.md

@@ -0,0 +1,83 @@
+# gamescope on NVIDIA (Blackwell / RTX 5090, driver 610)
+
+Status and notes for running gamescope on the NVIDIA proprietary driver, from
+testing on an RTX 5090 (Blackwell, sm120), driver 610.43.02, kernel 7.0.12-tkg-bore-llvm,
+under COSMIC (Wayland).
+
+## What works today
+
+- **Builds cleanly** against the vendored wlroots 0.19.3 (no NVIDIA-specific
+  build flags needed).
+- **Nested mode works** (gamescope as a Wayland client of another compositor —
+  e.g. COSMIC). Verified: the 5090 is selected as the render device, a flip-mode
+  swapchain is created, `VK_EXT_swapchain_maintenance1` is used, runs at the
+  output's refresh:
+
+  ```
+  Selected GPU 1: NVIDIA GeForce RTX 5090, type: DiscreteGpu
+  [Gamescope WSI] Forcing on VK_EXT_swapchain_maintenance1.
+  [Gamescope WSI] Created swapchain ... imageCount: 3 ... flip: true
+  ```
+
+- **Explicit-sync quirk is already handled**: the NVIDIA 555+ drivers advertise
+  `DRM_CAP_SYNCOBJ` but reject `IN_FENCE_FD` with `EPERM`; gamescope detects this
+  and disables `IN_FENCE_FD` on the first failing commit (see
+  `Backends/DRMBackend.cpp`, "NVIDIA 555 series" comment).
+
+## The real NVIDIA gaps
+
+1. **Standalone DRM backend** (gamescope as the session compositor directly on
+   KMS, not nested). This is where NVIDIA historically diverges from AMD:
+   atomic plane assignment via libliftoff, modeset, and explicit sync all run
+   against `nvidia-drm` instead of `amdgpu`. Needs on-hardware validation —
+   use `contrib/nvidia-standalone-test.sh` (guarded, auto-reverts).
+
+2. **HDR colour management.** gamescope's DRM HDR pipeline offloads the colour
+   transform to **AMD-only** plane properties — `AMD_PLANE_CTM`,
+   `AMD_PLANE_SHAPER_LUT/TF`, `AMD_PLANE_LUT3D`, `AMD_PLANE_BLEND_TF` (see
+   `DRMBackend.cpp`). NVIDIA exposes none of these. The HDR10 *signalling*
+   (`HDR_OUTPUT_METADATA` + `Colorspace`) NVIDIA *does* support, and gamescope's
+   GPU-agnostic **Vulkan composite** colour path (`color_helpers.cpp`,
+   `rendervulkan.cpp`) can perform the tone-mapping in-shader. The work is to
+   confirm gamescope cleanly falls back to the Vulkan colour path on NVIDIA when
+   the AMD plane props are absent, and that HDR output is correct — rather than
+   silently producing a wrong/clipped image.
+
+## Known crash: SPIR-V compiler SEGV on driver 610 (Blackwell)
+
+On driver 610.43.02 (RTX 5090), gamescope reproducibly **SIGSEGVs while compiling
+its composite/upscale shaders**, inside the NVIDIA SPIR-V→NVVM compiler. It dies
+right after backend init / EDID patch, before the first swapchain; nested clients
+then report `failed to read Wayland events: Broken pipe`.
+
+Backtrace (coredumpctl):
+
+```
+Signal: 11 (SEGV)
+#0  libnvidia-glvkspirv.so.610.43.02 + 0x9dd20
+...
+#10 _nv002nvvm (libnvidia-glvkspirv.so.610.43.02 + 0xa2816)
+#11 libnvidia-eglcore.so.610.43.02 + 0xd3061d
+```
+
+Notes:
+- Not the shader disk cache — purging `~/.cache/nvidia` and setting
+  `__GL_SHADER_DISK_CACHE=0` does not help.
+- Affects both nested and standalone (`--backend drm`) paths — same compile step.
+- Next: bisect which composite/upscale shader trips the compiler, capture a
+  minimal repro, file with NVIDIA, and add a gamescope-side fallback so startup
+  degrades instead of crashing.
+
+## Testing
+
+```sh
+# nested smoke test (safe, runs inside COSMIC):
+WAYLAND_DISPLAY=wayland-1 gamescope -W 1280 -H 720 -b -- vkcube
+
+# standalone + HDR (run from a bare TTY — Ctrl+Alt+F3 — never inside COSMIC):
+contrib/nvidia-standalone-test.sh            # standalone SDR
+HDR=1 contrib/nvidia-standalone-test.sh      # standalone HDR
+```
+
+The harness captures driver/DRM diagnostics, time-boxes the run, and has a
+watchdog that SIGKILLs gamescope and `chvt`s you back if it ever wedges.