graphics plans: record the device-memory fault defect (nextbsd-kernel#71)

A userland reproducer isolated an indefinite block affecting any driver whose
buffers live in device memory. The trigger is the scanout pin: a VRAM BO sits in
system memory until SETCRTC migrates it, after which every fault feeds a
device-BAR pfn to lkpi_vmf_insert_pfn_prot_locked(), which assumes managed shmem
pages, and the faulting thread never wakes.

  arm64 options   vmwgfx is blocked by this BEFORE its porting cost is reached --
                  it is TTM/VRAM-based and takes the identical path. A hard
                  prerequisite the plan did not previously state.

  virtio-gpu      shmem backing sidesteps the defect entirely, taking the same
                  branch i915 uses (verified working on hardware). Turns the shmem
                  helper from purely a cost into partly a benefit. Caveat recorded
                  for blob resources, which would map device memory.

  fallback        the VRAM-helper option gains a second, harder strike; the TTM
                  system-domain recommendation now has a correctness basis rather
                  than only Wayland capacity.

i915 on real hardware is confirmed unaffected.

Author: pkgdemon

nextbsd-arm64-kms-options.html

@@ -274,6 +274,9 @@ <h3>Option E &mdash; port Linux's virtio-gpu DRM driver</h3>
 
 <div class="opt">
 <h3>Option F &mdash; port vmwgfx</h3>
+<div class="callout callout-bad">
+<p><span class="pill pill-bad">Prerequisite</span> <strong>vmwgfx is blocked by <a href="https://github.com/nextbsd-redux/nextbsd-kernel/issues/71">nextbsd-kernel#71</a> before any of the porting cost below is even reached.</strong> vmwgfx is heavily TTM-based with VRAM-resident scanout buffers, so it takes the identical <code>is_iomem</code> fault path that currently blocks indefinitely for bochs: once a buffer is pinned for scanout it moves into device memory, and the fault handler feeds a device-BAR pfn to a function that assumes managed shmem pages. Porting vmwgfx without fixing that path first yields a driver that binds, creates <code>card0</code>, and then hangs on first access &mdash; precisely the bochs outcome.</p>
+</div>
 <p><span class="pill pill-warn">Genuinely supports ARM64 &mdash; but largest, and one hypervisor</span> <span class="verdict">Cost: months. 1.12 MB across 57 files.</span></p>
 <p>Contrary to an assumption made earlier in this project, <strong>vmwgfx is not x86-only</strong>. Linux's Kconfig reads <code>depends on (X86 &amp;&amp; HYPERVISOR_GUEST) || ARM64</code>, added by VMware's Zack Rusin in <code>523375c943e5</code> (2021-05-05, <em>&ldquo;drm/vmwgfx: Port vmwgfx to arm64&hellip; ARM support is provided in svga version 3&rdquo;</em>), which split the hypervisor backdoor into <code>vmwgfx_msg_x86.h</code> and <code>vmwgfx_msg_arm64.h</code>.</p>
 <p>Broadcom's own <a href="https://knowledge.broadcom.com/external/article/315602">KB 315602</a> confirms vmwgfx is the display driver for Arm guests on Apple Silicon, with accelerated 3D on kernel 5.19+ and Mesa 22.1.1+. The device is <strong>SVGA3</strong> (<code>VMWGFX_PCI_ID_SVGA3 0x0406</code>), not SVGA2 (<code>0x0405</code>).</p>

nextbsd-fallback-graphics-plan.html

@@ -201,6 +201,9 @@ <h2 id="build">6. What we would build, and what we already have</h2>
 <p>That last row is a gap of our own making. When <code>drm_gem_framebuffer_helper.c</code> was vendored, five functions were excluded under <code>#ifndef __FreeBSD__</code> &mdash; three because drm-kmod's stub already provides them, and the dma-buf CPU-access pair because nothing needed them at the time. simpledrm's damage-blit path calls exactly that pair. Re-enabling them is small and known, but it is a real prerequisite rather than a detail to discover at link time.</p>
 
 <h4>Buffer backing &mdash; decide before writing</h4>
+<div class="callout callout-bad">
+<p><span class="pill pill-bad">Second strike against the VRAM helper</span> Beyond the Wayland double-buffering problem below, <code>drm_gem_vram_helper</code> places the BO in <strong>device memory</strong> &mdash; precisely the path blocked by <a href="https://github.com/nextbsd-redux/nextbsd-kernel/issues/71">nextbsd-kernel#71</a>. TTM <em>system-domain</em> GEM, the recommendation below, stays in system memory and takes the branch verified working on i915 hardware. This is now a correctness reason, not only a capacity one.</p>
+</div>
 <p>The scanout target is fixed iomem; a GEM allocator is only needed for client-side buffers, which are then blitted in. <strong>TTM system-domain GEM</strong> is the right choice. <code>drm_gem_vram_helper</code> is already built and would appear to work, but a VRAM pool over a firmware aperture is exactly one screen's worth of memory &mdash; sufficient for Xorg <code>modesetting</code>, and <strong>fatal for any Wayland compositor that wants to double-buffer</strong>. Choosing it would produce a driver that passes a smoke test and fails the actual use case.</p>
 
 <h2 id="licence">7. Write or port: the licensing fork</h2>

nextbsd-virtio-gpu-plan.html

@@ -249,6 +249,11 @@ <h2 id="layout">6. Repository layout and module graph</h2>
 </div>
 
 <h2 id="shmem">7. The one hard prerequisite: shmem GEM</h2>
+<div class="callout callout-good">
+<p><span class="pill pill-good">Unexpected advantage</span> <strong>shmem backing sidesteps <a href="https://github.com/nextbsd-redux/nextbsd-kernel/issues/71">nextbsd-kernel#71</a> entirely.</strong> That defect blocks any driver whose buffers live in device memory: once a VRAM BO is pinned for scanout, the fault path hands a device-BAR pfn to a function that assumes managed shmem pages, and the faulting thread never wakes. It currently blocks bochs and vboxvideo, and it will block vmwgfx.</p>
+<p>virtio-gpu's buffers are shmem &mdash; system memory &mdash; so they take the same branch i915 does, which is <em>verified working on real hardware</em>. The shmem helper was already the largest single cost in this plan; it turns out to also be what keeps virtio-gpu clear of the defect that stops every VRAM-backed driver.</p>
+<p><strong>Caveat:</strong> blob resources / host-visible memory <em>do</em> map device memory and would re-enter that territory. They are already out of scope (FreeBSD's 12-method VIRTIO_BUS interface has no <code>get_shm_region</code>), but if that bus method is ever added, this constraint returns with it.</p>
+</div>
 
 <p>virtio-gpu is built on shmem-backed GEM objects &mdash; <code>virtgpu_object.c:145</code> calls <code>drm_gem_shmem_get_pages_sgt()</code>, and that is where <em>all</em> guest-page DMA mapping happens. It is not optional or incidental, unlike in the fallback driver where a TTM substitution works.</p>