Harden large CUDA host staging allocations

[voipmonitor]

Problem

InstantTensor buffered backends allocate one contiguous host staging buffer and register it with one cudaHostRegister call. At TP2 defaults this is commonly 2 GiB per rank. Some CUDA driver/kernel combinations reject both the single large registration and its segmented retries, so the loader aborts before reading weights even though runtime-allocated pinned memory is available.

Change

• Preserve the existing aligned allocation plus one-call registration fast path.
• If it fails, clear the pending CUDA/HIP runtime error and retry the same allocation in 256 MiB adjacent ranges.
• If registration itself remains unsupported, release the ordinary allocation and fall back to cudaHostAlloc or hipHostMalloc for the same staging size.
• Track whether storage is registered ordinary memory or runtime-pinned memory so destruction uses the matching unregister/free operation.
• Roll back all successful ranges if a segmented attempt fails.

No buffer size, io_depth, loading order, or tensor semantics change. Each fallback is entered only after the preceding path fails.

Validation

• Host-allocation helper coverage includes whole registration, segmented retry, rollback, runtime-pinned fallback success, and fallback failure: 1 passed.
• Existing InstantTensor CUDA copy tests against the earlier rebuilt wheel: 8 passed.
• 2 GiB CUDA probe: whole registration and eight adjacent 256 MiB registrations both completed and unregistered cleanly on the local host.
• Ruff check, Ruff format check, and git diff check passed.

The original failing NVIDIA 610.43.02 open-kernel host is not locally available. The new final fallback therefore still requires confirmation on that host; it is not claimed as field-validated yet.

[voipmonitor]

The wheel built from this head was validated inside the clean r21 inference image on an RTX PRO 6000 Blackwell Server Edition. A 2 GiB mapping registered successfully as one region (0.364 s), and the segmented path registered all 8 x 256 MiB ranges (0.045 s). An AIO_BUFFERED load with chunk_size=8 MiB and io_depth=256 loaded and verified all tensors. The original host where whole-region registration returns invalid-argument still needs confirmation that the segmented fallback succeeds there.

[voipmonitor]

The exact PR head 49b4010afc1cae0441e71fe0b0bffc24fa05e932 is included in the published r27 artifact:

voipmonitor/vllm:gilded-gnosis-v20-vllm966d57c-sibbbdccc-fi801d57a-cu132-20260803-r27
manifest: sha256:2605fda01797f33239af4c95ec7449505fe57d9b9de9687792f5b8273d3201a7

The image completed TP2 InstantTensor BUFFERED loading and the full DS4 runtime gate. The original open-driver host from rtx6kpro #52 rejects both whole-region and first-segment registration, so it has been asked to retest r27 specifically to exercise the new runtime-pinned allocation fallback. That external confirmation is still pending and is not being inferred from the validation host.

Reproduction and runbook:
https://github.com/local-inference-lab/rtx6kpro/blob/master/models/ds4dspark-v20.md

[arlo-scitix]

Thank you. Since this PR is fairly substantial, we’re considering how to refine and merge it gracefully while keeping the changes minimal. We also hope to stay in sync with you and continue moving the PR forward.