Skip to content

feat(amf): add native AMD AMF encoder#342

Open
RamazanKara wants to merge 55 commits into
Nonary:masterfrom
RamazanKara:feat/native-amd-amf-encoder
Open

feat(amf): add native AMD AMF encoder#342
RamazanKara wants to merge 55 commits into
Nonary:masterfrom
RamazanKara:feat/native-amd-amf-encoder

Conversation

@RamazanKara

@RamazanKara RamazanKara commented Jul 22, 2026

Copy link
Copy Markdown

Summary

Native AMD AMF encoder for Windows (D3D11, zero-copy), replacing the FFmpeg-based amdvce path. The PR contains AMD encoder code, AMD configuration/UI/docs, build integration, and AMD-focused tests, based on 1.18.1-stable.2 (6fe86ba4).

The native path supports H.264, HEVC, and AV1 with SDR/HDR where the selected driver mode supports it, dynamic bitrate, IDR and reference-frame invalidation, bounded queue/backpressure handling, and ordered startup/teardown. amdvce selects the native encoder; amdvce_legacy is the explicit rollback (exposed in the encoder dropdown and the docs Choices table). Automatic probing never silently falls back to the legacy path.

Design contracts

  • Verified property application. Every requested AMF property is set and read back; a rejected or post-Init-drifted value fails encoder creation instead of silently streaming with a different configuration. The client reference-frame limit and the H.264 intra-refresh two-reference floor are computed by one shared helper (effective_reference_frame_limit) used identically at configure time and post-Init verification, so the encoder never rejects its own raise.
  • FFmpeg-parity submit semantics. Only AMF_INPUT_FULL/AMF_DECODER_NO_FREE_SURFACES retry a submission. AMF_NEED_MORE_INPUT counts as consumed input with deferred output, which is what PreAnalysis lookahead priming returns while the pipeline fills.
  • Bounded everything. Submit retries are capped (20 attempts, 2 s wall clock), the input ring is capped at 32 plus transit, the surface pool grows lazily one texture at a time, and the teardown fence and the driver-destruction watchdog each get an independent 5 s budget. A genuinely wedged driver quarantines native AMF for the host's lifetime rather than hanging the process.
  • Wire continuity under load. A submission dropped by driver backpressure never consumes a frame index; the next capture reuses it, so the RTP frameIndex sequence stays contiguous and the client is never tricked into reference-invalidation round-trips for frames that never shipped.
  • Generation-tagged surfaces. Every input-surface reservation is stamped with a generation and each AMF wrapper carries the value it was created under; a driver-delayed release callback from an old wrapper cannot recycle a surface the VCN may still be reading.
  • Recoverable runtime fatals. A TDR, sustained submit backpressure, or an output stall rebuilds the same native session, bounded to three attempts without a stable run — never a silent legacy fallback, and fail-closed while quarantined.
  • Curated configuration surface. The UI and docs expose only choices a user can reason about: usage, rate control, QVBR level, quality preset, VBAQ, PreAnalysis, HRD, H.264 coder, and the AV1 content/latency modes. Deep driver knobs (input queue depth, LTR slots, Smart Access Video, the LOWLATENCY_MODE override, high-motion boost) are handled by the encoder's own policy and remain available only as undocumented expert config keys.
  • Cold-path work stays off the hot path. Bitstream extraction and input-pool texture allocation run outside the encoder state lock, the output pump waits interruptibly instead of sleeping in fixed quanta, and the encode loop performs one RTTI lookup per session rather than per frame, per-frame timestamp bookkeeping lives in a fixed ring instead of a heap-allocating map, and healthy session ends or runtime bitrate rebuilds detach their bounded teardown worker instead of stalling the encode thread.

Xbox intra-refresh

enableIntraRefresh == 1 follows the NVENC behavior with a 300-frame refresh cycle:

  • H.264 refreshes 16×16 macroblocks across 299 refresh frames.
  • HEVC refreshes 64×64 CTBs across 299 refresh frames.
  • AV1 uses continuous intra-refresh mode with a 300-frame cycle.
  • H.264 avoids conflicting LTR configuration; clients that send a single reference (or no limit at all) receive the required two-reference floor.
  • Requested properties are read back; driver rejection fails the native session rather than silently ignoring the Xbox request.

The mapping is covered by unit tests and exact-head CI. Desktop Moonlight does not send the Xbox request, so confirmation on actual Xbox hardware is still welcome.

Quality presets

Selection H.264 HEVC AV1
Speed 1 10 100
Balanced 0 5 70
Quality 2 0 30

Auto leaves the AMF quality property unset. Explicit presets are applied after usage and read back; a rejected or changed value fails the native session instead of silently selecting a different preset.

Also included

  • fix(nvhttp): PIN-stdin pairing (-0) built its certificate response and then fell through to a session lookup on a moved-from uniqueid, replacing the response with 400 Invalid uniqueid — stdin-driven pairing could never complete. Included here because the automated AMD latency validation pairs through exactly this path.

Validation

  • Exact-head Windows CI, native AMF lifecycle tests, packaging, and installer creation: workflow 30048792848.
  • Standalone AMF lifecycle suite: 25/25 checks (the same file runs under GTest on Arch/macOS CI). Frontend config-schema tests: 14/14.
  • Closed-loop hardware run at this head: RX 7900 XT, AMF runtime 1.5.2.0, Moonlight 6.1.0, AV1, 1920×1080 @ 120 FPS, 50 Mbps, deterministic full-screen workload, 50 s per profile, installed sunshine.exe SHA-256 a7d54ff43b30de76a51c9c86a9a71779e05c0d3ba75d6dd6ee2ec91c83fb7456. Every profile established a real client connection, delivered video, and completed with zero AMF warnings and zero fatals.
Profile Encode median / p95 / max Processing median Result
Queue 16 / Balanced 2.47 / 2.51 / 23.79 ms 2.68 ms pass
Queue 4 / Balanced 2.44 / 2.50 / 10.95 ms 2.64 ms pass
Queue 1 / Balanced 2.47 / 4.10 / 11.48 ms 2.70 ms pass
Queue 1 / Speed 2.28 / 2.34 / 11.95 ms 2.55 ms pass

The suite's recommendation rule (lowest encode p95, differences ≤0.05 ms treated as noise, q4 baseline preferred on ties) selects Queue 1 / Speed. Zero dropped submissions were recorded across every profile, and encode medians hold a 2.3–2.5 ms band run to run.

A tester build for this head is available: native-amf-polish-20260724 (unsigned; Windows may show a publisher warning). More AMD and Xbox reports are welcome — please include GPU, driver, codec, HDR state, selected AMD settings, reconnect/quit behavior, and whether the stream remained stable.

…b/components/app-edit/AppEditConfigOverridesSection.vue)
RamazanKara and others added 19 commits July 21, 2026 22:07
legacy_amf_session compared against &amdvce_legacy at preprocessor depth 0,
but the symbol is only declared under _WIN32 (video.h). The AMF integration
layer is otherwise portable, so define the flag as false elsewhere.

Signed-off-by: Ramazan Kara <github@r-kara.de>
…y applied

The post-Init readback compared MAX_NUM_REFRAMES against the raw client
value, but configure_reference_frames raises it to the intra-refresh
minimum (2) for H.264 — so every intra-refresh session from a numRefFrames=1
client failed encoder creation as a spurious 'driver changed the limit'.
Clients that send no limit (numRefFrames<=0) skipped the floor entirely,
leaving intra refresh on a driver-default single reference.

Share one effective_reference_frame_limit helper between the configure and
verify paths and cover it in the standalone suite.

Signed-off-by: Ramazan Kara <github@r-kara.de>
The refactor to a raw output_texture pointer only assigned it in
set_output_texture (dynamic/native-AMF path). NVENC and legacy sessions
left it null, so the RTX TrueHDR live readback (debug logging + P010)
dereferenced nullptr and crashed the host. Mirror the fixed texture into
the raw pointer and null-guard the readback.

Signed-off-by: Ramazan Kara <github@r-kara.de>
Per the AMF contract (and FFmpeg's amfenc.c, which retries only
AMF_INPUT_FULL), NEED_MORE_INPUT means the input was accepted and output
is deferred — PreAnalysis lookahead priming returns it while the pipeline
fills. Resubmitting the same surface duplicated frames inside AMF, and the
exhaustion path freed the ring slot while AMF still held references,
letting a stale OnSurfaceDataRelease recycle the slot's next reservation
while the VCN could still be reading it.

Signed-off-by: Ramazan Kara <github@r-kara.de>
The 5s deadline was computed before begin_teardown_until and reused as the
destruction watchdog, so gate contention was deducted from driver-teardown
time — contradicting the adjacent comment. Concurrent teardowns could
falsely quarantine a healthy runtime (permanent, host-restart messaging)
or leak a whole session. The fence wait and the destruction watchdog now
each get an independent 5s budget.

Signed-off-by: Ramazan Kara <github@r-kara.de>
A TDR, sustained submit backpressure, or a 2s output stall made encode_amf
signal reinit — but encode_thread turned that into a hard stream end while
every other encoder rebuilds its session in the same loop. Rebuild the same
native encoder (never a legacy fallback), bounded to 3 attempts without a
stable session, fail-closed while quarantined, and reset the bound once a
session runs cleanly.

Signed-off-by: Ramazan Kara <github@r-kara.de>
- Remove the unreachable 'raising input queue for PreAnalysis' log; the
  lifecycle policy deliberately preserves the user's queue choice.
- Remove the unused idr_event handle left from an abandoned plan.
- Remove the never-included d3d11_gpu_timer.h helper.
- Warn on unknown tri-state / amd_av1_latency_mode values instead of
  silently treating typos as auto, and warn on negative
  amd_qvbr_quality_level like the over-range case.
- Correct the config.h tri-state comment: 0 forces the property off, it is
  not a no-op.

Signed-off-by: Ramazan Kara <github@r-kara.de>
- Add amdvce_legacy to the web encoder dropdown and the docs encoder
  Choices table (it is the only rollback; auto never probes it), fixing
  the pre-existing rowspan miscount while there.
- Sync en_US/en_GB amd_quality_desc and amd_vbaq_desc with en: the config
  defaults are Balanced and VBAQ-enabled; Auto leaves properties unset.
- Make the auto-capable select test falsifiable (currentValue no longer
  injects the asserted option) and pin amd_rc as deliberately not
  auto-capable.

Signed-off-by: Ramazan Kara <github@r-kara.de>
… interruptible

- Extract encoded frames (multi-MB bitstream copy + driver property reads)
  before taking state_mutex; encode_frame takes the same mutex first thing
  on the submit path, so copying under the lock added per-packet jitter,
  worst on high-bitrate IDR frames. RFI flags resolve under the lock in the
  pump instead.
- Allocate input-pool expansion textures (~12 MB at 4K) outside the lock and
  commit the slot after relocking; expansion is encode-thread-only, so only
  observer-freed slots need re-checking.
- Replace the pump's fixed 1 ms sleeps with bounded CV waits so stop, drain,
  and new submissions wake it immediately instead of eating the quantum —
  on QUERY_TIMEOUT-less runtimes that sleep was the per-frame delivery poll.

Signed-off-by: Ramazan Kara <github@r-kara.de>
- Hoist the amf_encode_session_t dynamic_cast out of the encode loop; the
  session pointer is stable until the fail_guard teardown runs, so three
  RTTI lookups per frame collapse into one per session.
- Resolve Smart Access Video after concurrent-session suppression so a
  second session that suppressed Low Latency Mode keeps an explicit SAV
  opt-in instead of losing it to the stale pre-suppression value.

Signed-off-by: Ramazan Kara <github@r-kara.de>
The stdin path built the getservercert response and then fell through to
the session lookup with a moved-from uniqueID, replacing the response with
400 'Invalid uniqueid' — stdin-driven pairing (-0) could never complete.
Keep uniqID alive for the whole request and return after responding.

Signed-off-by: Ramazan Kara <github@r-kara.de>
Brings in the 1.18.1 release preparation plus the capture-above-refresh,
display stabilization fencing, and Playnite session-environment fixes.

Signed-off-by: Ramazan Kara <github@r-kara.de>
Remove amd_input_queue_size, amd_ltr_frames, amd_smart_access_video,
amd_lowlatency_mode, and amd_high_motion_quality_boost from the web UI,
per-app overrides, and user documentation. These are deep driver knobs a
user should never have to reason about: the encoder already picks safe
values, suppresses the unsafe combinations, and two of them carried
driver-freeze warnings. The config keys remain parsed as undocumented
expert escape hatches, so existing configs and automated tuning keep
working. The remaining native section exposes only the AV1 content and
latency choices.

Signed-off-by: Ramazan Kara <github@r-kara.de>
Brings the branch up to the 1.18.1-stable.2 release, including the
Vibepollo release-signing policy and approval-gate CI changes.

Signed-off-by: Ramazan Kara <github@r-kara.de>
Every reservation bumps a per-slot generation and each AMF wrapper is
stamped with the value it was created under. A driver-delayed
OnSurfaceDataRelease from an old wrapper can no longer free the slot's
next occupant while the VCN may still be reading it, and cannot corrupt
the synchronous-release accounting.

Signed-off-by: Ramazan Kara <github@r-kara.de>
- Wire continuity: a submission dropped by driver backpressure no longer
  consumes a frame index. The next capture reuses it, so the RTP
  frameIndex sequence stays contiguous and Moonlight stops issuing
  spurious reference-invalidation for frames that never shipped. Drops
  are counted in the periodic loop stats.
- Native amdvce now carries ASYNC_TEARDOWN: healthy session ends and
  runtime bitrate rebuilds detach the bounded destruction worker instead
  of stalling the encode thread 100-300 ms; the lifecycle gate still
  serializes it against the next initialization, and shutdown/reinit/
  fatal paths keep their ordered synchronous teardown.
- Per-frame timestamps move from std::map to a fixed 256-slot ring —
  no heap traffic on the submit path.
- The tight 5 s / 64-submission probe bounds now apply only to AMF
  probes; NVENC/QSV/software keep the pre-existing limits so this
  AMD-only PR cannot alter their negotiation on slow cold boots.
- Landing on the software encoder after every hardware probe failed is
  now loud: AMD users should never discover software encoding from
  stutter alone.

Signed-off-by: Ramazan Kara <github@r-kara.de>
100 ms around a 5 ms timeout flakes on loaded CI runners; 2 s still
proves control returned promptly instead of blocking on a wedged
destructor.

Signed-off-by: Ramazan Kara <github@r-kara.de>
@RunAway189

Copy link
Copy Markdown

I've quickly tested the latest build and the streaming got broken on AV1/HEVC/H264. The interesting thing is that Artemis told me my GPU wasn't capable of HDR when I started the stream. I only got a purple screen when the stream started. Sorry for not providing the log but I couldn't since I was in a hurry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants