feat(state): add reversible virtio device state - #122
Open
appcypher wants to merge 19 commits into
Open
Conversation
Capture and restore exact virtio-mmio, queue, and block state at a complete paused boundary. Stop block workers before admitting new descriptors and return their queue after terminal completion and durability fencing. Add caller-resolved raw and qcow2 dependency chains that deny implicit path opening, plus validated backend replacement while a device remains quiesced. Preserve capacity, identity, feature, cache, and read-only compatibility across replacement.
Expose a config-only virtio device for publishing immutable generation identifiers and observing exact guest processing acknowledgements. Provide monotonic requests, idempotent retries, supersession outcomes, and an event-driven waiter without adding work to ordinary execution or I/O paths. Attach the device after existing virtio devices to preserve their established MMIO and interrupt allocation order, and surface the control operations through VmControl.
Expose deterministic device inventory and bounded generic transport state so callers can admit and capture reversible device boundaries. Stop and join console, network, and vsock workers before returning queue ownership. Preserve source-local network continuation state, make backend support explicit, and add clean boundaries for balloon, CPU, memory, metrics, RNG, and VM generation devices. Add deterministic codec and lifecycle coverage for malformed state, blocked console workers, network worker stop/resume, and vsock reactivation.
Stop the virtio-fs worker at a descriptor ownership boundary and return its queues and event handles to the transport. Prioritize stop over queue events so checkpoint pause work is bounded by a request already in progress instead of the guest backlog.\n\nCover quiesce and reactivation with a lifecycle test.
Complete a consumed RX descriptor with zero length before returning on EOF, and give stop requests priority over dequeuing new TX work. Include queue indices in transport-state capture errors so an invalid device boundary identifies the responsible queue.
Remember which console ports owned host workers before reversible quiescence and restart them directly when the device is reactivated. The guest sends PORT_OPEN only during initial negotiation, so waiting for it again leaves agent traffic stalled after resume.
Preserve userspace IOAPIC registers and pending lines alongside the KVM clock when x86 uses split irqchip mode. Keep the existing PIT, PIC, and in-kernel IOAPIC capture for the kernel-owned model. Reject execution-state restoration across mismatched interrupt-controller models and rebuild host routing from restored guest-visible registers.
Permit a bounded buffered device to replace its backend only when the replacement uses direct I/O. Preserve the negotiated guest contract while dropping raw-offset writeback accounting that cannot describe formatted storage.
Reserve a distinct muxer stop token instead of colliding with the Windows epoll control token. Reject the matching proxy port tuple and register the stop handle before activation succeeds so quiescence cannot wait on an unreachable worker.
Avoid the token reserved by the Windows epoll control notifier so the virtio-net worker can register and observe its stop event during reversible quiescence.
appcypher
changed the title
7 tasks
Restore captured device state after memory and execution reconstruction but before the first guest instruction. Allow restore coordinators to publish the constructed VM at an explicit paused boundary and wait for that boundary without polling.\n\nOrdinary boot remains unchanged unless the caller opts into the activation gate.
Keep the time-sync condition variable available on macOS without leaving an unused import on Linux. Apply the workspace formatter to the two virtio-fs files reported by CI so the checkpoint stack passes its formatting gate.
Compile the deterministic MMIO inventory only with block support, which is its sole caller. Document and permit the intentionally inline KVM checkpoint state rather than adding heap indirection to a rarely constructed ABI-shaped value.
Scope KVM checkpoint imports and mapped host addresses to the architectures and feature profiles that consume them. Keep the cross-architecture device inventory exhaustive without treating x86's single configured variant as a warning.
Compile the pause-generation implementation and its I/O dependency only for host-readable VM profiles, matching the public checkpoint API's existing TEE exclusion.
Capture and restore host-bound virtio device state through bounded, device-specific payloads, including console, virtio-fs, block feature contracts, VM generation state, and backend execution details.\n\nReconstruct destination-local filesystem providers and worker resources, preserve Linux interrupt and IA32_XSS state, and make vCPU teardown deterministic across KVM, HVF, and WHP.
Add a latched host-time sample to the generation device and require the kernel to acknowledge identity and clock processing together. Expose clock capability and immediate failure through VmControl. Rebase HVF virtual timer offsets with one VM-wide counter displacement and keep KVM from injecting offline time into its restored clock. Qualify delayed restore, timer semantics, and repeated activation on macOS and Linux with the matching guest driver.
5 tasks
Recognize the stable virtio-fs device ID in framing without importing the disabled filesystem backend in TEE builds. AMD-SEV and TDX Clippy checks pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Add deterministic reversible state boundaries for virtio transports, block devices, and the standard worker-backed devices used by the VM runtime.
Description
Test Plan
cargo fmt --all -- --checkcargo test --locked --offline -p msb_krun --lib(36 passed)cargo test --locked --offline -p msb_krun_devices --lib(87 passed)cargo test --locked --offline -p msb_krun_vmm --lib(40 passed)msb_krun,msb_krun_devices, andmsb_krun_vmm(169 passed)qemu-img info --backing-chainverification for generated macOS and Linux qcow2 chains