diff --git a/CLAUDE.md b/CLAUDE.md index 9446fd9a..d833a1ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -108,6 +108,14 @@ The state module also contains protocol handler implementations (`*_handler.rs` Located in `src/screenshare/`. See [docs/developer/screenshare.md](./docs/developer/screenshare.md) for detailed architecture. +### Screen Capture (wlr-screencopy) + +`src/state/screencopy.rs` implements `zwlr_screencopy_manager_v1` for tools like `grim` and `wl-mirror`. Reads pixels from the Skia surface after render into SHM buffers. See [docs/developer/screencopy.md](./docs/developer/screencopy.md). + +### Virtual Pointer + +`src/state/virtual_pointer.rs` implements `wlr-virtual-pointer-unstable-v1` for automation tools (`wlrctl`, `wtype`). Together with the existing `virtual-keyboard` and `wlr-foreign-toplevel` protocols, enables full compositor remote control. See [docs/developer/virtual-pointer.md](./docs/developer/virtual-pointer.md). + ## Coordinate Systems & Naming Conventions Otto has two coordinate spaces — mixing them causes subtle scale-dependent bugs. @@ -157,6 +165,6 @@ Two tiers: - `docs/user/` — End-user configuration and usage guides - `docs/developer/` — Architecture, design docs, and implementation details -Key developer docs: `rendering.md`, `render_loop.md`, `wayland.md`, `screenshare.md`, `expose.md`, `dock-design.md`, `sc-layer-protocol-design.md`. +Key developer docs: `rendering.md`, `render_loop.md`, `rendering-optimizations.md`, `wayland.md`, `screenshare.md`, `screencopy.md`, `virtual-pointer.md`, `expose.md`, `dock-design.md`, `testing.md`, `sc-layer-protocol-design.md`. Review and documentation guidelines: `.github/instructions/review.instructions.md`, `.github/instructions/documentation.instructions.md`. diff --git a/docs/developer/rendering-optimizations.md b/docs/developer/rendering-optimizations.md new file mode 100644 index 00000000..46da36bc --- /dev/null +++ b/docs/developer/rendering-optimizations.md @@ -0,0 +1,62 @@ +# Rendering Optimizations + +This document covers Otto's rendering performance strategies beyond the core pipeline described in [rendering.md](./rendering.md) and [render_loop.md](./render_loop.md). + +## Per-window frame callback throttling + +**Source:** `src/state/window_throttle.rs` + +The single biggest lever for reducing GPU work is stopping frame callbacks to windows the user can't see. Well-behaved Wayland clients (Chromium, GTK4, Qt6) pause their internal render loops when frame callbacks stop arriving, which saves both compositor-side and client-side GPU work. + +### Window states + +Every mapped window is classified each frame into one of five states: + +| State | Rate | xdg activated | When | +|-------|------|---------------|------| +| **Focused** | Full refresh | yes | Top-of-stack on current workspace, fullscreen, or expose active | +| **Secondary** | ~30 Hz | no | On current workspace, visible, not focused | +| **Occluded** | ~2 Hz | no | On current workspace, fully covered by opaque content above | +| **Minimized** | ~2 Hz | no | Explicitly minimized by the user | +| **HiddenWorkspace** | ~2 Hz | no | Window's workspace is not active on any output | + +### Why 2 Hz instead of zero? + +Hidden states deliberately trickle callbacks at 2 Hz rather than stopping entirely. Chromium 115+ has an eviction heuristic that discards content buffers when callbacks stop for too long, causing a blank-canvas-on-restore bug. The 2 Hz rate satisfies the heuristic while saving essentially all the work. + +### Classification logic + +`classify_one()` is a pure function — no Wayland or lay-rs state, easy to unit test: + +``` +is_minimized? → Minimized +expose_active? → Focused (all windows get smooth previews) +is_fullscreen_window? → Focused +fullscreen_exists? → Occluded (behind the fullscreen) +is_top_of_stack? → Focused +in occluded_ids set? → Occluded +otherwise → Secondary +``` + +The `occluded_ids` set is computed by the lay-rs occlusion walk (when available). Currently empty — populated as a future refinement. + +### Integration with the render loop + +`classify_windows()` runs once per frame and produces a `HashMap`. The render loop passes each window's throttle duration to Smithay's `Window::send_frame()`, which skips the callback if insufficient time has elapsed. + +The `is_activated` flag is sent via `xdg_toplevel.configure`, signaling toolkits to self-throttle on top of the compositor's frame-callback throttling. + +## Damage tracking + +Otto uses Smithay's `OutputDamageTracker` to render only damaged regions. The render loop skips frame submission entirely when all of these are false: + +- `scene_has_damage` — lay-rs scene graph reports changes +- `dnd_needs_draw` — drag-and-drop icon is active +- `cursor_needs_draw` — pointer is in the output +- `has_screencopy` — a screencopy client is waiting for a frame + +This means an idle desktop with no animations and no cursor movement submits zero frames. + +## Screencopy render forcing + +When a screencopy client has a pending frame (`pending_screencopy_frames` is non-empty), `should_draw` is forced true regardless of damage state. This ensures capture tools always get a fresh frame. See [screencopy.md](./screencopy.md) for details. diff --git a/docs/developer/screencopy.md b/docs/developer/screencopy.md new file mode 100644 index 00000000..c2403546 --- /dev/null +++ b/docs/developer/screencopy.md @@ -0,0 +1,67 @@ +# wlr-screencopy-v1 + +Otto implements `zwlr_screencopy_manager_v1` (version 2), allowing screen capture tools to grab output frames via shared-memory buffers. + +## Usage + +```bash +# Full-screen screenshot (saves to file) +grim screenshot.png + +# Region capture +grim -g "100,100 800x600" region.png + +# Live mirroring +wl-mirror eDP-1 +``` + +Any client speaking wlr-screencopy-v1 works — `grim`, `wl-mirror`, `wlr-randr` (for output info), `wf-recorder`, etc. + +## Protocol flow + +``` +Client Otto + │ │ + ├─ capture_output(output) ─────►│ + │ ├─ create frame object + │◄── buffer(ARGB8888, w, h, s) ─┤ (advertise SHM format) + │◄── buffer_done ───────────────┤ (v3+) + │ │ + ├─ copy(wl_buffer) ────────────►│ + │ ├─ queue as PendingScreencopy + │ ├─ force render (even if no damage) + │ ├─ read_pixels from Skia surface + │◄── flags(empty) ──────────────┤ + │◄── ready(tv_sec, tv_nsec) ────┤ + │ │ + ├─ destroy ────────────────────►│ +``` + +`capture_output_region` follows the same flow but scales the requested logical region to physical pixels using the output's fractional scale. + +## Architecture + +### Key types (`src/state/screencopy.rs`) + +| Type | Role | +|------|------| +| `ScreencopyManagerState` | Holds the Wayland global; created in `Otto::init` | +| `ScreencopyFrameData` | Per-frame metadata: output, region, dimensions, state machine | +| `PendingScreencopy` | Queued frame + client buffer, waiting to be filled during render | + +### Render integration (`src/udev/render.rs`) + +1. When `pending_screencopy_frames` is non-empty, `should_draw` is forced true — the compositor renders even if the scene has no damage. +2. After `render_frame` completes and the output is scanned out, `complete_screencopy_for_output` is called. +3. It uses `skia_surface.read_pixels()` to copy the rendered frame into each pending SHM buffer, then sends `ready` or `failed`. + +### Buffer format + +Only `ARGB8888` (4 bytes/pixel) is advertised. Stride is `width * 4`. The Skia read uses `BGRA8888` color type which matches ARGB8888 on little-endian (the Wayland convention). + +## Limitations + +- **SHM only** — no DMA-BUF export. Every frame does a GPU-to-CPU readback via `read_pixels`. Fine for screenshots; a streaming client like `wl-mirror` will consume more bandwidth than a zero-copy DMA-BUF path would. +- **No damage reporting** — `copy_with_damage` is accepted but damage regions are not reported to the client. +- **Cursor overlay** — the `overlay_cursor` flag is stored but the cursor is always composited into the frame (same as overlay_cursor=1). +- **Udev backend only** — the `complete_screencopy_for_output` call site is in `src/udev/render.rs`. Winit/X11 backends don't call it yet. diff --git a/docs/developer/testing.md b/docs/developer/testing.md new file mode 100644 index 00000000..f342c224 --- /dev/null +++ b/docs/developer/testing.md @@ -0,0 +1,155 @@ +# Testing + +Otto has two test systems: headless integration tests for compositor behavior and WLCS for Wayland protocol conformance. + +## Headless integration tests + +Tests compositor behavior (gestures, expose, workspaces, window lifecycle, pointer interactions, scene graph) using a real compositor instance without GPU. + +```bash +cargo test --features headless --test headless_basic +``` + +### Architecture + +- **Backend:** `src/headless.rs` — `HeadlessHandle` starts the compositor on a background thread +- **Test client:** `components/otto-kit/src/testing.rs` — lightweight Wayland client with SHM buffers +- **Tests:** `tests/headless_basic.rs` +- **CI:** runs in the `check` job via `cargo test --features headless --test headless_basic` + +### HeadlessHandle API + +Start a compositor and interact with it: + +```rust +let handle = HeadlessHandle::start(HeadlessConfig::default()); +``` + +#### State access + +| Method | Returns | Description | +|--------|---------|-------------| +| `with_state(\|state\| { ... })` | — | Run a closure on the compositor thread (blocking) | +| `query(\|state\| value)` | `R` | Query a value from compositor state | +| `window_count()` | `usize` | Number of windows across all workspaces | +| `current_workspace_index()` | `usize` | Active workspace index | +| `is_expose_active()` | `bool` | Whether expose mode is open | +| `is_show_desktop_active()` | `bool` | Whether show-desktop mode is active | + +#### Gesture simulation + +| Method | Description | +|--------|-------------| +| `swipe_begin()` | Start a 3-finger swipe gesture | +| `swipe_update(dx, dy)` | Send swipe delta (pixels) | +| `swipe_end()` | End swipe gesture | +| `swipe(&[(dx, dy)])` | Complete swipe in one call | +| `pinch_begin()` | Start a 4-finger pinch | +| `pinch_update(scale)` | Send pinch scale | +| `pinch_end()` | End pinch gesture | + +#### Pointer simulation + +| Method | Description | +|--------|-------------| +| `pointer_move(x, y)` | Move pointer to absolute logical coordinates | +| `pointer_click()` | Left-button press + release at current position | + +**Note:** The first `pointer_move` into a new focus area triggers smithay's `enter` event (not `motion`). Selection updates only happen on `motion`. Establish focus with a priming move before testing hover behavior: + +```rust +// Prime focus on the window selector area +handle.pointer_move(5.0, 300.0); +handle.settle(2); +// Now this move triggers motion → selection update +handle.pointer_move(target_x, target_y); +handle.settle(10); +``` + +#### Expose queries + +| Method | Returns | Description | +|--------|---------|-------------| +| `expose_window_rects()` | `Vec<(title, x, y, w, h)>` | Expose layout rects in physical pixels | +| `expose_selected_title()` | `Option` | Currently hovered window title in expose | + +To convert expose rects (physical) to pointer coordinates (logical), divide by the output scale: + +```rust +let scale = handle.query(|state| { + state.workspaces.outputs().next() + .map(|o| o.current_scale().fractional_scale()) + .unwrap_or(1.0) +}); +let cx = (rect_x + rect_w / 2.0) as f64 / scale; +let cy = (rect_y + rect_h / 2.0) as f64 / scale; +``` + +#### Scene graph + +| Method | Returns | Description | +|--------|---------|-------------| +| `scene_snapshot()` | `SceneSnapshot` | Full scene graph snapshot | +| `scene_json()` | `String` | Scene graph as JSON | +| `scene_has_damage()` | `bool` | Whether scene has pending damage | +| `is_layer_hidden(key)` | `Option` | Hidden state of a named layer | + +#### Animation control + +| Method | Description | +|--------|-------------| +| `settle(max_frames)` | Advance scene at 60fps until animations finish or limit reached. Returns frames with damage. | +| `tick(dt)` | Advance one frame by `dt` seconds. Returns true if damage produced. | +| `wait(duration)` | Sleep the test thread (lets compositor event loop run). | + +`settle` is deterministic — no wall-clock sleeps. Use it after gestures/pointer events to let animations complete. + +### TestClient API + +Connect to the compositor and create windows: + +```rust +let mut client = TestClient::connect(&handle.socket_name)?; +let toplevel = client.create_toplevel("my-window", 640, 480); +handle.wait(Duration::from_millis(200)); +let _ = client.roundtrip(); +``` + +`create_toplevel` creates a toplevel surface with a SHM buffer, sets the title, and commits. The returned `Arc>` tracks configure events. + +### Writing tests + +Tests must use `#[serial]` (from `serial_test` crate) since they share a global compositor: + +```rust +#[test] +#[serial] +fn my_test() { + let handle = start_compositor(); + let mut client = connect_client(&handle); + // ... test logic ... +} +``` + +## WLCS protocol conformance + +Tests Wayland protocol compliance (surface roles, pointer/touch routing, xdg_shell). + +```bash +# One-time: build the WLCS test runner +./compile_wlcs.sh + +# Build the Otto WLCS adapter +cargo build -p wlcs_otto + +# Run specific test groups +./wlcs/wlcs target/debug/libwlcs_otto.so --gtest_filter='SelfTest*:FrameSubmission*' +./wlcs/wlcs target/debug/libwlcs_otto.so --gtest_filter='*/SurfacePointerMotionTest*' +./wlcs/wlcs target/debug/libwlcs_otto.so --gtest_filter='XdgToplevelStableTest.*' + +# List all tests +./wlcs/wlcs target/debug/libwlcs_otto.so --gtest_list_tests +``` + +- **Adapter:** `wlcs_otto/` — cdylib that WLCS loads, runs a headless Otto instance +- **Key files:** `wlcs_otto/src/main_loop.rs` (event handling), `wlcs_otto/src/ffi_wrappers.rs` (C FFI) diff --git a/docs/developer/virtual-pointer.md b/docs/developer/virtual-pointer.md new file mode 100644 index 00000000..b197ec1f --- /dev/null +++ b/docs/developer/virtual-pointer.md @@ -0,0 +1,75 @@ +# wlr-virtual-pointer-unstable-v1 + +Otto implements `zwlr_virtual_pointer_manager_v1`, allowing external tools to synthesize pointer events that feed into the compositor's input pipeline as if they came from a real pointing device. + +## Usage + +```bash +# Move pointer to absolute position and click +wlrctl pointer move 500 300 +wlrctl pointer click left + +# Type text (uses the existing virtual-keyboard protocol) +wlrctl keyboard type "hello" + +# Focus a specific window +wlrctl toplevel focus firefox +``` + +Compatible clients: `wlrctl`, `ydotool` (Wayland mode), `wtype`, and any custom automation driver. + +### The automation trio + +Otto exposes three protocols that together enable full remote control: + +| Protocol | Purpose | Implementation | +|----------|---------|----------------| +| `virtual-keyboard-unstable-v1` | Synthesize key events | Smithay delegate | +| `wlr-virtual-pointer-unstable-v1` | Synthesize pointer events | Hand-rolled (`src/state/virtual_pointer.rs`) | +| `wlr-foreign-toplevel-management-unstable-v1` | Enumerate/focus/close windows | Smithay delegate | + +## Supported events + +All events are accumulated per-frame and flushed on `frame`: + +| Request | Behavior | +|---------|----------| +| `motion` | Relative displacement added to current pointer position | +| `motion_absolute` | Normalized `[0, 1]` coordinates mapped to the first output's geometry | +| `button` | Left/right/middle button press/release (standard Linux button codes) | +| `axis` | Scroll amount on vertical/horizontal axis | +| `axis_source` | Sets the source (wheel, finger, etc.) on the pending `AxisFrame` | +| `axis_stop` | Stop notification for an axis (e.g. finger lifted from touchpad) | +| `axis_discrete` | Discrete scroll step (v120 high-resolution) | +| `frame` | Flushes all accumulated events to the pointer | + +## Architecture (`src/state/virtual_pointer.rs`) + +Smithay does not ship a virtual-pointer delegate, so the `GlobalDispatch`/`Dispatch` plumbing is hand-rolled. + +### Per-pointer state + +Each `ZwlrVirtualPointerV1` resource holds a `Mutex` with: +- `pending_motion` — accumulated `(dx, dy)` from `motion` requests +- `pending_absolute` — last `motion_absolute` position (overrides relative motion) +- `pending_buttons` — queued `(button_code, ButtonState)` pairs +- `axis_frame` — Smithay `AxisFrame` being built up across axis/axis_source/axis_stop/axis_discrete + +### Frame flush + +On `frame`, the accumulated state is committed in order: + +1. **Motion** — either absolute (mapped to output geometry) or relative (added to current location). Updates `pointer.motion()`, `layers_engine.pointer_move()`, and `surface_under()` focus. +2. **Buttons** — each queued button dispatched via `pointer.button()`. On press, `focus_window_under_cursor` is called so clicks behave like real ones (raise + focus). +3. **Axis** — the built-up `AxisFrame` sent via `pointer.axis()`. +4. **Pointer frame** — `pointer.frame()` finalizes the sequence. + +### Click-to-focus + +Real libinput clicks run `focus_window_under_cursor` + `layers_engine.pointer_button_down/up` from `on_pointer_button`. The virtual pointer mirrors this in its frame flush so that `motion_absolute` + `button` correctly focuses and raises the target window. + +## Limitations + +- **No pointer constraints** — locked/confined pointer regions are not honored for virtual events. +- **No relative-motion reporting** — the `wp_relative_pointer` extension is not notified for synthesized motion. +- **Single seat** — events are injected into the default seat. The `seat` parameter in `create_virtual_pointer` is accepted but ignored.