Skip to content

kit: Extend UI testing with semantic queries and Linux rendering - #3402

Merged
huacnlee merged 4 commits into
mainfrom
testing-detailed-interactions
Oct 8, 2026
Merged

huacnlee merged 4 commits into
mainfrom
testing-detailed-interactions

Conversation

@huacnlee

@huacnlee huacnlee commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

UI tests can now discover observed controls by accessible label or role, inspect individual pointer/key transitions, and exercise text composition through an explicit input-handler bridge. Ordinary pointer helpers preserve held modifiers instead of resetting them.

The real offscreen pixel suite now runs on Linux WGPU as well as macOS Metal, with Linux rendering required in CI. A new nested context-menu regression exposed both ancestor and child menus opening on one right click; listener registration order and event consumption now ensure the innermost menu wins.

Adds regressions for Unicode composition, scoped and invisible observations, repeat keys, resize/display scale, cross-window clipboard and Link URL requests. English/Chinese guides and vendored testing references describe the new APIs and native-platform limits.

Existing styled controls now use these helpers in 13 additional interaction tests: Button release cancellation, Checkbox held-key activation, Switch disabling during a gesture, Slider live changes/release events, scoped RadioGroup controlled selection, List release modifiers/navigation/cancellation, and Textarea/Editor multiline composition undo and readonly transitions. Two additional light/dark real-renderer cases verify that a controlled Checkbox's pixels follow its value after pointer release. The input regression script includes composition-bridge workflows.

Public API

gpui-kit

All additions are under gpui_kit::test, gated by test-support.

  • pub trait TestQueryExt (implemented for Window) — native accessibility queries.
  • TestQueryExt::elements(&self) -> Vec<ElementSnapshot> — enumerate observed elements.
  • TestQueryExt::find_by_label(&self, label: &str) -> ElementSnapshot — require a unique exact accessible label.
  • TestQueryExt::try_find_by_label(&self, label: &str) -> Option<ElementSnapshot> — allow absence while diagnosing ambiguity.
  • TestQueryExt::find_all_by_label(&self, label: &str) -> Vec<ElementSnapshot> — enumerate exact label matches.
  • TestQueryExt::find_all_by_role(&self, role: Role) -> Vec<ElementSnapshot> — enumerate native role matches.
  • ScopedWindow::elements(&self) -> Vec<ElementSnapshot> — enumerate strict observed descendants.
  • ScopedWindow::find_by_label(&self, label: &str) -> ElementSnapshot — require a unique descendant label.
  • ScopedWindow::try_find_by_label(&self, label: &str) -> Option<ElementSnapshot> — optional unique descendant query.
  • ScopedWindow::find_all_by_label(&self, label: &str) -> Vec<ElementSnapshot> — enumerate descendant label matches.
  • ScopedWindow::find_all_by_role(&self, role: Role) -> Vec<ElementSnapshot> — enumerate descendant role matches.
  • pub trait TestEventExt (implemented for Window) — individual native input events with frame completion.
  • TestEventExt::pointer_move(&mut self, position: Point<Pixels>, pressed_button: Option<MouseButton>, cx: &mut App) — move with explicit held-button state.
  • TestEventExt::pointer_down(&mut self, position: Point<Pixels>, button: MouseButton, cx: &mut App) — press a pointer button.
  • TestEventExt::pointer_up(&mut self, position: Point<Pixels>, button: MouseButton, cx: &mut App) — release a pointer button.
  • TestEventExt::change_modifiers(&mut self, modifiers: Modifiers, cx: &mut App) — dispatch modifier changes while preserving caps lock.
  • TestEventExt::key_down(&mut self, key: &str, is_held: bool, cx: &mut App) — press/repeat a key without synthetic text.
  • TestEventExt::key_up(&mut self, key: &str, cx: &mut App) — independently release a key.
  • pub struct TestInput<'a> — owned explicit handler plus borrowed window for protocol tests.
  • TestInput::new(handler: impl InputHandler, window: &'a mut Window) -> Self — construct the handler bridge.
  • TestInput::commit_text(&mut self, text: &str, cx: &mut App) — commit one whole-text insertion.
  • TestInput::compose_text(&mut self, replacement: Option<Range<usize>>, text: &str, selection: Option<Range<usize>>, cx: &mut App) — update UTF-16 marked text.
  • TestInput::unmark_text(&mut self, cx: &mut App) — retain text while ending composition.
  • TestInput::marked_text_range(&mut self, cx: &mut App) -> Option<Range<usize>> — inspect the UTF-16 marked range.
  • TestInput::selected_text_range(&mut self, cx: &mut App) -> Option<UTF16Selection> — inspect selection and direction.
  • pub use events::TestEventExt, pub use input::TestInput, pub use query::TestQueryExt — expose the additions through the existing testing seam.
  • Existing TestWindowExt::{click, click_at, right_click, double_click, hover, scroll, drag, drag_to} and corresponding scoped pointer methods — retain current window modifiers; signatures unchanged.

gpui-component

  • ContextMenuExt::context_menu(self, build: impl Fn(PopupMenu, &mut Window, &mut Context<PopupMenu>) -> PopupMenu + 'static) -> ContextMenu<Self> — existing signature unchanged; nested right clicks now open only the innermost menu.

Breaking Changes

Existing pointer helpers now retain held modifiers. Callers that deliberately require an unmodified click while modifiers are held can request it explicitly; ordinary calls and signatures remain valid.

- window.click("save", cx); // previously reset held modifiers implicitly
+ window.click_with_options("save", ClickOptions::new(), cx); // explicitly unmodified, then restores held state

Nested context-menu callers keep the same construction. The corrected interaction is:

- window.right_click("inner-trigger", cx); // both inner and ancestor menus opened
+ window.right_click("inner-trigger", cx); // only the innermost menu opens

Validation

  • Targeted Kit interaction, observation, composition and environment tests on Linux.
  • Kit tests without default features for the portable helpers.
  • Clippy with warnings denied for the library and new test targets.
  • Changed Rust files formatted; git diff --check.
  • Website documentation checks: 9 passed.
  • Latest component workflow targets: 15 tests passed (including two previously existing composition cases).
  • Linux WGPU offscreen rendering: all 12 real pixel cases passed.
  • Kit menu regressions: 7 passed, including nested context-menu precedence.
  • Component context-menu regressions: 4 passed.

This verifies in-process test-platform behavior. Native OS IME/candidate windows, operating-system clipboard services and pixels require separate native evidence.

Implementation and documentation were AI-assisted.

@huacnlee huacnlee changed the title kit: Add semantic queries and detailed UI test interactions kit: Extend UI testing with semantic queries and Linux rendering Oct 7, 2026
@endless-bot endless-bot added the Breaking Changes Contains breaking changes; excluded from automatic cherry-picks to stable. label Oct 7, 2026
@huacnlee
huacnlee merged commit c8fdf33 into main Oct 8, 2026
16 checks passed
@huacnlee
huacnlee deleted the testing-detailed-interactions branch October 8, 2026 03:04
linruohan pushed a commit to linruohan/gpui-component that referenced this pull request Oct 9, 2026
…gbridge#3402)

UI tests can now discover observed controls by accessible label or role,
inspect individual pointer/key transitions, and exercise text
composition through an explicit input-handler bridge. Ordinary pointer
helpers preserve held modifiers instead of resetting them.

The real offscreen pixel suite now runs on Linux WGPU as well as macOS
Metal, with Linux rendering required in CI. A new nested context-menu
regression exposed both ancestor and child menus opening on one right
click; listener registration order and event consumption now ensure the
innermost menu wins.

Adds regressions for Unicode composition, scoped and invisible
observations, repeat keys, resize/display scale, cross-window clipboard
and Link URL requests. English/Chinese guides and vendored testing
references describe the new APIs and native-platform limits.

Existing styled controls now use these helpers in 13 additional
interaction tests: Button release cancellation, Checkbox held-key
activation, Switch disabling during a gesture, Slider live
changes/release events, scoped RadioGroup controlled selection, List
release modifiers/navigation/cancellation, and Textarea/Editor multiline
composition undo and readonly transitions. Two additional light/dark
real-renderer cases verify that a controlled Checkbox's pixels follow
its value after pointer release. The input regression script includes
composition-bridge workflows.

## Public API

### gpui-kit

All additions are under `gpui_kit::test`, gated by `test-support`.

- `pub trait TestQueryExt` (implemented for `Window`) — native
accessibility queries.
- `TestQueryExt::elements(&self) -> Vec<ElementSnapshot>` — enumerate
observed elements.
- `TestQueryExt::find_by_label(&self, label: &str) -> ElementSnapshot` —
require a unique exact accessible label.
- `TestQueryExt::try_find_by_label(&self, label: &str) ->
Option<ElementSnapshot>` — allow absence while diagnosing ambiguity.
- `TestQueryExt::find_all_by_label(&self, label: &str) ->
Vec<ElementSnapshot>` — enumerate exact label matches.
- `TestQueryExt::find_all_by_role(&self, role: Role) ->
Vec<ElementSnapshot>` — enumerate native role matches.
- `ScopedWindow::elements(&self) -> Vec<ElementSnapshot>` — enumerate
strict observed descendants.
- `ScopedWindow::find_by_label(&self, label: &str) -> ElementSnapshot` —
require a unique descendant label.
- `ScopedWindow::try_find_by_label(&self, label: &str) ->
Option<ElementSnapshot>` — optional unique descendant query.
- `ScopedWindow::find_all_by_label(&self, label: &str) ->
Vec<ElementSnapshot>` — enumerate descendant label matches.
- `ScopedWindow::find_all_by_role(&self, role: Role) ->
Vec<ElementSnapshot>` — enumerate descendant role matches.
- `pub trait TestEventExt` (implemented for `Window`) — individual
native input events with frame completion.
- `TestEventExt::pointer_move(&mut self, position: Point<Pixels>,
pressed_button: Option<MouseButton>, cx: &mut App)` — move with explicit
held-button state.
- `TestEventExt::pointer_down(&mut self, position: Point<Pixels>,
button: MouseButton, cx: &mut App)` — press a pointer button.
- `TestEventExt::pointer_up(&mut self, position: Point<Pixels>, button:
MouseButton, cx: &mut App)` — release a pointer button.
- `TestEventExt::change_modifiers(&mut self, modifiers: Modifiers, cx:
&mut App)` — dispatch modifier changes while preserving caps lock.
- `TestEventExt::key_down(&mut self, key: &str, is_held: bool, cx: &mut
App)` — press/repeat a key without synthetic text.
- `TestEventExt::key_up(&mut self, key: &str, cx: &mut App)` —
independently release a key.
- `pub struct TestInput<'a>` — owned explicit handler plus borrowed
window for protocol tests.
- `TestInput::new(handler: impl InputHandler, window: &'a mut Window) ->
Self` — construct the handler bridge.
- `TestInput::commit_text(&mut self, text: &str, cx: &mut App)` — commit
one whole-text insertion.
- `TestInput::compose_text(&mut self, replacement: Option<Range<usize>>,
text: &str, selection: Option<Range<usize>>, cx: &mut App)` — update
UTF-16 marked text.
- `TestInput::unmark_text(&mut self, cx: &mut App)` — retain text while
ending composition.
- `TestInput::marked_text_range(&mut self, cx: &mut App) ->
Option<Range<usize>>` — inspect the UTF-16 marked range.
- `TestInput::selected_text_range(&mut self, cx: &mut App) ->
Option<UTF16Selection>` — inspect selection and direction.
- `pub use events::TestEventExt`, `pub use input::TestInput`, `pub use
query::TestQueryExt` — expose the additions through the existing testing
seam.
- Existing `TestWindowExt::{click, click_at, right_click, double_click,
hover, scroll, drag, drag_to}` and corresponding scoped pointer methods
— retain current window modifiers; signatures unchanged.

### gpui-component

- `ContextMenuExt::context_menu(self, build: impl Fn(PopupMenu, &mut
Window, &mut Context<PopupMenu>) -> PopupMenu + 'static) ->
ContextMenu<Self>` — existing signature unchanged; nested right clicks
now open only the innermost menu.

## Breaking Changes

Existing pointer helpers now retain held modifiers. Callers that
deliberately require an unmodified click while modifiers are held can
request it explicitly; ordinary calls and signatures remain valid.

```diff
- window.click("save", cx); // previously reset held modifiers implicitly
+ window.click_with_options("save", ClickOptions::new(), cx); // explicitly unmodified, then restores held state
```

Nested context-menu callers keep the same construction. The corrected
interaction is:

```diff
- window.right_click("inner-trigger", cx); // both inner and ancestor menus opened
+ window.right_click("inner-trigger", cx); // only the innermost menu opens
```

## Validation

- Targeted Kit interaction, observation, composition and environment
tests on Linux.
- Kit tests without default features for the portable helpers.
- Clippy with warnings denied for the library and new test targets.
- Changed Rust files formatted; `git diff --check`.
- Website documentation checks: 9 passed.
- Latest component workflow targets: 15 tests passed (including two
previously existing composition cases).
- Linux WGPU offscreen rendering: all 12 real pixel cases passed.
- Kit menu regressions: 7 passed, including nested context-menu
precedence.
- Component context-menu regressions: 4 passed.

This verifies in-process test-platform behavior. Native OS IME/candidate
windows, operating-system clipboard services and pixels require separate
native evidence.

Implementation and documentation were AI-assisted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Breaking Changes Contains breaking changes; excluded from automatic cherry-picks to stable.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants