Skip to content

test: automated contrast verification for all settings UI pages (light + dark) #327

Description

@deblasis

Problem

Two recent contrast regressions slipped past review and reached the windows branch:

  1. SearchResultsPage resolved theme-aware brushes via Application.Current.Resources[...] in code-behind. Application.RequestedTheme is pinned to Dark, so in light mode the secondary-text brushes (descriptions, breadcrumbs) came back with dark-mode values and were nearly invisible on the light background. Fixed by reading Resources.ThemeDictionaries against the page's ActualTheme (see #TBD).

  2. RawEditorPage used Color.FromArgb(0, 0, 0, 0) as a sentinel for "clear the highlight background" on ITextCharacterFormat.BackgroundColor. WinUI 3's RichEditBox ignores the alpha channel on that property, so it rendered as opaque black, painting black boxes over text after Ctrl+F cycles and after save+reload of the config (issue Light/dark mode awareness raw config black on black ctrl+f #325). Fixed by picking a theme-matched opaque color.

A trivial static-analysis rule could catch class 1 (don't call Application.Current.Resources for theme-keyed brushes from code-behind). It would not catch class 2, because the bug lives in a platform API quirk that only shows up in rendered pixels. We need rendered-pixel contrast verification to cover both classes and the next one we haven't thought of yet.

Goal

Catch contrast regressions on any Settings page regardless of root cause: brush-lookup mistakes, hardcoded colors, RichEdit API quirks, XAML typos, theme-dictionary key drift, etc.

Design outline

  • New test project Ghostty.Tests.Ui as a packaged WinUI 3 test app using the MSTest WinUI runner. The existing Ghostty.Tests is console xUnit and can't host RenderTargetBitmap.RenderAsync, which requires a real dispatcher and a real window.
  • ContrastFixture<TPage> helper:
    • Instantiates TPage in a transient Window.
    • Forces RequestedTheme = Light, waits for Loaded, renders via RenderTargetBitmap.RenderAsync.
    • Repeats for RequestedTheme = Dark.
    • Returns the bitmap plus a named-region map supplied by the page.
  • Each page exposes test regions via a partial-class file or #if TESTING method returning IEnumerable<(string name, Rect region)>. Example names: search-result-description, raw-editor-text, find-bar-input. Production code stays clean; the partial file is only compiled into the test project.
  • For each named region:
    • Sample pixels.
    • Compute effective foreground vs background. Simple first pass: darkest-quartile luminance vs lightest-quartile luminance in the region.
    • Assert WCAG contrast ratio >= 4.5:1 for body text. Threshold tunable per region (e.g. 3:1 for large text).
  • Rollout:
    • Phase 1: SearchResultsPage and RawEditorPage (known offenders).
    • Phase 2: one page at a time from windows/Ghostty/Settings/Pages/ so failures stay debuggable.
    • Phase 3: every remaining settings page has at least one contrast assertion.

Acceptance criteria

  • Ghostty.Tests.Ui project exists and builds on x64 in CI.
  • SearchResultsPage has contrast tests passing in both themes.
  • RawEditorPage has contrast tests passing in both themes.
  • Reverting the SearchResultsPage fix causes its contrast test to FAIL. Reverting the RawEditorPage fix causes its contrast test to FAIL. Both regressions are demonstrably caught by the harness.
  • At least one smoke-level contrast test runs per remaining page in windows/Ghostty/Settings/Pages/.
  • windows/Ghostty.Tests.Ui/README.md documents how to add a page, how to declare its regions, and how to run the suite locally.

Open questions and known risks

  • RenderTargetBitmap on WinUI 3 has documented limitations for SwapChainPanel and some MediaElement scenarios. If RichEditBox inside RawEditorPage doesn't cooperate, we may need a fallback (screen capture API, or a visual-tree walk that reads the computed CharacterFormat.BackgroundColor directly and asserts against it instead of pixels).
  • MSTest WinUI runner adds a second test framework alongside the existing xUnit project. Is that acceptable, or should we build a custom xUnit WinUI host to keep one framework?
  • GitHub Actions execution: running a packaged WinUI 3 test app requires a Windows runner with an interactive UI session. The current CI workflow may need a runner image change or runas wrapper to get a working desktop.
  • The "background pixel" heuristic (darkest/lightest quartile) is approximate. Pages with gradient backgrounds or overlapping translucent layers will need explicit named FG+BG region pairs so the assertion isn't comparing pixels from two unrelated visual layers.
  • Per-region threshold table needs a convention: small text 4.5:1, large text 3:1, UI chrome 3:1. Where does each region fall?

Non-goals

  • Keyboard navigation verification
  • Screen reader / UIA tree assertions
  • Animation frame sampling
  • Multi-monitor / per-monitor DPI matrix

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciNon-automated GitHub Actions changesenhancementNew feature or requestguiGUI or app issue regardless of platform (i.e. Swift, GTK)os/windows

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions