Skip to content

test: automated accessibility regression coverage (#56) - #65

Merged
spizeck merged 1 commit into
masterfrom
test/accessibility-regression-56
Sep 14, 2026
Merged

spizeck merged 1 commit into
masterfrom
test/accessibility-regression-56

Conversation

@spizeck

@spizeck spizeck commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Summary

Implements issue #56: an enforced accessibility regression barrier built on the Playwright fixture architecture from #63.

Baseline assessment: the site already had solid a11y foundations — skip link, labeled form fields with aria-invalid/aria-describedby, focus-trapping dialogs with focus restoration, aria-expanded/inert mobile menu, reduced-motion-aware carousel. The baseline axe inventory found two defect classes, both fixed here.

Coverage added (tests/e2e/accessibility.spec.ts)

  • Axe scans (full default ruleset, WCAG 2.x A/AA + best practices, zero rules disabled): /, /diving, /courses, /plan-your-trip, /contact, /book, /book?item=classic, /dive-sites, /about, /dive-log on desktop-chromium; representative mobile-viewport scans (/, /contact, /book) plus mobile-nav-open state on mobile-chromium.
  • Interactive states scanned: mobile nav open (after the open transition settles — mid-animation opacity is a scan artifact), contact form with validation errors displayed, dive-site dialog open, booking-unavailable fallback.
  • Keyboard: skip-link first tab stop + focus handoff into #main-content; desktop primary-nav traversal asserting focus destination names and a non-none focus indicator; mobile menu keyboard open/Escape-close plus proof closed-menu links are unreachable (inert); dialog focus trap (24 Tab/Shift+Tab cycles, none escape), Escape close, focus restore to trigger; contact form completed and submitted by keyboard alone.
  • Reduced motion: /about carousel must not autoplay under prefers-reduced-motion: reduce and must still advance without it — deterministic via Playwright's fake clock, no sleeps.

Browser/device strategy

Axe evaluates the rendered DOM and computed styles, so findings depend on viewport, not engine: full scans on desktop-chromium, mobile-layout scans on mobile-chromium, no mobile-webkit duplication. One exception handled explicitly: Playwright WebKit mirrors Safari's default where Tab visits only form controls, so the mobile-menu test asserts focusability rather than Tab destination there.

Defects found and fixed

Finding Severity Fix
--muted-foreground #6B7280 on #F5F6FA = 4.47:1 (site-wide, hundreds of elements) serious token → #5F6673 (~5.4:1)
text-muted-foreground/{50,60,70} on light bg — footer links 2.23:1 serious removed opacity modifiers; hover now → text-foreground
text-white/{40,45,50} on navy overlays — 4.42:1 and below serious raised to /60–/70
WhatsApp CTA green-600 text and bg-green-600+white ≈ 3:1 serious → green-700/800
h4 under h2 on /diving (6) and /dive-sites (per-region) moderate h3

Exceptions

None — no axe rules disabled, no selectors excluded. Policy documented in docs/TESTING.md.

Validation

  • npx playwright test tests/e2e/accessibility.spec.ts: 25 passed / 47 project-filtered skips
  • Full suite ×3: 140 passed / 58 skipped each run, no flakes
  • npm run test (unit+integration): 73 passed
  • npm run lint, npm run typecheck, npm run build:test: clean

Remaining gaps / follow-ups

  • Automated axe ≠ full WCAG conformance; manual audit still applies (per issue).
  • dive-sites-map.tsx/location-pin overlay text was raised conservatively; the map component is not mounted on /dive-sites today.
  • Text over hero photographs can't be fully evaluated by axe (incomplete, not violations) — worth a manual look.

Closes #56

Generated with Devin

Summary by Sourcery

Establish an automated accessibility regression barrier while correcting existing contrast and heading-structure issues.

New Features:

  • Add automated axe-core accessibility scans across key desktop and mobile pages and interactive states.
  • Add keyboard interaction coverage for skip links, navigation, mobile menus, dialogs, and contact form submission.
  • Add reduced-motion regression coverage for carousel autoplay behavior.

Bug Fixes:

  • Improve color contrast across muted text, overlays, footer links, WhatsApp actions, and other interface elements.
  • Correct heading hierarchy on diving and dive-site pages.

Enhancements:

  • Enforce accessibility regressions through the existing end-to-end test suite with no globally disabled axe rules.

Build:

  • Add @axe-core/playwright as a development dependency.

Documentation:

  • Document accessibility test coverage, browser strategy, exception policy, and remaining manual-audit requirements.

Tests:

  • Add Playwright accessibility regression tests and shared axe validation fixtures.

Adds @axe-core/playwright scans of every key public page plus the
interactive states where defects hide (mobile nav open, contact
validation errors, dive-site dialog, booking fallback), semantic
keyboard-operation tests (skip link, primary nav, mobile menu, dialog
focus trap/restore, contact form), and reduced-motion coverage for the
team carousel.

Fixes the violations the baseline scans surfaced:
- --muted-foreground darkened (#6B7280 -> #5F6673): measured 4.47:1 on
  the page background, just under WCAG AA 4.5:1
- text-muted-foreground/{50,60,70} and text-white/{40,45,50} opacity
  variants dropped real text below 4.5:1 (footer links were 2.23:1)
- WhatsApp CTA green-600 text/bg (~3:1) -> green-700/800
- skipped heading levels (h4 directly under h2) on /diving and
  /dive-sites corrected to h3

No axe rules disabled; exception policy documented in TESTING.md.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@vercel

vercel Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
seasaba-web Ready Ready Preview Sep 14, 2026 12:12am UTC

Request Review

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @spizeck, this account has used its review budget of 1,500,000 diff characters for the last 7 days.

You can request another review in 21 hours and 30 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

@sourcery-ai

sourcery-ai Bot commented Sep 14, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR establishes an enforced Playwright/axe accessibility regression gate across representative desktop, mobile, and interactive UI states, adds keyboard and reduced-motion behavioral coverage, documents the policy, and fixes the contrast and heading-hierarchy defects found by the new tests.

Sequence diagram for accessibility regression testing

sequenceDiagram
    participant CI
    participant Playwright
    participant Browser
    participant Axe
    participant Gate

    CI->>Playwright: Run accessibility.spec.ts
    Playwright->>Browser: Load page at target viewport
    Playwright->>Browser: Set interactive or preference state
    Playwright->>Axe: Analyze rendered DOM and computed styles
    Axe-->>Playwright: Return violations
    Playwright->>Browser: Exercise keyboard or reduced-motion behavior
    Browser-->>Playwright: Return focus and interaction results
    Playwright->>Gate: Report test results
    alt Accessibility regression found
        Gate-->>CI: Fail critical website tests
    else Checks pass
        Gate-->>CI: Pass accessibility gate
    end
Loading

Flow diagram for accessibility defect remediation

flowchart TD
    Scan[Run full axe ruleset] --> Finding{Violation found?}
    Finding -->|No| Pass[Keep accessibility gate passing]
    Finding -->|Yes| Diagnose[Identify contrast or heading defect]
    Diagnose --> Style[Adjust color tokens and text contrast]
    Diagnose --> Structure[Correct heading hierarchy]
    Style --> Rescan[Run accessibility tests again]
    Structure --> Rescan
    Rescan --> Finding
Loading

File-Level Changes

Change Details Files
Added an automated accessibility regression barrier covering page scans, interactive states, keyboard behavior, and reduced-motion behavior.
  • Added full axe scans for desktop pages and representative mobile layouts, including mobile navigation, form errors, dialog, and booking fallback states.
  • Added keyboard assertions for skip links, primary navigation, mobile menu inertness, dialog focus trapping/restoration, and keyboard-only form submission.
  • Added deterministic reduced-motion carousel tests using Playwright’s fake clock.
  • Added a shared axe helper that reports all violations and documented the scan policy, browser strategy, and exception guidance.
tests/e2e/accessibility.spec.ts
tests/e2e/fixtures.ts
docs/TESTING.md
package.json
package-lock.json
Fixed the accessibility defects identified by the baseline axe assessment.
  • Darkened the muted foreground color token to meet contrast requirements site-wide.
  • Removed low-opacity muted text treatments and increased white overlay text opacity.
  • Adjusted WhatsApp CTA colors to provide sufficient contrast.
  • Improved contrast for modal, footer, selector, dive-log, and map overlay text.
app/globals.css
app/page.tsx
components/contact-form.tsx
components/dive-log-client.tsx
components/dive-site-modal.tsx
components/experience-selector.tsx
components/footer.tsx
components/location-pin.tsx
app/(content)/diving/page.tsx
Corrected heading hierarchy in diving and dive-site content.
  • Changed subordinate section headings from h4 to h3 beneath h2 sections.
app/(content)/diving/page.tsx
app/(content)/dive-sites/page.tsx

Assessment against linked issues

Issue Objective Addressed Explanation
#56 Add enforced axe-based accessibility regression testing for representative critical public routes and interactive UI states, including desktop and mobile coverage, with serious violations failing CI.
#56 Add behavior-focused keyboard accessibility tests for navigation, dialogs, forms, and major interactive controls, including visible focus indicators, dialog focus trapping, Escape handling, and focus restoration.
#56 Review and remediate key accessibility concerns—including heading hierarchy, contrast, form/error states, reduced motion, and documented exceptions—without relying on brittle snapshots, while preserving manual accessibility review as a complement to automation.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@spizeck

spizeck commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

Visual check approved. The contrast improvements are very subtle, only the footer and WhatsApp button is noticeable and its an improvement.

@spizeck
spizeck merged commit 893fd9a into master Sep 14, 2026
9 checks passed
@spizeck
spizeck deleted the test/accessibility-regression-56 branch September 14, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add automated accessibility regression testing

1 participant