Skip to content

Add contributing guide for writing tests #813

Description

@Mystery-CLI

Background

The FuTuRe project uses a multi-layer test strategy covering unit tests with Vitest, end-to-end tests with Playwright, and mutation testing with Stryker. While the test infrastructure is in place and the CI pipeline runs all suites on every PR, the CONTRIBUTING.md file contains no guidance on how to write, run, or reason about these tests. This gap means that contributors — whether new team members or open-source participants — frequently write PRs that omit test coverage, misconfigure test environments, or write tests that pass locally but fail in CI.

Problem

Without a written guide, contributors encounter the following pain points repeatedly:

  • Discovery confusion. Contributors do not know which framework to use for which kind of test (unit vs integration vs e2e), leading to e.g. Playwright tests being written for logic that should be covered by Vitest.
  • Local run failures. Setting up the test environment for Playwright requires browser binaries and environment variables that are not documented anywhere, leading to "works in CI but not locally" or the reverse.
  • Low mutation scores ignored. Stryker mutation testing is run but contributors are not aware that a low mutation score is a quality signal, so coverage improves without test quality improving.
  • Boilerplate duplication. Each contributor invents their own test structure and helper patterns instead of following established conventions, making test suites inconsistent and harder to maintain.
  • Review bottlenecks. Reviewers spend significant time in code review commenting on test structure issues that could be caught by following a written guide.

Proposed Solution

Extend CONTRIBUTING.md with a dedicated Testing section. The section should cover:

  1. Test stack overview — a brief description of each framework (Vitest for unit/integration, Playwright for e2e, Stryker for mutation testing) and when to use each.
  2. Running tests locally — the exact commands to run each suite, including any prerequisite setup steps (environment variables, database seeding, browser installation).
  3. Writing unit tests — conventions for test file location, naming patterns, and using shared fixtures or factories.
  4. Writing e2e tests — how to run Playwright in headed vs headless mode, page object patterns used in this project, and how to handle authentication setup.
  5. Mutation testing — how to interpret Stryker output and what score thresholds to aim for.
  6. CI behaviour — which tests run on PRs vs main merges, and what to do when a CI suite fails.

Implementation Steps

  1. Review the current Vitest, Playwright, and Stryker configurations to document any non-obvious setup steps.
  2. Draft the Testing section in CONTRIBUTING.md using the outline above.
  3. Add a quick-reference command table near the top of the section for experienced contributors.
  4. Include at least one annotated example of a well-structured unit test and one Playwright test, referencing existing test files in the repository.
  5. Have the draft reviewed by at least two engineers who regularly write tests for this project.
  6. Update the onboarding checklist in CONTRIBUTING.md to include "run the test suite locally" as a first-day task.

Acceptance Criteria

  • CONTRIBUTING.md contains a Testing section covering all three frameworks.
  • A new contributor can install prerequisites and run all three test suites locally by following the guide alone.
  • The guide documents both the commands and the expected output for a clean run.
  • The Stryker section explains how to interpret the mutation score report.
  • At least one annotated example for unit and e2e tests is included.

Notes

The guide should link directly to relevant config files (vitest.config.js, playwright.config.ts, stryker.conf.js) so contributors can explore further without searching. Keep the guide concise — it is a starting point, not a full testing tutorial. Complex topics like test data management or flaky test debugging can be covered in separate deep-dive documents linked from this section.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions