Skip to content

Include a README.md as part of vrt:generate-backstop-config #79

@dalin-

Description

@dalin-

We need to give people more guidance. In the same way that we include a starter backstop.json, we can include a starter README.md and link to it from the root README.md

Example to append to the root README.md


## Visual Regression Testing (VRT)
Run VRT with `fire vrt:run`
For details see the BackStop [README.md](./tests/backstop/README.md)

Example README.md to add to tests/backstop

# What Is It?

BackstopJS uses a headless browser to create screenshots of webpages on different
environments (or at different moments in time) and then creates a diff of the
two images; the changed areas are highlighted.

# Pre-requisites

* Ddev
  * Orbstack is recommended for _much_ faster test runs.
* FIRE

# Terminology
It's important to understand the Backstop terminology:
* **reference** — the environment that we're comparing against.  For us this is typically the `live` environment, or maybe `test`.
* **test** — the environment where we've made changes, and we want to ensure that only the things that we expect appear different.  For us this is typically a `pr-*` environment, or maybe `dev`.

# Run the tests

`fire vrt:run`

# Review the results

At the end the report will open in your browser.

If you see no differences, you're done.

If the differences are:
* Intentionally caused by your code changes (move on to the next step about updating the reference)
* Unintentionally caused by a bug in your code changes. Either:
  * fix the code (You can run a single scenario with `--filter=<scenarioLabelRegex>`)
  * agree with the team to live with it (then continue)
* Obviously because of a content difference between the two environments (then continue)
* Possibly because of a content difference, but you're not sure (consult with someone who knows the site better, or who knows VRT better)

# Known Issues

* False positives
    * Any time content fades in, moves, or animates it will happen at slightly different speeds each time.  If you find one like this, add some `postInteractionWait` to delay the screenshot until after this is complete.
    * Sometimes if you are testing a local site (not a Pantheon environment) images will show as half loaded.  This is
      because Stage File Proxy is still downloading the image. Re-run the test
      to see better results.
    * Styleguide tests may fail because images/videos pull from a random image generator.
    * Search results may fail due to minor sorting differences ¯\_(ツ)_/¯ .
    * Lazy-loaded images are problematic.  See https://github.com/garris/BackstopJS/pull/1601
* Things we don't currently test:
    * Admin screens, or anything else logged in.  See `get-logged-in-cookie.example.sh` in combination with `cookiePath`
    * User interaction (e.g. clicking menus, submitting forms, scrolling),
      To add more, see the [BackstopJS Advanced Scenarios](https://github.com/garris/BackstopJS/#advanced-scenarios).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions