-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
We need to give people more guidance. In the same way that we include a starter backstop.json, we can include a starter README.md and link to it from the root README.md
Example to append to the root README.md
## Visual Regression Testing (VRT)
Run VRT with `fire vrt:run`
For details see the BackStop [README.md](./tests/backstop/README.md)
Example README.md to add to tests/backstop
# What Is It?
BackstopJS uses a headless browser to create screenshots of webpages on different
environments (or at different moments in time) and then creates a diff of the
two images; the changed areas are highlighted.
# Pre-requisites
* Ddev
* Orbstack is recommended for _much_ faster test runs.
* FIRE
# Terminology
It's important to understand the Backstop terminology:
* **reference** — the environment that we're comparing against. For us this is typically the `live` environment, or maybe `test`.
* **test** — the environment where we've made changes, and we want to ensure that only the things that we expect appear different. For us this is typically a `pr-*` environment, or maybe `dev`.
# Run the tests
`fire vrt:run`
# Review the results
At the end the report will open in your browser.
If you see no differences, you're done.
If the differences are:
* Intentionally caused by your code changes (move on to the next step about updating the reference)
* Unintentionally caused by a bug in your code changes. Either:
* fix the code (You can run a single scenario with `--filter=<scenarioLabelRegex>`)
* agree with the team to live with it (then continue)
* Obviously because of a content difference between the two environments (then continue)
* Possibly because of a content difference, but you're not sure (consult with someone who knows the site better, or who knows VRT better)
# Known Issues
* False positives
* Any time content fades in, moves, or animates it will happen at slightly different speeds each time. If you find one like this, add some `postInteractionWait` to delay the screenshot until after this is complete.
* Sometimes if you are testing a local site (not a Pantheon environment) images will show as half loaded. This is
because Stage File Proxy is still downloading the image. Re-run the test
to see better results.
* Styleguide tests may fail because images/videos pull from a random image generator.
* Search results may fail due to minor sorting differences ¯\_(ツ)_/¯ .
* Lazy-loaded images are problematic. See https://github.com/garris/BackstopJS/pull/1601
* Things we don't currently test:
* Admin screens, or anything else logged in. See `get-logged-in-cookie.example.sh` in combination with `cookiePath`
* User interaction (e.g. clicking menus, submitting forms, scrolling),
To add more, see the [BackstopJS Advanced Scenarios](https://github.com/garris/BackstopJS/#advanced-scenarios).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels