Sharing Experience in Building and Automating Visual Regression Tests with BackstopJS

The reason for choosing the free open-source BackstopJS instead of existing paid services (such as Percy and Chromatic) is its ability to overlay and compare images before and after changes simultaneously, along with its cost efficiency. This tool uses a headless Chrome browser and Puppeteer to capture rendered UIs, compare them with stored reference images, and visually highlight the changed areas.

Setup and Execution Process

After initializing with backstop init, scenarios are managed in backstop.json or a custom JS file (backstop.config.js). Using JS files is recommended to modularize complex configurations, and the --config option must be explicitly specified when running all commands. Before the first run, generate reference images with backstop reference, and subsequently verify changes with backstop test. If a test fails, update the new images as the new baseline using backstop approve.

Necessity of Docker Environment Setup

In local environments, tests may fail even without code changes due to differences in font rendering, among other factors. To resolve this, Headless Chromium is run within a Docker container to ensure a consistent environment. On Alpine Linux-based images, Chromium and related libraries (such as nss and freetype) must be installed manually, and the execution path for Puppeteer must be specified. Additionally, asyncCaptureLimit and asyncCompareLimit values can be adjusted to optimize test speed.

CI/CD Integration and Usage

Automation is achieved by adding Backstop test scripts to the Dockerfile that generates the Storybook preview server upon Git push. The HTML report of test results is configured to be accessible via the preview server URL (.../backstop_data/html_report/), allowing visual changes to be immediately verified during PR reviews. It is particularly effective for testing Organism-level components in Storybook, and can be flexibly adapted by configuring settings such as excluding dynamic elements (hideSelectors) or setting tolerance thresholds (mismatchThreshold).