Skip to content
The Think Blog

Visual Regression Testing: The Best Tool (With the Worst Name)

Two-paneled illustration with small differences

Quick, I’ll give you a dollar if you can spot all the differences in the two images above! Better yet, I’ll give you a hundred dollars if you can spot all the differences in one minute. For every second it takes over one minute, I’ll deduct a dollar. And if you miss anything, that’ll cost your team two hundred bucks.

Does that pressure sound familiar?

“We want to make the text in the copyright statement a little bit bigger. It shouldn’t affect anything but the footer. Quick, make the change, double-check the site, then publish it!”

If you’re a “move fast and break things” kind of organization, you’ll make the change, trust that your CSS is well-structured, and have a quick look at the site. You might miss that the “mouse text” class got improperly re-used only in the hamburger menu at the tablet breakpoint, but don’t worry—your Executive Vice President will find that problem two weeks later when they discover that the navigation is broken on their iPad Pro. Whoops!

If you’re a “belt and suspenders” kind of organization, you’ll prepare the change, and then you’ll kick off a round of Visual QA. Your QA team has lots of automated tools at their disposal, but the tools they use only test to see if items are present or missing on a page, or that the items do the right thing when clicked. The only way to see if lines are wrapping awkwardly due to an unexpected text size change is to have a tester look at every page of the site. And every time they find something that might not be right, they’ll raise an “observation” to be discussed with design at the next standup. You deploy the text size change with no errors… a month from now.

There’s a much better way to catch visual changes

I’ve fed both versions of the image to a computer and instructed it to highlight all the differences in magenta. Did you catch all of the changes?

  • The missing desk lamp?
  • The addition of text on the closest blank page?
  • All the color changes, including the blouse that changed from beige to a light pink?

Did you catch the fact that the entire second image has been shifted one pixel to the left, which will be completely undetectable to a human… until the user shifts to a dark theme, and suddenly the image has an unexpected white hairline all along one side?

A visual regression testing tool catches all those changes in one step, and makes them easy to spot.

Here’s how we use this tool at Think Company

First, we use a tool called Backstop to take pictures of entire web pages, and then compare those pictures to a different set of pictures. Backstop provides a “cool scrubber thingy” to show you how those snapshots are different. Here’s YouTube user Garris Shipon demonstrating how you can “scrub” across a page to make changes crystal-clear:

Backstop can turn a week-long hunt for visual differences into a ten-minute “push the button” exercise. No magenta? No changes. No changes? No problem—SHIP THAT SITE.

Second, we make visual testing a part of our development workflow at the component level. Here’s the set of tools we’re using right now—Jest for writing tests, and Puppeteer for making screenshots—so that every pull request compares every component on every commit.

Using visual regression as a part of development means that “Whoops, I accidentally made a visual change seventy-five pages away, in a breakpoint that shouldn’t exist” is no longer something we worry about. Those (inevitable!) problems surface right away, while the developer is working, and they’re surfaced right in front of the person who can do something about it… instead of waiting two weeks for an observation to come from the QA team.

Want to learn more?

We’d be happy to share our expertise or discuss the workshops and clinics we offer on visual regression testing.

Start a conversation

We’d be happy to show you how we do it

Using visual regression both as a part of your QA process—and as a part of your continuous integration flow—reduces stress and errors, makes Pull Requests easier to review, and (dare we say it?) puts a lot of the fun back in hammering on the front-end. We offer workshops and hands-on clinics to show you how we do it, and to teach your team to customize these tools to match your flow. Let us know if you’re interested in learning more!

Stay in the know

Receive blog posts, useful tools, and company updates straight to your inbox.

We keep it brief, make it easy to unsubscribe, and never share your information.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Send us a postcard, drop us a line

Interested in working with us?

We scope projects and build teams to meet your organization's unique design and development needs. Tell us about your project today to start the conversation.

Learn More