Visual regression testing with Ladle, so far the best Storybook alternative

What is Ladle

According to the founder, Ladle is a tool for developing and testing your React components in an isolated and faster environment than most real-world applications. It supports Component Story Format – a concept widely popular thanks to Storybook. Ladle has been developed as a drop-in replacement for Storybook – it should already work with your existing stories.

Why replacing Storybook

Storybook is an amazing tool with a thriving community of engineers. Storybook's vast ecosystem and feature richness bring their challenges and create a need for lighter/faster replacement. The main pain points when using Storybook are the following, according to the creator of Ladle:

• production build time - often, it's the slowest part of the CI

• dev mode start-up time - not always faster than the related prod app

• bundle output - hosting storage + slow initial time to interactive

• maintenance - we repackaged Storybook, its dependencies, and configuration to provide a seamless setup for our developers; however, the addon versioning and monorepo setup make maintenance difficult

• testing - for our automated visual testing, we need to crawl and parse stories in a separate process since Storybook doesn't export a static list of stories (and other metadata)

The previous points are hard to disagree with as many engineers when asked about their experience using Storybook on a large-scale project, complain about its cumbersomeness and maintenance overhead.

Visual tests in Ladle

As a perfect component playground, Ladle is frequently used for development & documentation purposes. This means that your Ladle instance holds your application's most important building blocks at any given time, starting from design system primitives and ending with pattern components. This makes Ladle a perfect tool for visual regression testing.

There are numerous ways of running the visual tests, you can start simple and use low-level tools like playwright:

import { test, expect } from "@playwright/test";
// we can't create tests asynchronously, thus using the sync-fetch lib
import fetch from "sync-fetch";

// URL where Ladle is served
const url = "http://127.0.0.1:61000";

// fetch Ladle's meta file
// https://ladle.dev/docs/meta
const stories = fetch(`${url}/meta.json`).json().stories;

// iterate through stories
Object.keys(stories).forEach((storyKey) => {
  // create a test for each story
  test(`${storyKey} - compare snapshots`, async ({ page }) => {
    // skip stories with `meta.skip` set to true
    test.skip(stories[storyKey].meta.skip, "meta.skip is true");
    // navigate to the story
    await page.goto(`${url}/?story=${storyKey}&mode=preview`);
    // stories are code-splitted, wait for them to be loaded
    await page.waitForSelector("[data-storyloaded]");
    // take a screenshot and compare it with the baseline
    await expect(page).toHaveScreenshot(`${storyKey}.png`);
  });
});

This works well for simpler projects with little maintenance and fewer collaborators. Whenever the need comes to set up visual tests on larger projects, it pays off to set up a more robust cloud solution.

Compared to simple Playwright visual regression testing, cloud platforms offer numerous benefits:

• easier maintenance flow with lots of utilities to compare images

• direct CI integration with zero-effort setup

• faster collaboration flow that allows easier sharing and working together on failing tests

• no overhead of keeping all of the images stored in the repository(becomes a real problem with lots of collaborators & commits)

Lost Pixel Platform

When creating Lost Pixel some half a year ago, we decided to bet on Ladle. This decision seems great in retrospect, as we see a great need for a "lighter/faster/better to maintain" Storybook. Lost Pixel offers native ladle support, meaning you could start testing with a simple pointer to the running instance.

To run Lost Pixel Platform on your CI, you must first set up a GitHub installation.

After that, create lostpixel.config.ts:

lostpixel.config.ts

import { CustomProjectConfig } from 'lost-pixel';

export const config: CustomProjectConfig = {
  ladleShots: {
  // IP should be localhost when running locally & 172.17.0.1 when running in GitHub action(CI example)
    baseUrl: 'http://172.17.0.1:61000',
  },

  lostPixelProjectId: "xxxx",
  process.env.LOST_PIXEL_API_KEY,
};

And, of course, a CI run declaration - here, we are using GitHub actions as it is one of the simplest ways of running your CI/CD pipelines on GitHub

visual-tests.yml

on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
   steps:
     - name: Checkout
       uses: actions/checkout@v3

     - name: Setup Node
       uses: actions/setup-node@v3
       with:
         node-version: 18.x
         cache: "npm"

     - name: Install dependencies
       run: npm install

     - name: Build ladle
       run: npm run build

     - name: Serve ladle
       run: npm run serve &

     - name: Lost Pixel
       uses: lost-pixel/[email protected]

After pushing some new commits, you will see a new Lost Pixel Check on your GitHub repo & and also have a nice new build on Lost Pixel Platform:

Summary

We have gone through the main aspects of why Ladle is a great alternative for Storybook and also covered how to do effortless visual regression tests with Ladle in Playwright & Lost Pixel Platform.

If you need help, I am always happy to chat on Visual Regression Testing Discord!