A/B Testing

Run controlled experiments on your storefront without code. Edit your live store by pointing and clicking, split traffic, and let Zolvio tell you—in plain language—which version wins, then roll it out with one click.

A/B tests list
Screenshot placeholder
1200×720px • /docs-images/ab-testing/tests-list-1200x720.png
Add a file at this path to replace the placeholder.
The tests list: status, visitors, conversions, and confidence at a glance.

What you can test

  • Visual change — edit elements on a page (text, colors, styles, images, show/hide). You can bundle several edits into one variant. This is the most common test type and uses the visual editor.
  • Split URL — send a share of visitors to a different page or template (for example, two landing pages) and compare them.
  • Offer test (early alpha) — test a percentage discount on a product. This is an early alpha and requires the Zolvio discount function; ask us to enable it for your store.

How it works

  • Zolvio assigns each visitor to a variant and keeps that assignment consistent (“sticky”).
  • An anti-flicker layer helps prevent a flash of the original content before the variant applies.
  • Conversions are attributed using exposure windows (add to cart, reach checkout, purchase, or a click you choose).
  • A single decision engine turns the raw numbers into a clear verdict—including how likely each variant is to be the best—so you don’t have to interpret p-values yourself.

Before you start

A/B testing changes what visitors see. Preview your changes first, and avoid running multiple tests on the same elements unless you understand interaction effects—Zolvio blocks overlapping running tests by default.

Creating a test

  1. Go to A/B Tests in the sidebar and click Create Test.
  2. Name the test and choose the page it runs on.
  3. Pick what you’re testing: Visual change, Split URL, or Offer.
  4. Build your variant(s). For visual changes, click Open visual editor to edit your live store by pointing and clicking (see Visual Editor). You can also add more than one variant (A/B/n).
  5. Choose your goal (add to cart, reach checkout, purchase, or a click) and, optionally, an audience.
  6. Review and start the test.

For enabling the theme extension and common Shopify gotchas, see Setup.

The visual editor

The visual editor opens your real storefront in a new tab with a Zolvio toolbar. Click any element to change it—no CSS required:

  • Edit text
  • Change background or text color
  • Swap an image
  • Hide or show an element
  • Stack several changes into one variant, and preview them live before saving

When you’re done, choose Send to Zolvio and your changes appear back in the test setup. Full walkthrough: Visual Editor.

Multiple variants (A/B/n) & traffic split

Test more than one idea at once by adding extra variants. Traffic is split evenly by default; you can set a custom split under Advanced. Zolvio compares every arm—including the control—and reports each one’s chance of being the best.

Reading results

The results view leads with a plain-language verdict and a per-variant scoreboard:

  • Conversion rate and lift vs. the control
  • Chance to win — the probability that a variant is the best of all arms (these add up to ~100% across variants)
  • Visitors and conversions per variant
  • A recommended next step, and a “what this test changes” summary

The verdict is honest about data quality: Zolvio won’t call a winner while a test is too new, under-sampled, has a skewed split (SRM), or a variant isn’t applying reliably—and the control is allowed to win. Deeper frequentist and Bayesian numbers live in the Advanced stats panel for when you want them.

When you have volume, use heatmaps and recordings by variant to explain why a variant won. See Heatmaps & Recordings, and Goals & Attribution for what counts as a conversion.

Publishing a winner

When you’re ready to act, you have three clear choices—see Publishing a Winner:

  • Roll out the winner to 100% of visitors through Zolvio (results freeze; delivery continues—no theme edits needed).
  • Make it permanent in your theme with a copy-paste snippet.
  • End the experiment to stop serving and finalize the results.

AI insights (optional)

Zolvio can generate a short, plain-English explanation of your test: the winner narrative, data-quality notes, key segments, and recommended next actions. This is optional and may be limited by your plan’s AI caps. See AI Usage & Controls.

Test limits

  • Free: No A/B testing
  • Starter: 1 concurrent test
  • Growth: 3 concurrent tests
  • Pro: 10 concurrent tests

More