A/B Testing: Goals & Attribution

How Zolvio decides which variant “gets credit” for a conversion.

Core Concepts

  • Exposure: the visitor was assigned a variant for a test
  • Conversion: the visitor completes a goal (click, add to cart, checkout, purchase)
  • Attribution window: how long after exposure a conversion can be credited
How will you measure success: Goal and difference to spot
Step 3 of Create A/B test: the goal, and how big a difference the test should be able to spot.

What Decides the Winner

Tests are decided on the conversion rate: the share of visitors who complete the goal. On a purchase goal you can choose Revenue per visitor under What decides the winner instead, for changes that could alter what people buy or how much (bundles, upsells, prices shown). It needs at least 20 orders on each version, so it takes longer.

Revenue is the order’s merchandise subtotal after discounts, before shipping and tax, in your store’s currency. Test orders (Shopify’s Bogus Gateway or a test-mode payment) aren’t counted.

Attribution Windows

Typical windows:

  • Click: short window (minutes to hours)
  • Add to cart: short-to-medium window
  • Checkout: up to a couple of days
  • Purchase: up to a couple of weeks

Why windows matter

If you sell products with longer consideration cycles, prefer purchase goals and run tests longer. For smaller stores, add-to-cart and click goals can be useful leading indicators.

What “Confidence” Means

Confidence is a statistical estimate that the observed lift is not due to randomness. Avoid stopping tests early — it increases false positives.

Next

To target experiments (device type, UTM params, etc.), see Targeting.