Two or three variants
Write variant A, B and an optional C, and set the traffic split between them.
Feature · A/B testing
A/B test the first thing shoppers read.
Try different opening messages for your AI chat, split traffic between them, and compare observed add-to-cart and AI-attributed purchase outcomes. Review a recommendation before promotion, or opt into the documented two-arm auto-promote rule.
Plans from $49/month. Try Growth for 14 days.
Included in every plan.
What it does
The opening message sets the tone for every conversation. Fixed-split tests assign a version deterministically from the session ID and compare outcomes. Assignment persists while that identity persists; changing devices or browser storage can change it.
Add-to-cart drives the recommendation. AI-attributed purchases are shown alongside it. Repeated checks, optional stopping and comparisons against a third arm require care; the current badges do not establish a corrected sequential significance test.
Capabilities
What you get
Results that matter
Compare assigned shoppers, add-to-cart rate and purchases per variant, with the observed signal strength shown next to each result.
Recommended winner
The UI flags an add-to-cart leader when its absolute pooled z-score reaches 1.96; review purchases and guardrails before promoting.
Optional auto-promote
Two-arm tests only: at least 200 sessions per arm and |z| of 2.33 on add-to-cart, checked when the results page renders. These are operational thresholds, not proof of a future gain.
How to get started
Write your variants
Start from your current opening message and change one thing: the offer, the tone, or the question you ask.
Let it run
Choose the split and a review plan before starting. Keep catalog, offers and other interventions steady so you can interpret the comparison.
Promote and repeat
Promote the winner, keep the test history, and start the next idea.
Good to know
- Add-to-cart is the deciding metric; purchases are AI-attributed, not all store orders or incremental profit.
- Auto-promote applies to two-variant tests only; optional C uses equal three-way traffic and manual promotion.
- Small samples, repeated checks and multiple comparisons can produce misleading leaders. See the guide for current method limitations.
- Widget theme A/B tests are part of premium widget themes, included from Growth. Prompt variant tracking uses administrator-configured variants; it is not a merchant raw-prompt editor.
Frequently asked questions
What can I test?
The opening message shoppers see when the chat opens. Run variant A against B, or add a third variant C, and choose how traffic is split. From Growth you can also A/B test widget themes.
How is a winner chosen?
Zubby compares add-to-cart rates using a pooled z-score and displays a recommendation at an absolute score of 1.96. Purchases are a separate, AI-attributed guardrail. The displayed score bands are exploratory heuristics, not a guarantee of statistical certainty; see the method and repeated-check limitations in the guide.
Can Zubby promote the winner for me?
Yes, for two-variant tests with auto-promote enabled. On an A/B Tests page render, Zubby checks for at least 200 sessions per arm and an absolute add-to-cart z-score of 2.33. It then promotes A or B and ends the test. Three-arm tests require manual promotion.
How long should a test run?
Predefine a review horizon, useful effect size, primary outcome and safety guardrails. Required traffic depends on the baseline and effect; a week or a minimum sample alone does not prove a winner.
Keep exploring
Try it on your store
See Zubby answer from your own catalog.
Start the 14-day Growth trial, connect your WooCommerce store, and read real shopper conversations before you open the assistant to all traffic.