Skip to main content
You don’t have to guess whether 39or39 or 49 earns more, or whether a sharper headline sells better. An A/B test shows each visitor one version of your product, counts who buys, and tells you in plain words when one version is clearly ahead. Then you keep the winner in one click.
A/B tests are part of the Pro plan and above.

​
What you can test

Version A is always your product as it is today. Add up to three versions (B, C, D) to test against it.

​
Start a test

1

Choose what to test

Go to A/B tests → New test and pick price, headline, cover image or checkout layout.
2

Choose the product

Any paid product. Price tests need a fixed-price product without variants. One test runs per product at a time.
3

Add your versions

Enter the new price, headline, cover or layout for each version. Each must differ from A and from the others.
4

Pick the traffic split

Even by default (50/50, 34/33/33, 25 each). Custom splits give every version 5–95%.
5

Start the test

It starts at once. New visitors are shared between the versions.

​
How it works

  • One version per visitor, every time. Each browser gets a random visitor id in a first-party cookie on your store. The same visitor always sees the same version — on the product page, at checkout and when they come back — so nobody sees two prices.
  • The price is enforced at checkout. The server decides the visitor’s version from its own record, never from the page. The page also says which version it actually showed; if that doesn’t match (for example the page couldn’t load the visitor’s version and showed your product as it is), the buyer pays the price they saw and that sale is left out of the test. The tested price is the list price: early-bird steps, flash sales, fair pricing, discount codes and personal deadlines apply to it the same way they apply to the product’s own price. Receipts, orders and webhooks show what the buyer paid.
  • A paid sale is a conversion. Each sale is credited to the version the buyer saw. Refunded sales are left out.
  • Visitors are counted once. Bots, crawlers and link previews never see a test and are never counted.
  • Editing what you test stops the test. Change the price during a price test (or the name during a headline test, the cover during a cover test, the layout during a layout test) and the test stops on its own, so the results never mix two versions. Other edits are fine.
  • Your plan. If your plan no longer includes A/B tests, every visitor sees version A until you upgrade; the test resumes where it was.

​
Reading the results

Each version shows its visitors, sales, conversion rate (sales ÷ visitors) and revenue per visitor.
  • Chance to beat A. How likely the version really converts better than A, given the sales so far. At 95% we call it a winner.
  • Price tests are judged on revenue per visitor. A lower price can sell more often and still earn less. For price tests the column is Chance to earn more: how likely the version brings in more money per visitor.
  • vs A. How much higher or lower the conversion rate is than A’s.
  • Too early to call until every version has 100 visitors and the versions have 10 sales between them — small numbers swing a lot.
The statistics behind it: a two-proportion z-test against A (shown as a p-value in the API) and a Bayesian probability with a flat prior, computed exactly. The chart shows each version’s conversion rate over time; the lines settle as data comes in.

​
Keep the winner

Click Declare B the winner (or Pick as winner on any row). The winning version becomes your product for everyone: its price, its name and description, its cover, or its checkout layout. The test ends and ab_test.completed fires. Declaring A ends the test and keeps your product as it is. Stop test ends it without a winner; every visitor sees version A again. Ended tests keep their results until you delete them.

​
From the API

Start tests with POST /ab-tests, read live results with GET /ab-tests/{id}, and keep the winner with POST /ab-tests/{id}/winner. AI agents can use the create_ab_test, get_ab_test_results and declare_ab_test_winner MCP tools. See A/B tests API.