> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pocketsflow.com/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B tests

> Test two prices, headlines, covers or checkout layouts on real visitors and keep the winner.

You don't have to guess whether $39 or $49 earns more, or whether a sharper
headline sells better. An **A/B test** shows each visitor one version of your
product, counts who buys, and tells you in plain words when one version is
clearly ahead. Then you keep the winner in one click.

<Note>
  A/B tests are part of the **Pro** plan and above.
</Note>

## What you can test

| Test | What changes for the visitor |
| - | - |
| **Price** | The product's price on the product page and at checkout. |
| **Headline** | The product name, and optionally the description. |
| **Cover image** | The product's cover on the product page and at checkout. |
| **Checkout layout** | The checkout's layout (Split, Focus, Hero, …). |

Version **A** is always your product as it is today. Add up to three versions
(B, C, D) to test against it.

## Start a test

<Steps>
  <Step title="Choose what to test">
    Go to **A/B tests → New test** and pick price, headline, cover image or
    checkout layout.
  </Step>

  <Step title="Choose the product">
    Any paid product. Price tests need a fixed-price product without variants.
    One test runs per product at a time.
  </Step>

  <Step title="Add your versions">
    Enter the new price, headline, cover or layout for each version. Each must
    differ from A and from the others.
  </Step>

  <Step title="Pick the traffic split">
    Even by default (50/50, 34/33/33, 25 each). Custom splits give every
    version 5–95%.
  </Step>

  <Step title="Start the test">
    It starts at once. New visitors are shared between the versions.
  </Step>
</Steps>

## How it works

* **One version per visitor, every time.** Each browser gets a random visitor
  id in a first-party cookie on your store. The same visitor always sees the
  same version — on the product page, at checkout and when they come back —
  so nobody sees two prices.
* **The price is enforced at checkout.** The server decides the visitor's
  version from its own record, never from the page. The page also says which
  version it actually showed; if that doesn't match (for example the page
  couldn't load the visitor's version and showed your product as it is), the
  buyer pays the price they saw and that sale is left out of the test. The tested price is the
  list price: early-bird steps, flash sales, fair pricing, discount codes and
  personal deadlines apply to it the same way they apply to the product's
  own price. Receipts, orders and
  webhooks show what the buyer paid.
* **A paid sale is a conversion.** Each sale is credited to the version the
  buyer saw. Refunded sales are left out.
* **Visitors are counted once.** Bots, crawlers and link previews never see a
  test and are never counted.
* **Editing what you test stops the test.** Change the price during a price
  test (or the name during a headline test, the cover during a cover test,
  the layout during a layout test) and the test stops on its own, so the
  results never mix two versions. Other edits are fine.
* **Your plan.** If your plan no longer includes A/B tests, every visitor sees
  version A until you upgrade; the test resumes where it was.

## Reading the results

Each version shows its **visitors**, **sales**, **conversion rate** (sales ÷
visitors) and **revenue per visitor**.

* **Chance to beat A.** How likely the version really converts better than A,
  given the sales so far. At 95% we call it a winner.
* **Price tests are judged on revenue per visitor.** A lower price can sell
  more often and still earn less. For price tests the column is **Chance to
  earn more**: how likely the version brings in more money per visitor.
* **vs A.** How much higher or lower the conversion rate is than A's.
* **Too early to call** until every version has 100 visitors and the versions
  have 10 sales between them — small numbers swing a lot.

The statistics behind it: a two-proportion z-test against A (shown as a
p-value in the API) and a Bayesian probability with a flat prior, computed
exactly. The chart shows each version's conversion rate over time; the lines
settle as data comes in.

## Keep the winner

Click **Declare B the winner** (or **Pick as winner** on any row). The
winning version becomes your product for everyone: its price, its name and
description, its cover, or its checkout layout. The test ends and
`ab_test.completed` fires. Declaring A ends the test and keeps your product
as it is.

**Stop test** ends it without a winner; every visitor sees version A again.
Ended tests keep their results until you delete them.

## From the API

Start tests with `POST /ab-tests`, read live results with
`GET /ab-tests/{id}`, and keep the winner with `POST /ab-tests/{id}/winner`.
AI agents can use the `create_ab_test`, `get_ab_test_results` and
`declare_ab_test_winner` MCP tools. See [A/B tests API](/api-reference/ab-tests).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.