A/B Testing for Revenue per Visitor: Why Conversion Rate Can Pick the Wrong Winner

The short answer: As of October 8, 2026, revenue per visitor (RPV) is the total revenue a test variant produced divided by the visitors who saw it. It is the right primary metric when variants can change what people buy, not only whether they buy: pricing pages, plan layouts, bundles, discounts and upsells. Conversion rate can crown a variant that sells more of the cheaper option and makes less money. RPV tests need more data because revenue is noisier than a yes or no conversion, and they are only as good as the link between a payment and the variant that caused it. A thank-you page snippet is the common setup; processor-verified revenue matched to the variant is the stricter one.

This guide is for growth teams and CRO agencies that test pages close to the money: pricing pages, checkout flows, plan pickers and offer pages. If your test goal is a newsletter signup, conversion rate is fine. If the variants can change the price people pay, read on.

TL;DR

• RPV = revenue ÷ visitors. Equivalent to conversion rate × average order value.
• Conversion rate counts buyers, not dollars. A variant that sells more of a cheaper plan can win on conversions and lose on revenue.
• RPV is noisier. Expect to need more visitors than a conversion-rate test for the same relative lift, and protect against single huge orders.
• Where revenue comes from matters. A thank-you page value is easy and fragile. Revenue matched from Stripe to the variant survives refunds, renewals and buyers who never reach the thank-you page.
• Keep guardrails. Watch conversion rate and refund rate next to RPV, so a revenue win is not hiding a problem.

What revenue per visitor is

Revenue per visitor is a simple ratio:

RPV = total revenue from a variant ÷ visitors assigned to that variant

It divides by all visitors, not only buyers. That is what makes it useful for testing. It can be rewritten as:

RPV = conversion rate × average order value

So RPV moves when either part moves. A variant that converts the same share of people but sells them a bigger order raises RPV. A variant that converts more people at a much smaller order can lower it.

Two related numbers often sit next to RPV in test reports, and they answer different questions:

Why conversion rate can pick the wrong winner

Here is a hypothetical pricing page test. Two plans: Starter at $29 and Pro at $99. Each variant gets 10,000 visitors. Variant B redesigns the page and makes Starter the highlighted plan.

Variant B lifts conversion rate by 25.0%. Scored on conversions, it is a clear win. Scored on revenue per visitor, it is a 16.0% loss: it sold more plans and brought in $2,460 less on the same traffic.

This is not an edge case. Any change that can shift the mix of what people buy can open this gap:

• Plan and pricing layouts that steer people toward a cheaper tier, a monthly plan instead of annual, or a free plan.
• Discounts and coupons that convert more people at a lower price.
• Free shipping thresholds and bundles that change cart size.
• Simplified checkouts that drop an upsell or order bump.
• Trial offers that win more trial starts from people who never pay.

The reverse also happens. A variant that adds an annual plan toggle might convert slightly fewer people and still make more money. Scored on conversion rate, it loses and gets thrown away.

Why RPV tests need more data

A conversion is a yes or no for each visitor. Revenue per visitor is a dollar amount for each visitor: zero for most, and anything from a small order to a very large one for the rest. That spread is variance, and required sample size grows with variance. For the same relative lift, an RPV test usually needs more visitors than a conversion-rate test.

How many more depends entirely on how spread out your order values are. There is no universal multiplier. To show the shape of it, here is the standard normal-approximation sample size for detecting a 10% relative lift (two-sided, 95% confidence, 80% power), computed on hypothetical control data:

Same conversion rate, same buyer count. Adding a small number of large annual orders nearly tripled the visitors the RPV test needs, because a few big payments make the average jumpy. These figures are illustrations of one formula on made-up data, not a benchmark. Run your own order values through a calculator before you commit to a test.

Three practical consequences:

1. One large order can decide a test. GrowthBook's documentation gives the example of a normal revenue per user around $40 and a single $5,000 order making that variant much more likely to "win" (GrowthBook Docs, Metrics). Tools handle this differently: GrowthBook supports capping (winsorization), and Optimizely offers outlier smoothing for revenue metrics, which replaces values more than three standard deviations above the mean (Optimizely, How Optimizely Experimentation handles outliers).
2. Low-traffic sites feel this first. If a conversion-rate test already takes months on your traffic, an RPV test will take longer. Our guide to A/B testing on low traffic covers what to do instead.
3. Do not stop early on a revenue lead. A revenue gap swings more from day to day than a conversion gap. Decide the sample size up front. Statistical significance explained covers why peeking inflates false winners.

How to set up revenue as the test goal

Every revenue goal has to answer one question: which variant did this payment come from? There are two common answers.

Option 1: purchase value on the thank-you page

A script on the order confirmation page reads the order total and sends it to the testing tool, which already knows the visitor's variant from its own cookie or storage. This is how most testing tools document revenue tracking. Convert uses a pushRevenue call (Convert, Add revenue tracking to your site). Optimizely uses a trackRevenue event with the value in cents, usually on the purchase confirmation page (Optimizely, Configure revenue tracking). VWO, now Wingify, uses a revenue goal code snippet on the goal page, with the revenue value typically available on the thank-you page (Wingify, Types of conversion goals).

It is quick to set up. Its weak spots:

• Missed thank-you pages. A buyer who closes the tab, pays on a hosted checkout that never returns, or pays on another device is not counted.
• The value is what the page says. If the page shows a pre-tax, pre-coupon or wrong currency amount, that is what the test records.
• No refunds. A refunded order still counts unless you send a correction.
• No renewals. For subscriptions, only the first charge on that page is seen.

Option 2: processor-verified revenue matched to the variant

Here the payment record itself carries the link. The visitor's session or variant identifier is attached to the payment at checkout, and the testing tool reads revenue from the processor and matches it back. With Stripe, that means writing an identifier into metadata on the object you create. In Humblytics the identifier is humblytics_view_id: read it on the page with window.Humblytics.viewId, pass it to your backend, and add it to the PaymentIntent, Checkout Session, Subscription, Invoice or Customer metadata (Humblytics Docs, Stripe API). Stripe Payment Links can use the {CHECKOUT_SESSION_ID} on the return URL instead, without code, though that depends on the buyer finishing in the same browser (Humblytics Docs, Stripe Payment Links).

Subscriptions: revenue lands after the test

For SaaS, the visitor who sees the variant today may start a trial today and pay in fourteen days, then renew monthly. Two rules:

• Put the identifier on the Subscription or Customer, not only the first payment. Stripe copies Subscription metadata onto the invoices it creates, so renewals keep pointing back to the original visit. The full setup is in tracking Meta and Google Ads ROAS through to Stripe subscriptions; the same link that ties a payment to a campaign ties it to a test variant.
• Wait for revenue to settle before you call it. A variant that starts more trials will look strong on day one and may not be. Read trial starts as an early signal and revenue once those trials have had time to convert.

Refunds

A variant that oversells, overpromises or attracts the wrong buyer can win on gross revenue and lose it back in refunds. Hypothetical example:

Both variants had 10,000 visitors and an $80 order. B wins on gross RPV and loses on net. If your tool cannot net out refunds, pull refunds per variant from the processor before you ship the winner.

Guardrails to keep next to RPV

A primary metric picks the winner. Guardrails stop you shipping a winner that broke something else.

If RPV is up and both conversion rate and refund rate held, ship it. If RPV is up and a guardrail dropped hard, look before you ship.

Which A/B testing tools support revenue goals

Claims below are limited to what each vendor's own documentation says, checked October 8, 2026. Pricing, plans and limits are left out on purpose; check them on the vendor's site.

Two observations. First, most client-side tools default to the thank-you page method, which is fine for one-off ecommerce orders and weak for subscriptions. Second, GrowthBook's warehouse approach can use processor-verified revenue if your billing data is already in the warehouse, which suits teams with a data engineer. For a wider comparison, see the best A/B testing tools for SaaS.

What this approach does not solve

• It does not make small sites testable. Revenue variance raises the traffic bar. Below it, test bigger changes or use analytics and qualitative research instead.
• RPV is short-term revenue. It does not capture lifetime value, churn or expansion months later unless your revenue link follows renewals and you wait for them.
• Matching is not perfect. Processor matching still depends on the identifier reaching the payment. Cross-device journeys and payments outside your checkout flow can go unmatched.
• A revenue win is not always the right call. A variant that earns more per visitor by selling fewer, bigger plans may be the wrong trade for a product that grows through seats or referrals. That is a strategy decision, not a statistics one.

How Humblytics fits

Humblytics is built around this exact problem: a test result that resolves to money in the payment processor, not to a click. Choose Revenue as the test goal, connect Stripe with a restricted read-only key (or Foxy), and each variant reports purchases, total revenue and revenue per visitor from processor data matched to the visitor's session.

Be precise about what the numbers mean:

• The confidence figure is calculated on purchases per visitor, using a two-proportion test against a 95% confidence threshold. Revenue per visitor and total revenue are reported beside it from Stripe, and the result flags "check first" if a variant's revenue per visitor fell 5% or more. Read the revenue row before you ship.
• Revenue results wait to settle. Stripe trials, payments and revenue keep arriving for 14 days after the last visitor enters a test, and a revenue goal is not called final before then.
• Processors: Stripe and Foxy only. Shopify Payments is not supported. Refund handling is not documented, so check refunds per variant in Stripe.
• Pricing: Business is $79 a month and Scale is $279 a month. See pricing.

See how revenue goals work on the A/B testing page, or read how to calculate return on ad spend if the next question is which campaigns are sending the visitors.

Sources and freshness

• Convert Help Center, Add revenue tracking to your site, retrieved October 8, 2026. Supports pushRevenue, the force_multiple option, and the Revenue per Visitor report with Average Order Value.
• Optimizely Support, Configure revenue tracking, retrieved October 8, 2026. Supports the trackRevenue event in cents and revenue per visitor, revenue per paying visitor, purchases and total revenue on the results page.
• Optimizely, How Optimizely Experimentation handles outliers, retrieved October 8, 2026. Supports outlier smoothing for revenue metrics and the three-standard-deviation threshold.
• Wingify (VWO) Help, Types of conversion goals and Track Revenue on Goal in a Test is not working, retrieved October 8, 2026. Supports the Track Revenue goal, the revenue code snippet, and the thank-you page as the typical place the value is available.
• GrowthBook Docs, Metrics and Fact Tables, retrieved October 8, 2026. Supports SQL Fact Tables over the warehouse, Mean metrics for revenue per user, the $40 versus $5,000 outlier example, and capping.
• Humblytics Docs, Stripe API and Stripe Payment Links, retrieved October 8, 2026. Supports humblytics_view_id, the Stripe objects checked, and revenue as a split test goal.
• Stripe Docs, Metadata, retrieved October 8, 2026. Supports Subscription metadata being copied onto the invoices a subscription creates.
• Humblytics product behavior (two-proportion significance on purchases, 95% threshold, revenue per visitor row, 5% "check first" flag, 14-day settling window), checked against the app source on October 8, 2026.

All test figures in this article are hypothetical and computed from the stated inputs. Sample sizes use the standard normal approximation and are illustrations, not benchmarks. Vendor features change; check the linked documentation before you rely on a specific capability.