Baseline conversion rate (%)Your control group's expected conversion rateMinimum detectable effect (%)The smallest relative change you want to detectStatistical significance (%)Confidence level (risk tolerance for Type I error)Statistical power (%)Probability of detecting a true effect (Type II error tolerance)

Required sample size

Updates as you type

8,150

visitors per variation

16,300

total visitors (both variations)

Why you need this many visitors

Every test result is a sample, and samples are noisy. Move across the figure to watch the two possible results pull apart as visitors add up. It uses the numbers in the calculator above.

A: controlB: variantEach ball is one possible result of the same test. Hopper A drops at your baseline (5.00%), hopper B at the lift you want to detect (6.00%); the pegs add the noise a finite sample really has. With few visitors the piles overlap and a real lift is easy to miss. At 8,150 visitors per version they sit far enough apart that 80% of tests would find it at 95% significance.

Understanding Type I and Type II Errors

Type I Error (False Positive)

Occurs when you reject the null hypothesis when it's actually true. You detect an effect that isn't really there. Controlled by the significance level (α), typically set at 5%.

Type II Error (False Negative)

Occurs when you fail to reject the null hypothesis when it's actually false. You miss detecting a real effect. Controlled by statistical power (1 - β), typically set at 80%.

Common Questions

What is Baseline Conversion Rate?

The expected conversion rate of your control group without any changes.

What is Minimum Detectable Effect?

The smallest change in conversion rate that you want to detect, expressed as a percentage increase or decrease from the baseline.

Why is Statistical Significance important?

It helps you determine the likelihood that your results are due to the changes made and not by random chance.

What does Statistical Power tell us?

It indicates the probability that your test will detect a true effect when there is one.

More for a/b testing

All 18 free tools

Test durationDays until you can call the test A/B test significanceDid your variant really win Conversion rateYour rate against benchmarks

Stop calculating. Start testing.

You know the sample size, now launch the test. Humblytics lets you create and run A/B tests visually, without writing code or filing developer tickets.

Start free trial Book a demo

14-day trial · Card required to start · Cancel anytime