MasterTool

A/B Test Calculator

Calculate a/b test using your data.

Loading calculator…

What this tool does

Calculate a/b test using your data. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.

How to use the A/B Test Calculator

  1. Enter or select visitors — a (control).
  2. Enter or select conversions — a.
  3. Enter or select visitors — b (variant).
  4. Enter or select conversions — b.
  5. Enter or select baseline conversion rate for sample-size estimate (%).
  6. Enter or select minimum detectable effect (percentage points).
  7. Read the calculated result; change any measurement to compare alternatives.

Formula

two-proportion z-test on pooled SE; difference CI uses unpooled SE; required n per variant from the normal-approximation power formula at α=0.05 (two-sided), 80% power
va
Visitors — A (control)
ca
Conversions — A
vb
Visitors — B (variant)
cb
Conversions — B
base
Baseline conversion rate for sample-size estimate (%)
mde
Minimum detectable effect (percentage points)

Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.

Worked example

For a/b test calculator, the following measurements illustrate the exact method: Visitors — A (control): 1000; Conversions — A: 120; Visitors — B (variant): 1000; Conversions — B: 150; Baseline conversion rate for sample-size estimate (%): 10; Minimum detectable effect (percentage points): 2.

Inputs

  • Visitors — A (control)1000
  • Conversions — A120
  • Visitors — B (variant)1000
  • Conversions — B150
  • Baseline conversion rate for sample-size estimate (%)10
  • Minimum detectable effect (percentage points)2

Result

  • Conversion rate B (%)15
  • Conversion rate A (%)12
  • Absolute lift (percentage points)3
  • Relative lift (%)25
  • Z statistic1.96
  • Two-sided p-value0.05
  • Significant at 5%?Yes
  • 95% CI for the difference — lower (pp)0.01
  • 95% CI for the difference — upper (pp)5.99
  • Required visitors per variant (baseline + MDE, 80% power)3,841

Results explained

Conversion rate B (%)
Conversion rate B (%) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Conversion rate A (%)
Conversion rate A (%) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Absolute lift (percentage points)
Absolute lift (percentage points) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Relative lift (%)
Relative lift (%) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Z statistic
Z statistic from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Two-sided p-value
Two-sided p-value from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Significant at 5%?
Significant at 5%? from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
95% CI for the difference — lower (pp)
95% CI for the difference — lower (pp) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
95% CI for the difference — upper (pp)
95% CI for the difference — upper (pp) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.
Required visitors per variant (baseline + MDE, 80% power)
Required visitors per variant (baseline + MDE, 80% power) from the formula above. Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.

Frequently asked questions

two-proportion z-test on pooled SE; difference CI uses unpooled SE; required n per variant from the normal-approximation power formula at α=0.05 (two-sided), 80% power

Independent visitors randomly assigned, one conversion counted per visitor. Peeking early and stopping on significance inflates false positives; fix the sample size or horizon in advance.

Enter numbers only, in the units and format each label describes. Remove missing values rather than substituting zero, unless zero is a real observation.

No. Results describe the numbers you entered. Statistical inference also depends on sampling design, independence, model fit and interpretation; a p-value is not the probability that a hypothesis is true.

No. Every calculation, including the distribution algorithms, runs entirely in your browser.