# AB Test Calculator

Group 1

Group 2

## How to Use AB Test Calculator?

Follow the below steps to calculate the most significant sample space
• Select the Confidence interval from the drop-down menu
• Enter the sample size
• Enter the number of possible results
• Enter the conversion rate
• Hit the “Calculate” button to check whether it is statistically significant or not.
• Click on the “Show Steps” button to see the step-by-step solution
• Click on the “reset” button to erase all inputs.

## AB Test Calculator

The AB test Calculator calculates the statistical significance of two sample spaces to find out whether the samples are statistically significant or not.

## What is AB Test?

Statistical significance test in which two populations are related to checking if they differ significantly on a characteristic. it is basically a kind of Z-test. The objective of the AB test is to show that two proportions are not the same.

## What is a Z-score?

A Z-score expresses a value's relationship to a group of values from the same distribution. The Z-score is calculated in the number of standard deviations from the group's mean. A higher Z-score means the value is more away from the distribution's mean.

## Method of calculation:

1. Determine their sample sizes (n1 and n2)
2. Determine the number of positive results in each group (t1 and t2)
3. Calculate the population's proportions using the formulas

p1 = t1 / n1 and p2 = t2 / n2

1. Calculate the overall sample proportion

p = t1 + t2 / n1 + n2

1. Calculate the Z-score 1. Compare the absolute values of the results. If the absolute value of the Z-score is greater than or equal to the absolute value of the alpha level’s Z-score then the samples are statistically significant.

## Example section:

In this section, we have given the step-by-step calculation of the AB test.

Example 1:

Compare the following groups and find out whether these are statistically significant or not,

if the confidence interval is 90%

Group 1

Sample size = 40

No. of positive results = 2

Conversion rate = 5%

Group 2

Sample size = 12

No. of positive results = 5

Conversion rate = 41.7%

Solution:

Step 1: Extract the data

Sample size = n1 = 40

No. of positive results = t1 = 2

The conversion rate in percentage (%) = 5

Group 2

Sample size = n2 =12

No. of positive results = t2 = 5

The conversion rate in percentage (%) = 41.7

Step 2: Apply the formula

p1 = t1 / n1

p1 = 2 / 40

p1 = 0.05

p2 = t2 / n2

p2 = 5 / 12

p2 = 0.417

Step 3: Calculate “p”

p = t1+t2 / n1+n2

p= 2+5 / 40 + 12

p = 0.135

Step 4: Calculate the Z-score

Z-score = p1 - p2 / √{p * (1 - p) * (1/n1 + 1/n2)}

Z-score = 0.05 - 0.417 / √{0.135 * (1 – 0.135) * (1 / 40 + 1 / 12)}

Z-score = -3.263

Alpha level’s z-score = 1.645

Step 5: Comparison

For the comparison, compare the absolute value of both the Z-score and Alpha level’s Z-score.

The absolute value of Z-score = 3.263

Absolute value Alpha level’s z-score = 1.645

As the absolute value of “Z-score” is greater than the absolute value of “Alpha level’s z-score” the samples are “statistically significant”.

## References:

A/B testing statistics: An intuitive guide for non-mathematicians | Conversion Sciences.

A/B testing - A complete guide to statistical testing. | Towards Data Science.