A/B Test Sample Size Calculator
Want to know how long to leave an A/B test running? Stop guessing, use our sample size calculator to find out!
Test settings
Values
Explanation
Weeks of data in import
Used to estimate how much time you should leave the exp going
Confidence level / Statistical Significance
Protection against Type I error (False Positive) Go for 95% minimum
Statistical Power (1-β)
Protection against Type II error (False Negative) Go for 80% minimum
Minimum Detectable Effect
Relative change in %. Don’t go below 10% or your sample size will balloon up
% visitors from User Base included in the test
Default: 100%
Users *
Conversions *
Required Sample Size per Variant 
Not calculated
Weeks to run
Not calculated
You now know that you will have to leave this A/B test running for 111 weeks or until you have 4612 users on Control and Variations.
How to use this tool
We use this tool to help our clients decide which experiment ideas will deliver the quickest and most impactful results.
Create a segment using your analytics platform. We recommend checking the data for 1 month or 3 months.
Write down the number of sessions/unique visitors in the “User” field.
Write down the number of conversions in the ”Conversions” field.
In the “Measured period” field write down the period of time selected in step 1 in weeks
Minimum detectable effect.
You're ready to go!
EXAMPLE
Let’s say you want to change the hero banner on the homepage and because you’re data driven you know how important A/B testing is before making chances.
- The first thing you need to do is go in your analytics platform and create a segment for the users that have visited the homepage in the last month. Let’s say we have 1000 users and 200 clicks on the hero banner.
- Our goal is to increase the number of clicks by 20%. The fields for the tool will look like this:
Test settings
Values
Explanation
Weeks of data in import
Used to estimate how much time you should leave the exp going
Confidence level / Statistical Significance
Protection against Type I error (False Positive) Go for 95% minimum
Statistical Power (1-β)
Protection against Type II error (False Negative) Go for 80% minimum
Minimum Detectable Effect
Relative change in %. Don’t go below 10% or your sample size will balloon up
% visitors from User Base included in the test
Default: 100%
Users *
Conversions *
Required Sample Size per Variant
Not calculated
Weeks to run
Not calculated
You now know that you will have to leave this A/B test running for 111 weeks or until you have 4612 users on Control and Variations.
Convert more visitors with OmniConvert Explore
Check our customer case studies and see how we helped companies like yours
A/B Sample Size Calculator
Quickly and conveniently estimate your A/B test sample size.
No mathematics required.
Manual calculations of your sample size can be complex and prone to human error. They also tend to require advanced math skills or a statistical mind.
And yet, you still need to know this number – as the number of participants for each variation is tightly linked to the accuracy and power of your experiments.
If you need to find the sample size, but don’t really have the time to go through all the process, use Omniconvert’s Online Sample Size Calculator!
Our calculator saves you time and effort, delivering fast, reliable results without the fuss.
Simply input all the relevant data in the calculator above and let us do the rest!
