What Is a Margin of Error? Definition, Formula & Examples

A margin of error is the range around a sample estimate within which the true value for the whole population is likely to fall, at a stated level of confidence. When you measure something from a sample rather than the entire population, a survey of 1,000 customers, an A/B test on a fraction of your traffic, the number you get is an estimate, not the exact truth, because a different sample would give a slightly different result. The margin of error quantifies that uncertainty. If a survey finds that 60% of customers prefer a feature with a margin of error of plus or minus 3 percentage points at 95% confidence, it means the true figure is very likely between 57% and 63%. The margin of error is always tied to a confidence level, usually 95%, which says how often the interval would contain the true value if you repeated the study many times. A smaller margin of error means a more precise estimate; a larger one means more uncertainty. It is one of the most important, and most often ignored, numbers behind any result drawn from a sample.

For a proportion, the margin of error is the critical z-value for your confidence level multiplied by the standard error of the proportion: margin of error = z × √(p(1 − p) / n). Here p is the observed proportion (for example 0.5 for 50%), n is the sample size, and z is the number of standard deviations for the confidence level, which is 1.96 for 95% confidence and 2.58 for 99%. A worked example: with an observed proportion of 0.5 and a sample of 1,000, the standard error is √(0.5 × 0.5 / 1,000) ≈ 0.0158, and multiplying by 1.96 gives a margin of error of about 0.031, or plus or minus 3.1 percentage points. Two things fall out of the formula. First, precision comes from sample size through a square root, so to halve the margin of error you must roughly quadruple the sample. Second, the margin of error is largest when the proportion is near 50%, which is why a proportion of 0.5 is often used as a conservative, worst-case assumption when planning a study.

Three things drive the size of a margin of error: the sample size, the confidence level, and the variability in the data. Sample size is the lever you usually control. A larger sample gives a smaller margin of error, but because it enters the formula through a square root, the return diminishes: quadrupling the sample only halves the margin of error, so precision gets expensive. Confidence level sets how sure you want to be, and higher confidence widens the interval: a 99% margin of error is wider than a 95% one for the same data, because being more certain that the interval contains the true value means casting a wider net. Variability is the spread in what you are measuring; for a proportion, uncertainty is greatest near 50% and shrinks as the proportion moves toward 0% or 100%. So you narrow a margin of error mainly by collecting more data, and you widen it by demanding higher confidence, a genuine trade-off between precision and certainty that no amount of analysis can escape, only more data can.

Margin of error matters in A/B testing because every conversion rate you measure in a test is a sample estimate, not an exact truth, and the margin of error tells you how much to trust it. Suppose variation B converts at 5% and the control at 4%. That looks like a win, but if each estimate carries a margin of error of plus or minus 1.5 percentage points, the two ranges overlap heavily and the apparent difference could easily be noise. Only when the samples are large enough that the margins of error are small, and the difference between variations is bigger than that uncertainty, can you trust the result. This is the same idea that statistical significance and confidence intervals express: a significant result is one where the observed difference is unlikely to be explained by sampling variation alone. The practical lesson is to run tests until they reach an adequate sample size and significance, rather than calling a winner early on a difference that sits inside the margin of error, which is one of the most common and costly mistakes in experimentation.

They are two parts of the same statement and are always used together, but they answer different questions. The confidence level is how sure you are, expressed as a percentage, usually 95%. It describes the reliability of the method: if you repeated the study many times, about 95% of the intervals you built would contain the true population value. The margin of error is how wide the interval is, expressed in the units of what you measured, such as plus or minus 3 percentage points. It describes the precision of this particular estimate. A complete result needs both: 60% preference, plus or minus 3 points, at 95% confidence, ties the estimate to a precision and a reliability. The two are linked: for the same data, asking for a higher confidence level widens the margin of error, because being more certain that the range contains the truth forces the range to be wider. So confidence level is your chosen certainty, and margin of error is the precision you get as a result, at that certainty and that sample size.

The main way to reduce a margin of error is to increase the sample size, but because sample size enters the formula through a square root, the gains diminish, so it pays to be deliberate. To halve the margin of error you must roughly quadruple the sample, and to cut it to a third you need about nine times the data, which is why chasing a very tight margin can be costly. Beyond collecting more data, you can lower the confidence level if a little less certainty is acceptable, since a 90% interval is narrower than a 95% one, though most work keeps 95% as the standard. You can also reduce variability where the study design allows, for example by measuring a more homogeneous group or a metric with less spread, which tightens the interval for the same sample size. What you cannot do is wish the margin of error away: a small sample carries real uncertainty, and reporting an estimate without its margin of error hides that. The honest approach is to plan the sample size for the precision you need before you start, and to report the margin of error alongside every estimate drawn from a sample.

Margin of error is the reason an A/B test result can look convincing and still be noise, and Omniconvert Explore is designed to keep you on the right side of that line. Explore is an A/B testing and experimentation platform that reports statistical significance and confidence for each experiment, which is the applied form of the margin-of-error idea: it tells you whether the difference between a variation and the control is large enough, relative to the uncertainty in the samples, to be trusted rather than explained by chance. Because Explore surfaces this, you are steered away from the classic mistake of calling a winner early on a difference that actually sits within the margin of error, before the samples are big enough to be sure. Its segmentation is useful here too: when you slice results by audience, each segment is a smaller sample with a wider margin of error, and reading significance per segment keeps you from over-interpreting a thin slice. Drawing on more than 70,000 experiments across 7,000+ websites, with an average uplift of 23.2%, Explore helps you act on differences that are real, not on noise inside the margin of error.