Null vs Alternative Hypothesis: A/B Guide (2026)
- The null hypothesis (H0) claims no real difference between control and variant; the alternative hypothesis (H1) claims there is one. Exactly one is true.
- A test assumes the null is true and looks for evidence to reject it. You never prove the null, you only reject it or fail to reject it.
- The p-value measures how likely your data is under the null. Below your significance level (often 0.05), you reject the null and accept the alternative.
- Use a two-tailed alternative by default so you catch a change in either direction, including a variant that hurts conversion.
- Write hypotheses in an if-then form that is specific and measurable. Omniconvert Explore turns them into A/B tests and reports the confidence level.
The null hypothesis (H0) is the default claim that there is no real difference between the control and the variant, and that any variation you see is due to random chance; the alternative hypothesis (H1) is the opposite claim, that a real difference exists because of the change you made. Every A/B test is a contest between these two statements. Omniconvert has run hypothesis-driven experiments across the CROBenchmark dataset of 7,000+ websites in 15+ industries, against 248+ audit criteria, drawing on 13 years in eCommerce conversion rate optimization [CROBenchmark Report 2026, Omniconvert].
Omniconvert Explore is the conversion rate optimization platform that turns a hypothesis into a running A/B test and reports whether the evidence is strong enough to reject the null. This guide defines both hypotheses, shows how they differ, how to write one that is actually testable, and how the p-value and significance level decide which one wins in an eCommerce test. Every section answers the question directly, then goes deeper.
What are the null and alternative hypothesis?
Hypothesis testing works by contradiction. Rather than trying to prove your idea directly, you assume the opposite, the null, and then check whether the data makes that assumption look unlikely. If it does, you reject the null and accept the alternative. If it does not, you fail to reject the null and treat the result as inconclusive. The two hypotheses are always a matched pair:
| Null hypothesis (H0) | Alternative hypothesis (H1) | |
|---|---|---|
| What it claims | No real difference or effect | A real difference or effect exists |
| In A/B testing | The variant performs the same as the control | The variant performs differently from the control |
| Assumed at the start | Yes, treated as true until disproven | No, it is what you are trying to support |
| The test can | Reject it, or fail to reject it | Be accepted if the null is rejected |
| Cause of any difference seen | Random chance | The change you made |
What is the null hypothesis?
The null hypothesis is deliberately skeptical. It represents the status quo, the assumption that your new headline, button, or layout changes nothing. That skepticism is the point: it forces the burden of proof onto the change. A variant does not get credit for looking better; it has to produce data unlikely enough under the null that chance becomes an implausible explanation.
A concrete example: you want to test whether a blue Buy Now button beats the current green one. The null hypothesis is, changing the button color from green to blue has no effect on the conversion rate. Until the test produces strong evidence otherwise, that is what you assume is true, and it is what the variant must overturn.
What is the alternative hypothesis?
The alternative hypothesis is your idea stated so it can be tested. Continuing the button example, the alternative is: changing the button color from green to blue changes the conversion rate. If the data lets you reject the null, this is the statement you accept. The alternative comes in two shapes, and choosing the right one matters:
- Two-tailed (nondirectional): Predicts only that the variant differs from the control, up or down. This is the safer default, because it catches a change in either direction, including a variant that quietly hurts conversion.
- One-tailed (directional): Predicts a change in one specific direction, for example that the variant increases conversion. It has more statistical power for that direction but is blind to a move the other way, so use it only when the opposite result would not change your decision.
For most eCommerce A/B tests, a two-tailed alternative is the responsible choice: shipping a variant that looked like a winner but actually lowered revenue is exactly the mistake a directional test can hide.
How to write a strong hypothesis
A hypothesis you can test is grounded in evidence and written to be falsifiable. Follow four steps:
-
Start from a real observationBase the idea on data, not a hunch: a drop-off in a funnel step, a heatmap, or a survey answer. Relevance to an actual problem is what makes a result worth acting on.
-
Write it in if-then formState it as "if we make this change, then this metric will move." For example, if we move the reviews above the fold, then the add-to-cart rate will increase. The structure forces a cause and an expected effect.
-
Name the exact metricDecide in advance what you measure, conversion rate, revenue per visitor, or add-to-cart rate, so the test has one clear yardstick and you cannot rationalize the result after the fact.
-
Make it specific and measurableSay what changes, for whom, and where. "Change the PDP" is vague; "add a shipping-time line under the price on mobile product pages" is testable. Specificity is what separates a hypothesis from a wish.
The null and alternative hypothesis in A/B testing
The mechanics are the same for every test. You define the null and the alternative, set the significance level (alpha) that draws the line for how much evidence you need, then run the experiment until it reaches a reliable sample. Explore calculates the p-value and the matching confidence level, and the decision rule is simple: if the p-value is below alpha, the result is statistically significant, so you reject the null; if not, you fail to reject it.
Two mistakes shadow every decision. Rejecting a null that was actually true is a false positive, a Type 1 error, and it ships a change that does nothing. Failing to reject a null that was actually false is a false negative, a Type 2 error, and it discards a real winner. Your significance level and sample size are what keep both in check. The table shows real Omniconvert Explore experiments where the data was strong enough to reject the null and accept the alternative.
| Alternative hypothesis tested | Client | Result (null rejected) |
|---|---|---|
| Adding a "New" badge to a newly launched product lifts conversion | AliveCor | +25.17% conversion rate, +29.58% revenue per user (99.4% confidence) |
| A redesigned homepage CTA increases conversions | Bonia | +218% conversion rate on women's watches |
| Cart urgency messaging increases completed orders | Orange Romania | +7.65% conversion rate, +11.53% revenue per visitor |
| Personalizing category pages by attribute lifts conversion | Avon | +96.63% conversion rate |
| Simplified filtering improves mobile conversion | CLX Gaming | +123.4% conversion rate on mobile |
Have a hypothesis worth testing? See whether the data is strong enough to reject the null.
Run your A/B test with Omniconvert Explore →Testing your hypothesis with Omniconvert Explore
A hypothesis is only useful once you can test it cleanly, and that is where Omniconvert Explore fits. You translate the null and alternative into a control and a variant, launch the split, and Explore handles the statistics, tracking conversions and reporting the confidence level so you know the moment the evidence is strong enough to reject the null. Because it isolates one change at a time, you can trust that the movement came from the variable you tested, not a confounding factor.
The number is only half the answer. Explore pairs each test with heatmaps, session recordings, and on-site surveys, so when you reject or fail to reject the null you also understand why. And optimizing one page is a single win: Nexus by Omniconvert is the AI eCommerce growth engine that turns the customer and profit data behind those tests into ranked actions, so a validated hypothesis feeds a wider growth loop instead of ending at one result.
Frequently Asked Questions
The null hypothesis, written H0, is the default claim that there is no real difference or effect, and that any variation in the data is due to random chance. In A/B testing it states that the variant performs no differently from the control: the change you made had no effect on the conversion rate. A test does not try to prove the null hypothesis; it assumes it is true and looks for evidence strong enough to reject it. If the evidence is not strong enough, you fail to reject the null and treat the result as inconclusive rather than a win.
The alternative hypothesis, written H1 or Ha, is the claim that there is a real difference or effect, the thing you actually want to find out. In A/B testing it states that the variant performs differently from the control because of the change you made, not because of chance. It is the mirror image of the null hypothesis: if the data gives you enough evidence to reject the null, you accept the alternative. A good alternative hypothesis is specific about what changes, for whom, and on which metric, so the test can clearly confirm or fail to support it.
The null hypothesis says there is no difference between the control and the variant; the alternative hypothesis says there is a difference. They are mutually exclusive: exactly one can be true. A test assumes the null is true and measures how likely the observed data would be under that assumption, expressed as a p-value. If that probability is below your significance level, you reject the null and accept the alternative; if not, you fail to reject the null. You never prove the null true, you only fail to find enough evidence against it.
Say you want to test whether a blue Buy Now button converts better than the current green one. The null hypothesis is: changing the button color from green to blue has no effect on the conversion rate. The alternative hypothesis is: changing the button color from green to blue changes the conversion rate. You split traffic between the two versions, measure conversions, and calculate a p-value. If it falls below your significance level, you reject the null and conclude the color changed conversion; if not, the result is inconclusive and the color likely made no reliable difference.
No. Hypothesis testing can only reject the null hypothesis or fail to reject it, never prove it true. Failing to reject the null means the test did not find enough evidence of a difference, which is not the same as proving there is none: the effect could be real but too small for your sample to detect, a Type 2 error. This is why results are reported as statistically significant or not significant, and why an inconclusive A/B test means keep the control or test again with more traffic, not the change definitely does nothing.
A one-tailed (directional) alternative hypothesis predicts a change in one specific direction, for example that the variant increases conversion. A two-tailed (nondirectional) alternative hypothesis predicts only that the variant differs from the control, without saying whether it goes up or down. Two-tailed is the safer default for most A/B tests because it catches a change in either direction, including a variant that unexpectedly hurts conversion. Use a one-tailed test only when a change in the opposite direction would be irrelevant to your decision.
The p-value is the probability of seeing your result, or one more extreme, if the null hypothesis were true. A small p-value means the data would be unlikely under the null, which is evidence against it. When the p-value falls below your significance level, often 0.05, you reject the null and accept the alternative. The p-value is not the probability that the null is true, nor the size of the effect: it only measures how surprising the data is under the assumption of no difference, which is why you also check the actual lift.
Omniconvert Explore is the conversion rate optimization platform that turns a hypothesis into a running A/B test without engineering: state the null and alternative, build the variant in a visual editor, split traffic, and let Explore track conversions and significance until the result is reliable. It reports the confidence level so you know whether to reject the null, and pairs the numbers with heatmaps and on-site surveys that explain why a variant won or lost. Across 70,000+ experiments, that keeps decisions grounded in evidence, not opinion.
Before your next A/B test, write both hypotheses down in one line each. State the null (this change has no effect on conversion) and the alternative (this change moves conversion), name the exact metric, and decide your significance level in advance, usually 0.05. That small habit forces a specific, testable idea instead of a vague hunch, and it stops you from moving the goalposts once the data comes in. Then run the test to significance rather than stopping the moment a variant looks ahead. A hypothesis you can reject is worth more than an opinion you cannot.
Turn your hypothesis into a real test with Explore
Omniconvert Explore lets you state a hypothesis, build the variant in a visual editor, split traffic, and track significance until the result is reliable, then read the why with heatmaps and on-site surveys, all in one CRO platform. Stop guessing whether a change worked and let the data reject or confirm it. Free A/B testing for up to 50,000 visitors per month, trusted across 70,000+ experiments.