What 70,000 CRO Experiments Reveal
- Treat any ranked list of winning CRO tactics as unusable: effects do not transfer between stores.
- Expect most of your available gain to sit in the checkout and the product page, not the homepage.
- Declare one metric before the test runs, because a test with two metrics has no result.
- Measure revenue per visitor beside conversion, or you will ship margin-destroying wins.
- Run fewer, larger changes rather than many small ones, when traffic is limited.
- Build the programme to compound: a documented losing test is worth more than an undocumented win.
People ask me for the list. After 70,000 CRO experiments, which changes win? It is the right question to ask and the answer disappoints everybody, including me: there is no list, because the thing that transfers between stores is not the change. It is the way the programme around the change is run. Last updated: November 2026.
I have spent 13 years in eCommerce, and Omniconvert has averaged a 23.2% conversion uplift across 70,000+ experiments for 2,500+ brands, measured alongside the CROBenchmark dataset of 7,000+ websites in 15+ industries against 248+ audit criteria [CROBenchmark Report 2026, Omniconvert]. That uplift figure is the one worth quoting. A ranked table of winning tactics drawn from the same body of work would be the least defensible thing I could publish.
So this piece explains why, names the places the losses genuinely concentrate, and sets out the four design decisions that separate a programme which compounds from one which generates activity. The broader argument that these functions only pay off when read together is in CRO, creative & AI visibility tie into one growth system.
What 70,000 CRO experiments say about winning tactics
Take the most-cited example in the industry: button colour. Somewhere there is a real experiment where changing it produced a real, significant lift. There are also real experiments where the same change did nothing and real experiments where it lost. All of those results are correct.
What differs is everything the test was embedded in. The traffic mix, the price point, how much the visitor already trusted the brand, what the rest of the page was doing, what season it was. A result is a measurement of a change inside a context, and the context is most of it.
This is why a published list of winning tactics does more harm than a blank page. A blank page produces a hypothesis about your own customers. A list produces a backlog of other people's answers, run against your traffic, generating inconclusive results that get recorded as wins because somebody needs the quarter to have gone well.
The useful version of other people's work is the reasoning rather than the outcome. Reading how a team arrived at a hypothesis is transferable, and a selection of worked examples sits in 10 inspirational CRO case studies. Ways to generate your own candidates are in CRO experiment ideas for eCommerce.
Where the available gain actually sits
This is the one thing close to a general finding, and it is a statement about where to look rather than about what to change.
Abandonment concentrates at the point of commitment. Unexpected cost has been the most frequently cited reason shoppers abandon a purchase for as long as Baymard Institute has measured it, alongside around 70% average cart abandonment [Baymard Institute, 2025]. That is a checkout problem and a pricing-presentation problem, and no amount of homepage work reaches it.
The product page is the other concentration, because it is where a shopper decides whether the thing is right for them. In practice that means the questions a page fails to answer: sizing, materials, what happens if it is wrong, when it arrives.
Homepages get tested most because they are what the leadership team sees, what the agency presents, and what everybody has an opinion about. I have watched a quarter of testing capacity go into a homepage hero on a store where most sessions arrived on a product page from paid social and never saw it.
One metric, declared before the test runs
This is the cheapest discipline in experimentation and the most frequently skipped. Write down, before the test starts, the single number that decides it, and what size of difference would count.
The reason it matters is not statistical purity. It is that most interesting tests move two numbers in opposite directions. Simplify a checkout and conversion often rises while average order value falls, because the upsell you removed was doing something. Both effects are real, and only a pre-declared metric tells you which one you were buying.
Revenue per visitor is the metric I would default to for most commerce tests, precisely because it refuses to let a team celebrate a conversion lift that sold cheaper baskets. The measurement groundwork for that sits in CRO metrics, and the margin version of the same argument is in how to measure True Profit, not ROAS.
AliveCor used Omniconvert to run a structured A/B testing programme and achieved +21% conversion rate, +5% revenue per visitor, and 94% statistical relevance across their experiments. [Omniconvert, AliveCor case study] The detail I would draw attention to is the third figure rather than the first two: they knew how confident they were, which is what made the first two worth acting on.
| Programme property | What it decides | Failure mode when undefined |
|---|---|---|
| Where you test | Size of available gain | Capacity spent on the homepage |
| Declared metric | Whether a result exists | The flattering number wins |
| Change size | Time to significance | Inconclusive read as positive |
| Stopping rule | Confidence in the result | Test stopped when it looks good |
| Hypothesis source | Whether learning compounds | Other people's tactics, re-run |
| Documentation | Value of a losing test | The same test, run again in a year |
The way teams most often get burned here is by inheriting a testing roadmap built from a conference talk. Somebody returns with fourteen tactics that worked for a brand an order of magnitude larger, the list becomes the quarter's backlog because it is concrete and already prioritised, and each item is run as a small isolated change against traffic that cannot resolve it. Twelve come back inconclusive, two show a flattering movement inside the noise, and both get shipped. The quarter is reported as a success with a cumulative uplift figure, the store's actual conversion rate is unchanged, and the team has learned nothing about its own customers, because not one of the fourteen hypotheses came from them.
Shopify's own merchant guidance has pointed teams at checkout and product-page friction rather than at homepage design for years, which matches where this dataset puts the recoverable loss [Shopify, 2026].
Fewer tests, larger changes
There is a counting instinct in this work that does real damage. Tests run per quarter is easy to report, so it becomes the measure of a programme, and the fastest way to raise it is to test smaller things.
The arithmetic goes the other way. The smaller the true effect, the more traffic you need to distinguish it from noise, and most stores do not have the traffic to resolve a small effect within a sensible timeframe. So a programme of many small tests produces mostly inconclusive results.
Inconclusive results are where programmes go wrong, because they are rarely reported as inconclusive. They get reported as small wins, shipped, and added to a cumulative uplift figure that nobody can reproduce.
Bigger changes are harder to agree internally and they are what a limited-traffic store can actually learn from. A genuinely different product page beats fourteen variations of a heading, whichever way it goes.
What makes a programme compound
The difference between a programme that improves for years and one that plateaus is not talent or budget. It is whether the team can answer what it believed, what it tested, what happened and what it now believes instead.
Most teams record outcomes. Won, lost, shipped, reverted. That record is almost useless a year later, because it does not contain the reasoning, so the same hypotheses come back around with new staff attached to them.
A losing test with its reasoning written down is a genuine asset. It removes a hypothesis from the space and it usually suggests a sharper one, which is why experienced programmes get faster rather than slower as the easy wins disappear.
This is also the honest reason the 23.2% average uplift figure means something while a tactic list would not. It is a measure of what a disciplined programme produces over many stores and many years, not a claim that any specific change will do that for you. How the process is built, including where automation helps, is in the AI CRO process, and the practical mechanics are in CRO technique.
Once experiment results, creative performance and margin all have to be read against each other, this stops being a testing problem and becomes a data one. Omniconvert Commander (formerly Nexus) is an AI for eCommerce growth engine that unifies commerce data, prioritises experiments by True Profit, and generates campaigns and creative you approve before they go live.
FAQ
What do 70,000 CRO experiments actually reveal?
That programme design predicts results more reliably than tactic selection does. The same change produces opposite outcomes in two stores often enough that no tactic can be recommended on evidence alone, while the teams that improve consistently share how they choose, measure and document tests. The transferable knowledge is in the method rather than in the changes.
Why can I not just copy the winning tests?
Because an experiment measures one change against one audience on one site at one moment, and all four of those conditions differ at your store. A result tells you what happened there, not what will happen for you. Copied tests are the most common reason a programme produces activity and no compounding gain.
Where does most of the available gain sit?
In the checkout and on the product page, which is where the decision is actually made and where the largest documented losses occur. Homepages attract disproportionate testing attention because they are the most visible page internally, and they are rarely where a purchase is won or lost.
Why does declaring one metric matter so much?
Because a test with two metrics has no result. If conversion rises and revenue per visitor falls, a team without a pre-declared metric will choose whichever number supports the outcome it preferred. Declaring the metric before the test runs is the cheapest discipline available and the one most often skipped.
Should I run many small tests or fewer large ones?
Fewer and larger, when traffic is limited. A small change needs more traffic to produce a trustworthy result than most stores have, so a programme of small tests generates inconclusive outcomes that get read as wins. A larger change reaches significance sooner and teaches more, whichever way it goes.
What makes a testing programme compound?
Documentation of what was learned rather than of what shipped. A losing test that is written up changes which hypotheses the team forms next, while a winning test that is not written up teaches nobody anything after the quarter it ran in. The programmes that improve over years are the ones that treat the record as the asset.
What is the bottom line after 70,000 experiments?
Stop looking for the list. Every hour spent hunting transferable tactics is an hour not spent on the four decisions that actually predict your results: where you test, which single metric decides it, how large the changes are, and whether anybody writes down what was learned.
Start with the metric, because it costs nothing and it changes what counts as a win this quarter. Then move your testing capacity off the homepage and onto the checkout and the product page. That reallocation is uncomfortable, it will be argued about, and it is where the recoverable loss has been sitting the whole time.