Before we tackle whether personalization has replaced A/B testing, let’s slow down and take a minute to talk about the two most popular Conversion Rate Optimization (CRO) tools used by marketers today.
First we have personalization, a CRO tool that has increased in popularity over the last 18 months. When you think about it, this is no major surprise. Your website should be your best sales asset, and the best way to do that is to cater a message/offer to the right person, at the right place, and at the right time.
Then there’s the reigning king of optimization tools, A/B testing. A/B testing, simply put, is the process of taking two different pages, splitting the traffic, and seeing which increased your desired conversion metric. This conceptually simple process that supercharges the bottom line has grown from a sparsely used tool to a cornerstone in a digital marketer’s tool belt.
In fact the sixth annual Conversion Rate Optimization Report by RedEye & eConsultancy, found that 67% of companies A/B test. This is the fourth year in a row that surveyed companies have ranked A/B testing as the #1 CRO method.
Both tools are extremely powerful and every marketer should already be using them. For those that are looking for instant gratification, here’s the answer to the question asked in the title: No, personalization has not replaced A/B testing and here’s why:
The two tools are foundationally different! Personalization is meant to hyper-target a visitor with dynamic content based on advanced segments.
A/B testing is a verification tool – nothing more, nothing less. If we do not verify changes made to our site, marketing campaigns, or emails in a test environment we lose data & insights. Without data and insights digital marketers are flying blind and aren’t making changes based on what works best, but what they think works best.
Personalization is rapidly growing in popularity A WhichTestWon study showed that 59% of marketers are doing some kind of segmentation work and 32% are using dynamic content. Personalization, as I mentioned before, is a combination of the two practices, i.e., providing relevant content via dynamic insertion for a recognized segment.
Personalization is not new; marketers have been using these tactics prior to having the ability to dynamically insert content based on advanced segments. Think about abandoned cart emails, these messages were sent to a specific subset of visitors and became one of the best retargeting strategies in an ecommerce marketer’s arsenal.
Personalization, like A/B testing, shouldn’t be done haphazardly. Identify your most important segments, and begin targeting them with personalized content. Here are some segments to initially target if you are new to personalization:
• New Vs. Return Visitors
• The Visitor’s Device
• The Visitor’s Geolocation
I also recommend looking at page behavior to properly segment offers. Metrics to look at include page depth, time on site, scroll patterns, etc…
This is only the tip of the iceberg when it comes to personalization. The real trick comes in identifying pages that can benefit from personalization. Dig through your analytics to identify problem pages and ask yourself ‘Will adding personalization help this page?’ If you answered ‘Yes’ then build your hypothesis and test it out!
A/B Testing As A Verification Tool
As I said above, A/B testing is a verification tool, nothing more. After you’ve developed a test hypothesis and launched your test, your A/B test will only tell you if the control or variation won. Again, that’s it!
The power of A/B testing comes in how a marketer develops the test and analyzes the results. This is why no other optimization tool we add to our testing tool belt will usurp the king of CRO. Personalization is a tool to increase conversions, where A/B testing is the tool that verifies if it is actually working.
Where The Two Butt Heads
For an A/B test to be valid, you need both a large and representative sample size. This is where tension occurs between the two methods. Personalization is only triggered on a subset of traffic, so it reduces the sample size and is not a true representation of the whole.
That said, there are times where you simply cannot test your personalization efforts because you don’t have a big enough sample size. Furthermore, the learnings gathered from small-scale personalization efforts are not scalable to other aspects of your site. That said, you can still use personalization on these pages! Just because you can’t test, doesn’t mean you can’t optimize – and personalization is one of the best ways to optimize in 2014-2015.
Just Use Both!
Repeat after me:
“Not all pages are test worthy”
Once you understand this important CRO maxim, it is easy to see how personalization and AB testing compliment one another. Personalization has a proven track record to increase conversions, but due to highly target segments may not be testable.
However, you can take learnings you get from testing larger subsets of traffic and apply these learnings to these ‘un-testable’ segments.
Just like any good CRO campaign, you have to pick your battles. Know when and where it is appropriate to test, and focus your A/B testing efforts there. As for the pages that are not test worthy, use tangential data and other CRO tools, e.g., personalization, to lift your conversion rate!