This episode of the eCommerce Growth Show is all about the little things you didn’t know about CRO.
There is no need for a formal introduction for our guest, Ton Wesseling, because his session says it all. Without further ado, check it out!
PS: Enjoy the graphics Ton is drawing during the show!
Who is Ton Wesseling?
Ton Wesseling is known for being the founding father of Online Dialogue – thought leaders in evidence-based growth. He has more than 20 years of experience in online optimization and is recognized worldwide as an influential thinker, writer and public speaker on conversion optimization and A/B-testing.
Ton hosts the A/B-testing mastery course on the CXL Institute, owns the ABtestguide.com platform and is also the founder of The Conference formerly known as Conversion Hotel.
Key takeaways from this episode
What is evidence-based growth?
I would like to draw what is called the Pyramid of Evidence.
? Check out the video at 10:19 to see the graphic version drawn by Ton Wesseling! ?
- The risk of bias: going up the pyramid, you have a low risk of bias; going down the pyramid, you have a high risk of bias
- The quality of evidence: going up the pyramid, you have a high quality of evidence; going down the pyramid, you have a low quality of evidence
There are several layers in the pyramid.
- On the lowest layer, there is the Expert Opinion, looking at your website without any data to back up and just based on gut feeling and experience. There is a high risk of bias and low quality of evidence.
- On the highest layer, there is A/B Testing, with the lowest risk of bias and the highest quality of evidence.
- On top of the pyramid, there is a systematic review of randomized controlled trials. If you have run 100, 200 experiments, then you can do a systematic review and really gain insights about consumer behavior.
Outperforming your competition is a matter of gaining evidence. If you have more evidence, you will be able to outperform the competitor that has less evidence.
If you look at companies like Google, Facebook, Amazon, what they do is built up more data. They buy companies to get more data because once they have more data, they can make more decisions. They are speeding up the number of experiments they’re are running really fast.
The process of experimentation
- The first point is about implementing.
There are many companies out there that run a successful A/B testing program and come up with all sorts of winners, but they are not getting implemented by the developers, IT department, product team, or cross-functional team. That’s a real nightmare!
Hopefully, you’re getting insights from the hypothesis you were testing and you’re able to feed those insights to the marketing team so that they can build better campaigns and adverts. But if it’s not getting implemented, then something is fundamentally wrong. Stop doing what you’re doing!
I’ve been lucky to have been helping companies for the last like 20 years with setting up experimentation and I’ve made many mistakes. Usually, we would start with marketing or products and come up with ideas and then get stuck with IT because they were not implementing.
My advice is nowadays if I help companies setting up a culture of experimentation, the first thing we have to do is get development on board, the people that develop the codes, that are able to push the button to release something. Those are the ones where we should start because once they believe in the fact that every line of code that they are going to ship should be shipped as an experiment, then we’re done. We never have to convince marketing or product again that they run experiments because everything they ship is an experiment. In the end, we’ll find out which teams are really doing well and which teams are not doing so well.
Many experiments nowadays are being built as codes being shipped already. If nothing gets implemented, stop doing what you’re doing. Don’t worry about your next experiments.
- The second point is about the CRO team.
I’ve seen many companies starting with one conversion optimizer or one digital marketer that does something with data, then it grows at some point to a growth team or a conversion optimization team.
A conversion optimization team is fun. I’ve been there. I’m addicted to running A/B experiments and optimizing a website and trying to understand why certain behavior is happening. So this is a fun team.
What I’ve seen in most companies is that this specific team has some really proud winning people. They are the ones that they want to go forward and change things and they have a big force in the company. They’re not introverts, they’re extroverts. They’re really good at convincing people that they should get more money to make the team even bigger. They’re making money for the company, too, because with their experiments and if they’re able to get them implemented, that’s even better.
At some point, you will understand that if you get bigger, every marketing team, every product team should be running experiments. Even the customer service department should be running experiments, even the packaging department should be running experiments.
? Check out the video at 28:35 to see the graphic version drawn by Ton Wesseling! ?
At some point, once there is belief and there is some sort of a framework, you want to move on. So then what happens in many companies is that they are trying to teach those departments how to run experiments. So to their marketing team or their product team: this is how you should run an experiment. This is why you should experiment on this, this is how you should do it. This is how you should use data to base your decisions on.
In the beginning, they will make mistakes. They are new to the game. They will make statistical mistakes, bad hypotheses. They start doing button color testing and so on. So they will tell them, “Oh, that’s wrong! That’s really bad. You are really bad at running experiments.”
This is not a way to get experimentation in the whole company because, in the end, it doesn’t make sense. The decision-makers are seeing, “OK. This is good. They’re making money. Now, everyone needs to adopt this approach of experimentation and data-driven decision making.”
So, they’re using them to teach everyone. But in the end, they’re all running away because they hate it. They hate the way this group is behaving because these are the winners and once they become teachers, they should become enablers. They should be humble and should allow departments to make mistakes and not tell them what they’re doing wrong but tapping on their back saying, “Good, good. You’ve run your first experiment. Even if the outcome is rubbish and it’s not getting implemented, you’re on your way.”
Everyone needs to embrace this and they should become a small local center of excellence supporting those teams and running experiments. They should take care of the trustworthiness of the data, they should take care of the tools, the quality of the tools, the user-friendliness of the tools, they should come up with courses, explain them, but they should be the ones running all those experiments and making those decisions and using those tools and being enabled by a center of excellence.
Getting more lifetime value out of your customers
KPI (Key Performance Indicator)/OEC (Overall Evaluation Criterion)
? Check out the video at 43:05 to see the graphic version drawn by Ton Wesseling! ?
- Customer Lifetime Value
- Profit margin
- Transactions
- Behavior
- Clicks
