How to Conduct Qualitative Research

Conversions take place when targeted traffic meets a relevant offer. To be able to provide that offer in a format the traffic prefers, you first have to know your target audience and their needs.

How do you understand your target audience?

Through data.

Why you need data

Data reveals how visitors interact with your site.

When you regularly collect and analyze data, you can better define a group of users that are likely a good fit for your product. You can also pinpoint locations that are creating resistance to conversions.

Without data, the CRO process will be left to intuition rather than actionable insight. This makes optimization less a scientific process and more a matter of guesswork – not something that can create sustainable results.

Data tells you:

• Where your site’s visitors come from, and what pages they visit on your site.
• Your visitors’ on-page behavior. Do they truly engage with your content or do they just bounce off without clicking anything?
• The stage where users drop-off in the conversion funnel.
• How different demographics and devices interact with your site.

And so on.

You may be wondering where to find such data?

Let’s start with qualitative research.

Qualitative Research

The goal of qualitative data is to gather insights into your users’ behavior and determine the reason(s) for it.  Peeking inside the mind of your customers allows you to tailor your site and content in a manner that provides them the greatest value.

There are a variety of methods to collect qualitative data, such as:

• Customer surveys
• Interviews
• Usability tests
• Eye-tracking and mouse tracking
• NPS scores

Let’s take a look at how you can use each to gather actionable insights.

I. Customer Surveys

A survey is one of the most used methods for qualitative data collection since it is cost-effective and easy to run.

Surveys can reveal what convinced your current customers to buy and what is drawing away potential customers. By learning directly from the source, you can implement an optimization strategy based on the customers’ needs.

Surveys are generally conducted online through websites or emails, though you can also pick up the phone and call customers up directly (works well in industries with older audiences).

Another popular method for conducting on-site surveys is through slide-in “widgets” or pop-ups. Such on-site surveys are especially useful for identifying the source of friction as they target customers who have not bought from you.

Such on-page surveys can help you:

• Locate exact locations of bottlenecks where users drop off
• Understand the root cause of abandonment at a specific stage in the conversion funnel.

How to Create Effective Surveys

From a CRO perspective, when you design your survey, you want to ensure that you learn 4 key facts about your existing and potential visitors:

• Who are they?
• What issue were they seeking to resolve while visiting your site?
• What was their purchasing journey like while choosing your product? For example, the number of comparisons they did or how many alternatives services did they browse.
• Any fears, doubts or hesitations they faced before purchase?

Before we look at how you can conduct surveys, keep the following points in mind:

• Don’t ask too many questions at once: Time is valuable and lengthy surveys annoy customers. Limit your survey questions to about 5-10 in total.
• Offer a prize as an incentive to complete surveys: A survey can feel like work. To compensate customers for their time, try offering something in return. Offering a prize such as a gift card or an iPad is guaranteed to increase the number of people who take the time to fill out your survey.
• Ask open-ended questions: While multiple-choice options may be quick to answer, they don’t provide subjective insights. You want to ask open-ended questions and allow users to write a response in their own words, although this might affect the total number of responses you get.

With the above goals in mind, you can begin to frame your questions which lead to the above insights.

Here are some examples of questions you can ask your visitors to help you get started:

• Describe yourself in one sentence. It helps understand who your respondents are and their background.
• What are you using our product for? It helps you understand exactly how customers use your product.
• How did you hear about our product? It helps you connect the dots in your customer acquisition process.
• What feature would you miss the most if it wasn’t available? This question is a great way to zero-in on critical features.
• What feature you would NOT miss if it was available? It helps identify features that aren’t used much and thus, should be a lower priority for development.
• What are the top 3 things that led you to buy from us? Great way to figure out what you are doing right.
• What doubts and hesitations did you have before joining/buy? It helps identify FUDs (Fears, Uncertainties, Doubts).
• Would you recommend us to a friend? If not, why not? A good way to gauge the overall effectiveness of your marketing and your product.

A Note About Surveys

While customer surveys provide a cost-effective method to gather consumer insights, they are not completely reliable as participants may not answer questions with real intent or honesty.  As there is little accountability for providing true or false information, many visitors may simply be trying to finish as quickly as possible for the possible incentive or freebie.  

Furthermore, surveys only reveal the emotions of visitors while browsing your site. There may be lapses in your user interface that are preventing conversions. Surveys can’t help you find these lapses; you’ll have to dig through your data and conduct usability tests to fix them.

II. Usability Tests

Usability testing is a method to evaluate how easy a website is to use. The test involves real users and measures how usable or intuitive a website is in guiding users to their goals.

Users are asked to complete tasks while being observed to determine where they encounter problems and experience confusion. If many users happen to experience similar issues, recommendations can be made to overcome these user issues.  

Usability tests essentially give you a real look at how your target customers use your site. This goes beyond merely collecting analytics data and making conjectures based on your target demographic. With a usability test, you get to actually see a real person navigate (and stumble) around your site.

This is a powerful weapon for not just increasing conversions but also crafting a better user experience for your customers.

Broadly speaking, usability tests can be of three types:

• Moderated (in-person): In these tests, the user is accompanied in-person by a trained researcher who records the user’s on-page behavior and takes notes. These are expensive to run but reveal a great deal about your site.
• Moderated (remote): This is similar to the above test, except the researcher and the user are in different locations. By using screen sharing tools, researchers can analyze and understand the user’s on-page behavior. These are cheaper than in-person tests, but still prohibitively expensive for small businesses.
• Unmoderated: In this test, the user is shown a website and asked for his/her responses. The responses are recorded remotely. Instead of a trained researcher, you get raw footage of the user’s on-page behavior for analysis.

Besides this, you can also use screen recording tools to record a user as he/she moves around the page. These tools also track the cursor movement so you know exactly what your users are doing on your site.

How to Conduct Usability Tests

It’s very easy to spend a small fortune on usability testing and not get any actionable insights. Choosing what to test, selecting test participants and using the right questions can have a big impact on your test results.

Here are a few things you should keep in mind for getting the most from your usability tests:

• Choose the right test participants: If you run a website reviewing tech products, testing with older women who don’t even use smartphones will not provide you any value. Use your customer demographics data as well as your buyer personas to find the right test participants.

• Test for different personas: Your target market will have several buyer-types based on age, income, gender, education and so on. Create different personas based on these elements and funnel them through different usability tests to truly understand their specific needs. The broader your product offering, the more personas you’ll have to test.
• Use your lead language carefully: The language you use to describe your site and the test to users can influence their performance. For instance, if you describe your site as a “highly user-friendly platform”, your users might actively try to find faults in your user-friendliness (over other elements) to test your claims.
• Plan your test structure: It is best to ask questions in a logical order and move from general questions to specific questions. For instance, your first questions should ask users about their general impressions of the site. Later questions should ask for specific details, such as the checkout process or payments page.

A Note About Usability Tests

Understand that usability tests are designed to improve a site’s UI/UX, not their conversion rates.

While in most cases these are one and the same thing – better UI/UX leads to better conversion rates – it isn’t necessarily so all the time.

Sometimes, designs that hamper the user experience might actually lead to better conversion rates. Amazon’s navigation menu free checkout process, for instance, isn’t great from a user experience perspective. Anytime customers have to go back to the product page, they have to press the “back” button several times.

Yet, from a CRO perspective, such a distraction-free design is great for getting more conversions.

Similarly, the pop-up form is widely derided for harming a site’s usability. Yet, it consistently leads to more leads and more conversions. Wall-Street.ro, for example, managed to capture 279% more emails by moving from this “user-friendly” (but obscure) sidebar form:

To a prominent, screen-covering pop-up:

At the same time, remember that the environment in which testing is conducted is not 100% representative of a real-life scenario. At best, it is an approximation. User testing sites recruit their users from all walks of life, but they may not always align with your ideal customers.

Furthermore, user testing is always done in an “artificial” setting where the user is only focused on testing the site. In real life, your users will likely have multiple open website tabs, distracting smartphones, TVs or co-workers around them, which will impact their user experience.

Keep this in mind when you consider results from a usability test.

III. Interviews

If you want access to insights about your customers’ innermost feelings, there is no substitute for picking up the phone and talking.

Interviews represent a lot of investment in terms of time and energy, but they can reveal deep insights into your site, product, and target audience. Interviews are about quality rather than quantity. Even one interview can lead to campaign-changing test hypotheses.  

The key to conducting a successful interview is not to make it feel like one in the first place. Your interview should feel like a conversation between two people. Chatting with your customers in real-time allows people to open up and reveal emotions that underlie their behavior.

In addition to existing customers, you can also interview prospects, leads, and your customer support staff as they interact with customers daily and are likely to have valuable insights.

How to Conduct Better Interviews

An interview should not be scripted but instead should be unstructured. Your job as the interviewer is to listen and occasionally steer the conversation in the direction of relevant questions.

What type of questions should you ask?  

That depends on what you hope to achieve from the interview.

• Do you want to create richer customer personas?
• Are you trying to find the voice of the customer and adapt your copy accordingly?
• Are you trying to understand the customer journey better?

Answering these questions will help you define your interview goal better. This, in turn, will help you in coming up with useful questions.

Here are some sample questions you can use:

• What were you feeling while using the product?
• When you discovered our solution, how did that make you feel?
• What features do you love using? What features do you miss?
• How easy is it for you to accomplish [a specific task]?

Avoid asking leading questions and asking ‘why’. This will unnecessarily influence their responses. Instead, focus on asking questions that reveal their emotions.

Now that you know what questions to ask, you may be wondering where to find customers willing to participate.

Here is a list of resources where you can find people for interviewing:

• Social search: Twitter is a goldmine to find people who are posting about your product or solution and starting a relationship with them can be as easy as a @ mention.
• Introductions: Ask for introductions from first degree contacts (LinkedIn is a great source for this). People are generally open to connect with you when introduced through close friends and family
• Existing customers: Offer existing customers an incentive for an interview. Do not underestimate the value people place in a prize or a giveaway no matter how insignificant it is.
• Cold calls. Cold call people, you are targeting. When you cold call, you have nothing to lose and all to gain with the possibility of someone willing to sit down and talk to you.  

You can also email your existing customer base and schedule interviews. Alex Turnbull of GrooveHQ sent out this email and interviewed close to 500 customers.

IV. NPS scores

Surveys, usability testing, and interviews are all methods that serve to measure customer satisfaction by means of direct inquiry with customers.

What if there was another way to measure a customer’s satisfaction without the need for interaction?

Enter Net Promoter Score (NPS)

The net promoter score (NPS) measures satisfaction by means of customer loyalty and sorts customers into three categories known as:

• Promoters
• Passives
• Detractors.

This is done by asking customers a simple question: “How likely is it that you would recommend us to a friend?”. Customers can rate the likelihood on a 10-points scale. They subsequently also be asked the reason for their rating.

This helps measure customer loyalty.

You might be wondering: why measure customer loyalty?

Because loyal and passionate customers stick around longer, contribute their suggestions and recommend you to their friends and family. In the long run, they translate into organic growth for your business.

For example, according to Bain – which came up with the NPS score – loyal customers classified as “promoters” are worth $9,500 more than detractors for one business.

Similarly, Bain found a close correlation between customer loyalty and growth rates for banks.

The NPS score is based on a 0 (unlikely to recommend) to 10 (likely to recommend) scale. Here is how the score relates to the three categories specified above:

• Score 9-10: Customers who score the business as 9 or 10 are classified as “promoters”. These customers exhibit loyalty and are highly likely to recommend and revisit you for repeat purchases.
• Score 7-8: These customers are classified as “passives”. Passive customers are dispassionate about your business, but they don’t actively deride your business. With better service and persuasion, you can turn passives into promoters.
• Score 0-6: Customers who score the business between 0 and 6 are “detractors”. Detractors do not display value-creating behaviors. Instead, they might actively harm your brand.

The NPS is calculated by a simple formula:

(NPS= # of promoters – # of detractors / total users)

The NPS is appealing for several reasons:

• Simplicity: There are no complicated methodologies to calculate the score. Customers are quickly segregated into broad, easy to understand categories.
• Ease of use: For businesses, setting up the test is exceptionally easy. For customers, giving feedback is just a two-steps process.
• Insight: Despite the limited data, the NPS score gives a fairly good overall assessment of how your customers feel about your products and how loyal they are to your business.

Omniconvert lets you run Net Promoter Score surveys.

Our platform includes a full-featured survey tool with a dedicated module for calculating NPS scores. You can include this in any survey format – as an on-site widget or in pop-ups. You can also customize the survey to include additional questions based on your customer’s score.

IV. Heat maps (Eye-Tracking and Mouse Tracking)

Heat maps are graphical representations of data where individual values are contained as colors within a matrix. They look something like this:

Here, areas that get more attention are shaded in red while others are green. This can help you understand what users are doing on a single page.

While heat maps are more visual than standard analytics tools, they provide information on how users are engaging with your page. They also show you which areas on the page are not being visited.

Keep in mind that heat maps require a large amount of data to be collected before they can be accurately analyzed. It’s only after extensive data collection can you be sure that your results indicate trends, not one-off anomalies.

Two key activities of the user are measured in heat maps

• Where their eyes are going (eye-tracking)
• What they are clicking on the page (mouse-tracking)

Let’s take a look at each in further detail.

Eye Tracking

Eye-tracking heat maps provide information on where users’ are focusing their attention on your page. As a result, you can take this insight and focus on enhancing and optimizing the area receiving the most attention and scaling back efforts from other areas.   

Here is an example of an eye-tracking heat map study:

The area of red indicates heavy activity and green where users have also gazed (notice the F-shaped pattern)

Eye-tracking may also be in the form of “Saccade pathways” showing the path followed by a user’s gaze on any page.

Click Maps (or Mouse Tracking)

Click maps provide an overview of how users are interacting with your page by way of where they click on your site.

Click maps allow you to identify where users are clicking or attempting to click. For example, you may discover an image or sentence on your site that users are trying to click but you haven’t put a link yet.

Here is an example of a click map.

Since click maps only show clicks on a page, they can inform you that users are clicking on a page somewhere along your conversion funnel, but how they got there or what they will do after is left to uncertainty.   

However, research shows that click maps can be highly inaccurate as compared to eye tracking. One recent paper, for example, shows only a 32% correlation between eye and mouse tracking.

If you can’t afford eye tracking, it’s a good idea to skip mouse tracking altogether.

Do Conversion Rate Optimization the Right way.

Explore helps you make the most out of your CRO efforts through advanced A/B testing, surveys, advanced segmentation and optimised customer journeys.

Test your insights. Run experiments. Win.
Or learn. And then win.

Learn More

And don’t forget to subscribe to our newsletter below!

Like what you’re reading?

Join the informed eCommerce crowd!

Stay connected to what’s hot in eCommerce.
We will never bug you with irrelevant info.

Name

Email

Subscribe to newsletter

By clicking the Button, you confirm that you agree with our Terms and Conditions.