Pros and Cons of Website Analysis Methodologies

Woohoo! Your Director of Marketing finally gave you permission to put together a conversion rate optimization strategy (it’s about time—especially when you consider that companies using a CRO tool to develop strategies see a 233 percent increase in ROI). The understanding is that if you present valid experiments and opportunities for improvement, these optimizations can be your highest priority.

But before you have time to get too excited, the first obstacle appears. You know deep in your digital marketing soul that your website could be more of an asset to your company, but you don’t have the details needed to put together a data-driven plan of attack. And, unfortunately, that plan of attack is exactly what you need.

In order to put together a list of impactful experiments, you must first start with a website audit and analysis. And because there are so many different analysis methodologies available, that shouldn’t be hard, right?

Understanding the Methodologies

That’s right. Analyzing your website shouldn’t be—and won’t be—hard. The key is to determine which of the many methodologies you’ll use to understand not only how your website currently performs, but also where your opportunities for improvement lie.

However, before you start with what might be the most simple, accessible, or even comprehensive methodology, I want to be sure that you understand the pros and cons of each.

For simplicity’s sake, we’ll break out these methodologies by their analytical category: qualitative, quantitative, or both.

Qualitative Methodologies

These methodologies do not demonstrate measurable, tangible results or benchmarks. They will, however, provide you with context, which you can use to understand how your website visitors consciously and subconsciously react to various elements on your website. These methodologies include:

• Remote user testing
• In-person user testing

Although both of these methodologies are variants of one general methodology (user testing), there are slight differences between the two, namely whether or not the user testing takes place in person or virtually. In terms of the pros, using the right questions and further probing the respondents’ answers can provide you with great, detailed insights. However, there are two major downsides to these types of methodologies.

The first has to do with cognitive bias. If your respondents’ true motivation stems from anything less than selflessness and altruism (and let’s be honest, many purchases are made to demonstrate status or power, or to accomplish a less than honorable goal), chances are, respondents will be reluctant to share their true motivations with you. So, although you may get great, detailed answers, chances are that these responses are not 100 percent honest.

The other downside of user testing (either in person or remote) comes from its capabilities. At its core, user testing provides the customers’ voice to you. Even if you are granted completely honest, transparent answers, you are only receiving insights into your respondents’ conscious decision-making process—and unfortunately you’re not even receiving all of their insights; you’re only hearing the ones that are memorable or top of mind. But only 5 percent of our decision-making takes place in our conscious mind. The rest is determined by totally subconscious responses to stimuli. When conducting user testing, these subconscious factors remain unknown.

Quantitative Methodologies

Opposite of the qualitative methodologies, these methodologies show the results of the stimuli presented on your website, but they don’t show the reason for the results. You can see that visitors do something on your website pages, but you cannot necessarily determine why. Some commonly used quantitative methodologies include:

• Analytics
• Eye tracking
• Heat mapping

As long as the sample size is large enough to be representative, these methodologies present you with highly valid results that show how visitors respond to both conscious and subconscious stimuli. However, , you receive no insights into your visitors’ reality. You must instead infer why something happens. Why are so many visitors’ eyes drawn to one call to action over the others? Why do only 42 percent of visitors make it to a certain section on the page? Why is the bounce rate of one page so much higher than others?

Just because you know that something is a certain way, doesn’t necessarily mean that you know how to improve it. This is why you might want to use methodologies that can be both qualitative and quantitative.

Combination Methodologies

These methodologies are unique in that they are able to provide context and insights regarding why visitors behave a certain way, but they are also able to share measurable data. These methodologies include:

• Online surveys
• Expert evaluation

Now, although each can provide both types of insights, they have different advantages and disadvantages. Online surveys, for example, can be hugely valid (with enough surveys conducted in order to be significant), however you run into the same issue as with user testing: You only receive insights into respondents’ conscious decision-making process.

An expert evaluation can be used to provide insights into visitors’ conscious and subconscious behaviors, but its success is hugely dependent on one key factor: How capable are you, as the expert, of adapting to your personas’ mindset and decision-making processes? Doing so is far easier said than done, so it’s fair to say that relying solely on your own evaluation might provide interesting inferences, but they may not be valid. And unfortunately, there is no way to know for certain.

How to Analyze Your Website

With all types of methodologies having specific advantages and disadvantages, what are you supposed to do? How are you supposed to put together a comprehensive website analysis that is both valid and representative of users’ thoughts during the transaction or website visit?

Most likely, you’ll need to use multiple methodologies. After all, you ultimately want to be able to understand user behavior, reactions, and thoughts enough that you can—with your understanding of best practices, general user behavior, and a buyer psychology—conduct an expert evaluation that is reliable. In order to accomplish this, you’ll need to use a combination of methodologies that will:

• Provide valid insights.
• Answer why visitors aren’t doing what is expected of them.
• Focus on visitors’ conscious and subconscious behaviors.
• Produce results quickly and effectively—you want these methodologies to be agile and require little effort on your part.

Once you understand why visitors are behaving the way that they are, you’ll be able to develop experiments that can change user behavior and responses. After all, that is the true goal of conversion rate optimization.