3 Types of Bias You’ll Confront in Consumer Research
As humans, we are, for better or worse, limited by biases and mental heuristics that impact the way we process information and make decisions. In certain situations, these cognitive shortcuts save us from likely pain and destruction. In other settings, they prevent us from tapping into the truth. For marketing leaders, research biases have the latter impact — introducing inaccuracies and confusion into consumer research.
3 Consumer Research Biases That Impact Your Findings
As any experienced marketing executive knows, biases are ever-present in consumer research. It’s unwise and shortsighted to assume that any piece of data is 100% free from the influence of these factors.
However, there are plenty of steps that can be taken to remove obvious biases and end up with objective takeaways that yield tangible value in your marketing endeavors. The first step is to get clear on which biases may be impacting your consumer research. There are a few common ones, including:
1. Confirmation Bias
One of the most pervasive forms of bias in consumer research is the idea of confirmation bias. This occurs when a researcher forms an opinion or belief before conducting research and allows these prejudices to influence the way the study is executed and analyzed.
The problem with confirmation bias is that it’s deep-seated and rarely obvious. Unless we’re actively looking for it, it tends to blend in and go unnoticed.
Let’s say, for example, that your business is investing large sums of money into a new product and you’ve been tasked with leading a research team to analyze customer opinions on the initial product prototypes. You know that the CEO is super excited about the product and everyone really wants it to be successful. Whether you realize it or not, when you go into a product testing phase with actual customers, this desire for success will leach into the way the study is conducted. As a result, the data may indicate a greater receptivity than is actually present. If you’re unaware that confirmation bias exists, then you’re unlikely to catch yourself in the act.
2. Culture Bias
Culture bias is a big deal with international companies that market their products and services to customers in different countries. It’s rooted in ethnocentrism, which is the principle of judging another culture based on the values and standards of one’s own culture. This stands in stark contrast to the principle of cultural relativism, which says it’s more appropriate to view a culture’s beliefs and activities through the lens of that culture.
“To minimize culture bias, researchers must move toward cultural relativism by showing unconditional positive regard and being cognizant of their own cultural assumptions,” researcher Rebecca Sarniak writes. “Complete cultural relativism is never 100% achievable.”
3. Question-Order Bias
For customer surveys, it’s important to be aware of question-order bias. This bias refers to how one question can influence the answers to subsequent questions. Respondents can easily become primed by words and ideas and adjust their responses according to other signals. While sometimes unavoidable, a more strategic and thoughtful structure will lower the risk of question-order bias.
Consider a customer survey that’s being used to gather insights into how your existing customers feel about your business (which is of paramount importance to your future marketing campaigns). While the following two examples may seem similar, they’ll actually produce two very different sets of results:
Survey A
Please rate your satisfaction with the following aspects of our restaurant on a 5-point scale (5=Very Satisfied; 1=Very Dissatisfied).
- Overall Experience
- Speed of Service
- Friendliness of the Wait Staff
- Atmosphere in the Dining Room
- Food Quality
- Menu Selection
- Value
Survey B
Please rate your satisfaction with the following aspects of our restaurant on a 5-point scale (5=Very Satisfied; 1=Very Dissatisfied).
- Speed of Service
- Friendliness of the Wait Staff
- Atmosphere in the Dining Room
- Food Quality
- Menu Selection
- Value
- Overall Experience
The question order in these surveys is almost identical, but with one exception. Customers are asked to rate their “overall experience” at the beginning of Survey A and at the end of Survey B. Neither one is right or wrong, but both impact the data.
In Survey A, customers are asked to think generically and then specifically. There’s a tendency for customers to force the answers from questions two through seven into alignment with the answer from question one. In Survey B, customers are asked to think specifically and then generically. The tendency here is for the answer to number seven to mimic the answers to the previous couple of questions. If five and six receive low grades, then seven will, too.
You might not be able to totally eliminate question-order bias from your consumer research, but it’s imperative that you spend time thinking about it so you’re aware of how it influences your results. Using multiple surveys with different question orders may mitigate the impact on your overall data.
Pursuing Data-Driven Objectivity
Identifying the presence of a bias is one thing. Figuring out a way to eliminate the bias and strip away outside influence is much tougher. Here are some suggestions:
- Be aware of weaknesses. Every research team will be more inclined to fall victim to one bias or another. Being aware of the biases that are most likely to affect your results will help you remain vigilant.
- Hypothesis over expectations. It’s okay to go into a research opportunity with a hypothesis, but it’s unwise to enter in with strong expectations; the latter will get you into trouble. Nix the expectations and let your results guide your thinking.
- Run tests. Sometimes you don’t know you’re exposing your data to a bias until you get some actual results from your research. The best thing you can do is run a handful of tests on the same research topic and compare results. Online surveys are great for this. You can create multiple surveys within the same research study and frame the questions, structure, flow, etc., differently on each. If there’s a high degree of variance between the survey results, this indicates a high presence of research bias. If the results are relatively consistent — despite the unique delivery of each survey — you can feel more confident in your results.
- Involve multiple people. You’re far more inclined to fall victim to research biases when you’re the only one calling the shots. For a more objective approach, involve multiple people. This will lower the risk of a bias going undetected. (Though you’ll have to be careful not to let groupthink characterize your team.)
- Study your data. Finally, be sure to study your data meticulously and cautiously. The results of your research tell a story, and it’s important you don’t jump to conclusions without considering it from every angle.
You’ll never obtain 100% objective results from your market research. There will always be some trace presence of bias, and you can’t do anything about it. However, by eliminating the obvious biases and emphasizing the need for greater subjectivity, you can enhance your results and maximize the value of your market research endeavors.
