Quantitative research is appealing because it forces researchers to think in measurable terms. Just saying things like, “people tend to…” or “there’s support for…” are highly scrutinized because statements that start like this either (1) deliberately omit quantitative data or (2) do not have strong statistical backing.
But when researchers are accustomed to giving quantitative answers (and clients are accustomed to hearing them), it becomes difficult to identify when questions are fundamentally qualitative. Here are some examples of (paraphrased) objectives I’ve observed in real RFP’s that were ultimately addressed with quantitative methods that would have been better addressed qualitatively.
- “Investigate the issues that matter most to stakeholders…”
- “Dimensionalize what [CONCEPT] means to various stakeholders…”
- “Understand what motivates [GROUP] allies to support [GROUP CAUSE]…”
Of course there are other considerations when determining the optimal research approach; so, I can’t necessarily say that a quantitative approach was wrong. The best research approach is a decision that needs to weigh several factors. Nevertheless, I think analyzing real RFP objectives is helpful.
Qualitative Myths and Aversions
Here, I describe some myths and aversions clients and researchers have towards qualitative research.
Myth 1: Qualitative Research is Touchy Feely
Please describe to me how a market comes into equilibrium using statistics. Ok, that’s a hard one.
Please describe how a hurricane forms using statistics. Not a meteorologist?
Why are some Super Bowl ads more popular than others? Ok, everyone has an opinion here. But this is still impossible to answer using statistics.
These are all questions that I wouldn’t describe as “touchy feely,” but they are all qualitative in nature.
As scientists, we all deploy a process of thinking about how the world works. Describing how a process works is necessarily a qualitative exercise.
Myth 2: Qualitative Research is Subjective
This is a common misconception which does a great disservice to practitioners of qualitative research. Accusing a researcher of subjectivity is akin to an accusation of bias — it undermines the legitimacy of qualitative methods.
Although the data collected may itself be subjective, the collection, analysis, and reporting of it is methodical and descriptive. Qualitative researchers are not giving their opinions of the data collected.
Myth 3: Use Qualitative Research when the Quant Budget is Limited
Often researchers and/or clients will deploy qualitative methods when they do not have the budget for quantitative sample sizes.
I’ve seen this a few times where clients settle for qualitative work. The irony is that the few cases I’ve seen this happen, I think the resulting research was successful. But not because qualitative is a substitute for quantitative, rather, because the original project was qualitative in nature.
Regardless, if you intend to use quantitative methods to measure the size of an effect, compute the incidence of a population within a larger one, or determine the more impactful ad (among two similar ads), then qualitative simply won’t give you the information your looking for.
I find that clients and researchers alike are averse to qualitative research for several reasons.
Qualitative Aversion 1: Quantitative Complacency
Research buyers are accustomed to contracting work with suppliers when there is a need for measurement or calibration. At the center of the research is typically a decision that needs to be made “at the margin.” By this I mean, for example, “which of two similar ads should we run?”, or “how should investment be allocated across two stakeholder groups with similar impacts on a company’s reputation?” These are marginal questions that need statistical answers.
But somewhere in the research buyer-supplier relationship, the parties became complacent and just continued to use quantitative solutions for qualitative questions (like the ones described above). In short, buyers and suppliers simply expect quantitative methods to be the right method. The comfort and inertia apparently bias researchers to favor quantitative methods.
Qualitative Aversion 2: Need for Statistics
As discussed above, research is often called upon when a “marginal” decision is needed. This, I believe, has led to the expectation for research to provide operational decision making tools.
Typically, when somebody asks for outside help in making a decision, they are on the fence. But qualitative research simply does not provide the quantitative support when somebody is on the fence about a decision.
Qualitative Aversion 3: Hubris
Many believe that qualitative research won’t tell the client/researcher anything that they don’t already know. “Why should I do a focus group with industry experts when I am the industry?”
Clients/researcher often think they already have the answers — they already know the possible hypotheses — and all they need is measurement and confirmation.
(But if they were to really analyze their RFPs, they might rethink how well they know the issues key to the research).
Qualitative Aversion 4: Lack of Scalability
Leaders in research firms deemphasize investments in qualitative research solutions because it is so human labor intensive. It simply does not make sense to spend much time focused on research solutions that do not have strong returns to scale.
This can lead to research consultants being poorly incentived to offer qualitative solutions even if the qualitative research is the optimal approach.
Resources
Here are a few resources I’ve found helpful concerning qualitative research. Still, there’s a lot of false information out there. I saw a Udemy blog post suggesting that qualitative research was “subjective,” which is not true. So, caveat emptor.
Yale University’s Dr. Leslie Curry (video):
Qualitative methods can generate a comprehensive description of processes, mechanisms or settings.
QRCA:
-Develop hypotheses for further testing and for qualitative questionnaire development
-Understand the feelings, values, and perceptions that underlie and influence behavior
-Identify customer needs
-Capture the language and imagery customers use to describe and relate to a product, service, brand, etc.
-Perceptions of marketing/communication messages
-Information obtained in quantitative study and to better understand the context/meaning of the data
-Generate ideas for improvements and/or extensions of a product, line, or brand
-Uncover potential strategic directions for branding or communications programs
-Understand how people perceive a marketing message or communication piece
-Develop parameters (i.e., relevant questions, range of responses) for a quantitative study
There are several resources out there, but be careful because there is a lot of mis-information.
Other (Specify) 