QUESTION: We were brand managers when we attended your 1991 AMR (Applied Marketing Research) seminar. You simplified marketing research from both the user and doer sides so much that we were convinced we could set up our own small research agency. That way, we would become business owners instead of mere employees.
We wanted to make sure that we would have enough research projects to at least break even in the first year. So we convinced our employer to give us just half of her research projects for the year. Most of the projects we got were qualitative research, both FGDs (Focus group discussions) and IDIs (In-depth interviews).
Our most bothersome problem has to do with respondents. We followed your advice then that we shouldn’t follow the quantitative research rule of getting “representative and randomly selected respondents.” You told us to make sure that we get FGD or IDI respondents “who can talk.” This worked well and also not so well.
We understand that you have a new and better system of respondent recruiting for qualitative research. You’ve always told us that you like sharing new research techniques with your students. Please share.
Answer: Frankly speaking, it was unnecessary to sugarcoat your request. Many MRx column readers who ask questions were not my students and I still answered.
For your question, let’s start from what is still true and correct. In qualitative research, do not apply the respondent selection norm of the quantitative survey. That norm places its priority on external validity. This validity is about the survey data whose story is true of the larger population from where the sample of respondents were drawn to provide the data. This validity requirement must satisfy two rules. One is to have a representative list of the total population members known as “sampling frame.” The other is drawing a statistically random sample of respondents from the list and who would provide the survey data. This was the validity requirement that my recently presented nationwide consumer coping survey was designed and conducted to satisfy.
In a qualitative research, FGD or IDI, the priority validity concern is with the issue of whether you are researching what you intended to research, whether you are measuring what you intended to measure. This issue is about “internal validity.” Of course, internal validity is also of importance to a quantitative research but it is not its priority. In qualitative research, it is the priority.
Let’s speak in the concrete. During the industry-wide presentation of the consumer coping survey last May 8, 2015, one survey finding that most of the attendees questioned was about the top staple product category, the “product I cannot live, I cannot do without.” In NCR, for example, the top staples were bath soap, toothpaste and detergent. Rice was also a staple but came after these 3. The audience asked: “Why? How come?”
The answer could have come from probing this finding. But the survey could not probe. It covered 151 product and service categories. To complete the data collection in 20 minutes or less, the survey innovated in its questionnaire formulation by asking only one closed-ended question, and inno vated again in the data analysis for uncovering the business-growing opportunities per product category.
When the question was asked why or how come detergent, for example, was the NCR consumers’ top #1 staple product, I answered that we needed to IDI the right respondent to understand why. And who is this correct IDI respondent? It’s the respondents who said they cannot do without a detergent. It’s not the respondent “who can talk” or the respondent that IDI or FGD recruiters are used to recruiting.
Why is the right respondent the one who answered in the quantitative coping survey that for her she can’t do without a detergent? Because that’s the respondent from whom you can learn the most about why detergent is an absolute necessity. This insight can only come from the actual source and owner of that insight.
What’s the scientific bases of this respondent definition? There are two. The first is in the practice of clinical psychology. To understand depression among adults, Freud and his followers did not research a representative random sample of adults. Instead, they talked to the depressed, the respondent from whom they can learn the most about depression. This is not to say that there’s nothing to learn from those who were not depressed. The question was: “From whom can we learn the most?”
The second scientific basis of my proposed qualitative respondent definition is in the meaning of internal validity. In pursuing internal validity, we are after researching what we intended to research. That’s to understand, for example, why detergent is a staple or a product consumers can’t live without. Who can tell us why? That’s no other than that respondent who can tell us why or from whom we’ll learn the most.
Did we ever find out why, for example, detergent is a product consumers can’t do without? In the 2008 coping survey, one sponsor provided the needed small budget for the IDI of just six qualified respondents. All six had similar stories as typified by this Class D respondent who said: “Hirap na nga ang buhay at kung di pa kami maglalaba ng isusuot namin, di lalo na kaming maghihirap pag mabaho pa ang mga suot namin.” (Times are already hard. If we don’t wash our clothes, life would be even harder because of dirty and stinking clothes.) So, to give the correct answer, we will understand from the respondent from whom we can learn the most about the “why” question.
Keep your questions coming. Send them to me at ned.roberto@gmail.com.