‘Who do we FGD? Random, representative sample of consumers?’

Q: We’re a group of graduating students majoring in marketing. Every Monday, our marketing research elective class go over your preceding Friday’s Marketing Rx column. That’s whether or not the topic of that column was marketing research.

We read your column last Friday (August 24). Then the following Monday, we had a heated discussion. This was especially with regard to your answer to the question of “who the IDI [in-depth interview] should interview to find out the motivator for using a product among non-users.”

Our class ended with our marketing research professor asking: “Now, what about an FGD? Who should be your FGD respondents? Remember this. Even if it’s an FGD, your sample of respondents has to be random and representative of the consumers of your target market segment. FGD is still research even if it’s qualitative.”

We all, or almost all, had questions. My group mates think that this is because it’s a non-issue. We all believe that your recommended rule for the IDI respondents should apply here precisely because the FGD is also qualitative. Our professor said that our next class will be on another topic and the answers to our questions should be our assignment. So may we have your opinion about our professor’s idea that in an FGD, the sample respondents should be random and representative?

A: You are basically correct in suggesting that our criterion for selecting the IDI respondents also applies to the selection of the FGD respondents. That criterion is to select the respondents based on whether they are the consumers from whom you will learn the most about what the IDI or the FGD is trying to find out.

But the specific question about the IDI respondent in last Friday’s column was actually different. It wanted to talk to the consumer who can tell you the motivator for using a product that’s targeting non-users.  In the case of the FGD that your professor presented, he was asking if the criteria of representativeness and random selection should apply in qualifying the FGD respondents. There was no purpose of uncovering a consumer motivator attached to this question. So answer your professor’s question directly in terms of the relevance of the representativeness and the random drawing criteria for qualifying respondents.

That your respondents should be representative of the total population under study and randomly drawn is a statistical requirement for large sample quantitative surveys. It’s the basis for claiming that your data is projectable to the total population. That’s about the issue of “external validity.” Your marketing research professor must be a strict statistician to whom validity is solely a matter of obeying the statistical rule of representativeness and random selection.

Because IDI and FGD are not large sample surveys but small sample research of 4, 6, 8 or 10 respondents or groups, their validity is not about projectability to the larger population.

Instead, the concern in a qualitative research is about finding out what you intended to find out. It’s about “measuring what you intended to measure.” That’s what behavioral scientists like up-to-date, knowledgeable market researchers call “internal validity.”

As you can see, your basis for qualifying respondents is now exactly the opposite of the basis for quantitative large sample survey respondents.

So you need to appreciate the research reality that there will be occasions when the demands of external validity and internal validity will clash. In such a case, what do you do? What’s the correct choice to make?

Let’s take a specific research case that we recently experienced. Here’s a client who wanted to find out all the reasons that cigarette smokers have for continuing to smoke. In selecting the respondents of the first two FGDs, we proceeded according to the internal validity rule. We told our FGD respondent recruiters to first bring smokers who are firmly convinced that there are more good things to smoking than bad. These were smokers who have been smoking for 10, 20 or more years. For the second FGD, we instructed recruiters to bring smokers who are new to smoking.  These were smokers who have been smoking for three years or less. They were mostly young adults.

The first group came out with about six reasons. In the second group, smokers mentioned reasons that smokers in the first group already cited and so we did not count them toward the research objective of “getting all the reasons…” But this second group mentioned about two “new” reasons of their own. So the total number of reasons was then eight.

Those first two groups were all males. So for the next two FGDs, we shifted to females but qualified in a similar way. The third FGD of long-term female smokers mentioned reasons already with the first, and one or two cited by the second group. Its “newly” mentioned reasons came to only one raising the total to nine.

The fourth group of young female smokers had really no new reason to add to the total reasons for continuing to smoke. All the reasons that this group mentioned were already in those found with the first, second or third group. So all it took to find out what the research intended to find out, to measure what it intended to measure (that is, to gain internal validity) was a total sample of four FGDs!

Of course, the issue of which of the total of nine reasons is the smokers’ “highest priority” reason awaits a quantitative research confirmation. That’s essentially an external validity issue. But in the foregoing is where the partnership between qualitative and quantitative research must be upheld instead of following the misguided choice of “either quanti or quali.” The better way to state the choice is in terms of “both this and that,” both quali and quanti.

Keep your questions coming. Send them to us at [email protected] or [email protected]. God bless!