Reflections of a market researcher
Gary de Ocampo —CONTRIBUTED PHOTO
Gary de Ocampo is president of Kantar Philippines and managing director of its insights division. He is retiring today to explore other ventures, including entrepreneurship, after almost three decades in market research. In this interview, he reflects on his experience as a research practitioner, starting as a statistician to being a junior research executive and rising from the ranks to become CEO of a major research group.
Q: Looking back on your career, what are some of the most significant changes or developments have you seen in the field of market research?
When I started in this industry, survey data collection mostly used pen and paper. Some surveys were done via telephone interviewing. Project briefings were always a big event run by researchers and attended by clients. The upside was that there was always a big group of people testing the “fieldability” of the questionnaires and revisions to improve the instrument were done real-time with clients as they heard firsthand possible responses, whose relevance to their business they could immediately assess.
However, despite the scrutiny, interviews would still normally run from as short as one hour to as long as three. The training of field interviewers was of utmost importance because what happened in the field was completely in their hands. The scandalous length of interviews would, however, sometimes tempt a few of these poor interviewers to take things into their own hands and fill out the impossibly long questionnaires under mango trees with made-up answers.
The downsides, as you can imagine, were mostly because, at each stage, there was complete dependence on individuals’ personal discipline, competence, resilience and most importantly, integrity. After completing all the questionnaires, a long and painful process of manual encoding and coding was required before the machines could take over and run the program to churn out data tables, which would then be the basis for charts that researchers would create—also manually—depending on the agreed analysis plan and what the clients required.
Over time, as technology progressed and the environment evolved, data capture moved to computer-assisted interviewing with the use of devices that field interviewers brought along with them from house to house.
Currently, at least in Kantar, there are now more online surveys being done than face to face, the advantage of which is that it effectively addresses perennial issues that came from the use of human interviewers (inconsistent interview conduct, fraud, etc. even though there are more honest and competent interviewers than rogue ones). The con, of course, is the endless debate about how representative the online sample is of the market population that clients are interested in studying.
Data then automatically go through the computer program, which is either precoded if a proprietary solution is used in the research, or customized to address the specific information requirements of the project. Human intervention remains but to a significantly reduced extent. Output is usually delivered via online dashboards, which both the researchers and the clients can access, enabling more perspectives to surface and enriching the final analysis. Use of nonsurvey data such as social media buzz, passive measurement data and secondary data from several sources can now more easily be integrated into the analysis via advanced analytics.
The recent surge in the use of artificial intelligence (AI) appears poised to upend market research further. Will AI eventually replace market researchers? I think that this is possible if we think about those who do the most basic outputs of market research—data tables, charts and first-level analysis. For in-depth and customized reports that require creativity and deep understanding of people, culture and business, humans will always be required. This means then that there will be a need for market researchers to upgrade their skills across a whole slew of disciplines—from psychology to sociology, economics, politics and perhaps even epidemiology, on top of the usual and even more advanced statistics and data science expertise—and bring them all together to come up with genuine understanding of the consumer, delivered to clients in a way that effectively answers business questions and helps them make timely business decisions.
Q: How has technology changed the way we approach market research, and what do you see as the biggest opportunities and challenges in this area moving forward?
With the advent of new technologies—eye-trackers, facial coding, augmented reality, AI and others—the data collection and analysis process has become faster and more efficient. For certain types of research, data collection can even be done in real time! This allows businesses to respond quickly to changes in consumer behavior and market trends. Another opportunity is the ability to collect data from a wide range of sources, such as social media and mobile devices.
However, there are also downsides to all this. For example, for research that requires projection to the actual market, the classic representativeness of samples and the appropriateness of statistical tests being applied may be forgotten.
It is now commonplace to not have any discussion about whether the online survey sample truly represents the consumer segment or market being studied. Often, this matter is dismissed by saying that the source panel of online respondents is anyway “large enough.” Sometimes, little conversation had to make use of some statistical remedies such as applying weights appropriately to correct for any disproportionate representation in the sample. Another atrocity that’s being practiced is the application of significance testing to samples whose characteristics are not even assessed in any way to determine whether the statistical tests being applied are appropriate or not. The other extreme of this is not to bother with significance testing at all, paving the way to needless panic or undeserved celebration at every little decline or increase in a client brand’s key performance metric.
There seems to be no widespread understanding anymore that survey data are most often sample data, not population data. Hence, not everything that happens in the sample is necessarily happening as well in the real market.
The industry still needs to resolve conflicting ideas about how to move forward with the new and shiny while keeping the necessary disciplines of the old.
Initially, the new may dazzle clients but without the required disciplines and expertise that will keep the market research grounded in true science, the hollowness of some of these new toys will eventually be found out, as business recommendations become misleading, unreliable and possibly even detrimental to the clients’ businesses.
Q: Looking ahead, besides technology, what do you think are the biggest challenges facing the market research industry and what will it take to overcome them?
As research companies move toward standardization and using online panels, the challenge of keeping the solid basics will become even bigger. Black-box metrics and normative databases become the selling points without any mention at all about the fundamentals such as sample representativeness, confidence levels, proper articulation of questions, appropriate rating scales and the like.
How do you keep one foot in the genuine science of the work and the other foot in the technological advancements of data collection and delivery? AI is coming and we will see its impact on market research soon. All these will gain strong attraction because they will offer speed and shine, but you will have to understand what you may be giving up when you use them and be clear about the implications of the trade-off.
I think the industry should continue to have an open and honest conversation about these matters until everyone is clear about it, or it is resolved. Most remain in the dark about the debate, and there is an impression that these issues are simply being dismissed and swept under the rug as if old always equates to [being] irrelevant.
—contributed
Kantar is the research and validation partner of the Mansmith Young Market Masters Conference (YMMA) for the last 18 years. The YMMA search is ongoing through www.youngmarketmasters.com. Full text of this interview is available at josiahgo.com.
Josiah Go is chair and chief innovation strategist of Mansmith and Fielders Inc. He chairs the 14th Market Masters Conference featuring 16 of the biggest names in marketing. Registration is ongoing through www.marketmastersconference.com.
