Are methodologists becoming irrelevant?

Editor’s note: Paul Richard “Dick” McCullough is president of Macro Consulting, a Scotts Valley, Calif., research firm.

I imagine that, as dinosaurs slowly slipped into extinction, they were the last ones to notice. It might be the same with marketing research methodologists today. Being a methodologist of sorts myself, it took an unfortunate and seemingly unrelated event to bring the possibility of my professional demise into focus.

Recently, the American Marketing Association (AMA), in a rather clumsy way, purged Marketing Research magazine, a gold mine of accessible research methodology literature, of its long-standing editor-in-chief, two columnists and I’m not really sure how many editorial review board members. All this occurred ostensibly due to a disagreement over censorship. What surprised me most was the AMA’s apparent indifference to the opinions of several giants of the methodological community who stepped forward to complain. It seemed to me that AMA brass simply acted like these very important people did not matter.

What did the AMA see that I wasn’t seeing? Initially, I just chalked it up to stupidity and bad manners. However, concurrent with and most probably prior to the purge, the AMA had been preparing to relaunch and reposition Marketing Research, re-titling it Marketing Insights. It may not be coincidental that just before this magazine undergoes a profound positioning change, the old guard have all voluntarily resigned in protest. Also, why would the AMA completely retool a highly acclaimed publication? The only explanation that made sense, other than incompetence, was that AMA execs might believe the future of marketing will not include methodologists or methodological literature.

Why would they think that? Feeling like a grazing brontosaurus who had just been hit on the head with an errant boulder, I raised my thick skull from the grass and scanned the vista. OMG, how the world has changed! :-o, blogs, peer reviews, Twitter, text mining, Web analytics, Facebook, mobile apps, etc. All these new technologies spewing new kinds of data, together cascading into an unending waterfall of unimaginable size, complexity and diversity: big data.

Big data is everywhere. It’s the Gangnam Style of marketing research; the Next Big Thing. Of course, we don’t even know what big data actually is yet. Articles are popping up everywhere trying to define it. But we know it’s here, we know it’s big and we know it will change marketing research forever.

What’s this got to do with the eradication of research methodologists? Let me quote the title of an article by Chris Anderson to give you a hint: “The end of theory: The data deluge makes the scientific method obsolete.” Anderson argues that we no longer need to concern ourselves with causality; correlation will do. We no longer need to hypothesize and test. We have all the data. We can just look. Regardless of the validity of his argument, I do admit big data is reshaping the landscape just like that big asteroid off the Yucatan reshaped the weather.

Big data is not just big. Big data is also different. Old problems, e.g., item collinearity, sparseness, model validity, etc, have been replaced by new problems, e.g., incompatible data sources, vagueness, data management, etc. The old tools need to be replaced by new tools.

But are methodologists becoming irrelevant? Perhaps, in their current form. But regardless of the label placed on analysts, data alone are never enough. David Brooks, in a New York Times op-ed piece “What data can’t do,” writes that big data has many limitations and that human judgment is needed to make big data useful. There still needs to be a human to draw proper conclusions. And the definition of proper is the domain of methodologists.

Guszcza, et al., summarize this point well in “Too big to ignore”: “… analytics initiatives ultimately do not begin with data; they begin with clearly articulated problems to be addressed and opportunities to be pursued. Second, more data does not guarantee better decisions. But the right data – properly analyzed and acted upon – often does.”

Big data is not ushering in a new Cenozoic era, it’s launching a brand new Wild West era. We’re overwhelmed with the three Vs of big data: volume, variety and velocity; to say nothing of the fourth V: vagueness. We don’t yet know what to do or what not to do. So everything is fair game. There are no rules. You’ve got text mining software? Cool! Find out what the world thinks of you. No more sampling. No more sampling error? Not so fast.

The Wild West was a chaotic, dangerous place until the townsfolk all chipped in to pay for a sheriff. Methodologists need to evolve, to be sure. The old techniques will necessarily be replaced with new ones (and old methodologists with younger ones). But the principles of logic and reason that have always been the foundation of methodology still apply. Data do not turn into information on their own. And finding truth – especially relevant, actionable truth – still requires a guiding hand.

This entry was posted in Market Research Best Practices, Quantitative Research, Research Industry Trends, Social Media and Marketing Research, Statistical Analysis. Bookmark the permalink.

4 Responses to Are methodologists becoming irrelevant?

  1. Giliam Winckler says:

    I love this part: “We’re overwhelmed with the three Vs of big data: volume, variety and velocity; to say nothing of the fourth V: vagueness. We don’t yet know what to do or what not to do. So everything is fair game. There are no rules. You’ve got text mining software? Cool! Find out what the world thinks of you. No more sampling. No more sampling error? Not so fast.”

  2. Pingback: Random Sampling: Our Weekly Roundup of Ideas to Grow By | Erickson Research

  3. Shaili Bhatt says:

    Interesting perspective on Big Data! I agree that all of this data will not turn into insights on its own, and there’s still room for the quality found in our evolving market research industry.

    Methods will adapt, flex and change for efficiency, added context, and dare I say, value. The three Vs of big data– volume, variety and velocity–are certainly formidable. All the while, I am pretty confident that keen analysis and business strategy will always be in demand.

  4. Well said. We can assemble data, manage them, describe them – indeed, torture them until they confess! But unless approaches to interpretation are couched (cautiously) by rigorously framed explanations, accounts which extend beyond mere description, we will, as a rule, be none the wiser for our work. That’s where wise guiding hands, hands steadied by experience and reason come into play. We need such hands to not only frame research foci but also to guide methodological frameworks and attendant methods. We need to do this so that others might learn from the inevitable mistakes that will occur as well as learning from whatever small successes may be achieved. That way, future investigators will learn from the things we omitted to do and from the errors we committed as we worked. Big data might then become more relevant.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>