Quirk's Blog

Question for research firms: To app or not to app?

Posted on March 7, 2013 by Gloria Park Bartolone

Editor’s note: Gloria Park Bartolone is senior vice president, global fieldwork operations at Maritz Research Inc., Fenton, Mo. This is an edited version of a post that originally appeared here under the title “To app or not to app (with apologies to William Shakespeare).”

Should apps be more prominent in market research and more specifically in customer experience surveys? There are basically two polarized camps when the topic comes up. I think I’ve heard nearly all the arguments.

On the plus side:

• Apps allow you to use the native capability of a smartphone.

• Apps have the best functionality for a smartphone and make surveys easy for customers.

• Apps for market research use are on the increase.

On the minus side:

• Apps for market research use are on the decline.

• Survey apps only make sense when imbedded in a brand’s app or what is referred to as a white-label app.

• No one would ever download an app for a single, event-based survey.

I know: How could apps be on the increase and on the decline? It depends on who you listen to. Several market research companies have chosen to go with mobile app solutions. More troubling are the companies that have focused on one smartphone brand rather than those that comprise the largest market share (Apple, Android and BlackBerry). It’s not hard to understand why they would choose only one. It’s difficult to keep up-to-date with app development considering ongoing device proliferation. Just for starters, each has its own operating system and some have varying screen sizes. But what is the best solution to capture that coveted respondent’s attention?

Well, it depends.

Apps make sense for diary studies and other qualitative work where a respondent or panel member is involved over an extended period of time. It also helps that these individuals are paid for their participation. It even makes sense for mystery shopping, where the same person is engaged in multiple assignments. White-label apps make sense for feedback but push-notification is necessary to proactively engage your customer.

Geolocation fencing (tracking respondents through their mobile device) might be the answer to obtain in-the-moment results but the FTC is recommending that apps obtain “just-in-time” disclosure and consent before allowing access sensitive content like geolocation. It may go further than recommendation to law. Senator Al Franken, D-Minn., proposed a Location Privacy Protection Act which requires mobile apps to obtain permission before collecting and sharing location data. The bill was approved by the Senate Judiciary Committee in January 2013 and goes before the full Senate next. Will this type of heightened concern about being tracked keep respondents out of mobile apps?

What then is the solution for event-based or transactional follow-up that is the realm of most customer experience programs? Do apps have a place here? I’ll go back to the questions above. Why would a person download an app for a one-time survey, or at best, a periodic survey? Furthermore, would limiting your universe only to people who will download an app (or worse, a single operating system-app) truly represent your customer base?

Where do we go from here?

Over the years, many customer satisfaction programs have moved to Web-based methodologies. As a result, an interesting phenomenon has emerged. It’s the rise of the “unintentional” or “accidental” mobile respondent. They are the customer who is sent or given a link to an online survey who chooses to use their smartphone to complete it.

Maritz Research has been tracking this phenomenon over the past few years. Looking back to the beginning of 2011, smartphone survey starts were less than 5 percent of Maritz Web-based studies. Based on the current trend illustrated below, they are forecasted to be over 20 percent by the end of 2013, with tablets in hot pursuit. (The brighter-colored bars represent estimates of future quarters.)

maritzBecause of this trend, Maritz Research has been focused on improving mobile browser rendering. How a survey looks and functions on a mobile device is a reflection of a brand in customer experience programs. Our work takes that into account. We want to extend a brand’s impression in its Web-based survey. We start by addressing the simple issue of mobile sizing and follow through to the other end of the spectrum to mobile optimization. For starters, this includes the best use of a brand’s logo and the presentation of different question types.

Since mobile browsers are approaching the capability of desktop browsers, it is only a matter of time before this technology will make the mobile app obsolete. Forrester Research advocates promoting emerging research methodologies in its What Needs to Happen in Market Research in 2013 white paper. So which one should companies bet on for customer experience programs? I’m placing my chips on mobile browsers.

Posted in Mobile Interviewing, Mystery Shopping, Online Surveys and Research, Research Industry Trends, The Business of Research | Comment

Make 2013 the year of the customer

Posted on February 25, 2013 by Joe Langner

Editor’s note: Joe Langner is executive vice president, midmarket solutions, at Sage North America, an Irvine, Calif., business management software and services firm.

The year 2012 was a whirlwind of influences on the business landscape – some of them good and some of them not so good. The fiscal cliff, the Presidential election, an anemic economic recovery and Hurricane Sandy’s effects on business are a few things that come to mind. Despite the ever-changing business landscape, one central tenet always rings true: keep your customers happy.

So what can we do in 2013 to continue this mantra?

At Sage, our focus is the customer experience. We’ve done a number of things in the last year that I’d like to share with you. While Sage is a large company, many of the lessons we’ve learned can easily be applied to any business – small or large, in services or manufacturing, a brick-and-mortar or e-commerce. Let me share these lessons with you:

Know your customers and adapt to them. We recognized that our customers had grown accustomed to purchasing products and services in their personal lives on a monthly subscription basis (Netflix is a good example of this) with the benefits of a short-term commitment and without the need to spend a lot of money up front.

To that end, we decided to offer our products in a payment method that our customers are already familiar with: as a recurring service based on subscription. Our business management software is now available as a subscription license, meaning no huge investment up front, which makes it easier for businesses to reduce initial capital expenditure to improve their cash flow and simplify budgeting. We knew this would significantly help our customers – especially in light of the recent economic situation.

Put yourself into your customers’ shoes. Perhaps you can offer more flexible payment or billing options that are more convenient to your customer. If you run an e-commerce business, consider extending your customer service hours by one hour in the morning and one in the evening – to accommodate shoppers in different time zones. If you are a distributor, consider a change to your customer communication regarding their sales orders, perhaps via text message or other more immediate modes of communication.

Give your customers more. As a big firm, Sage has the luxury of access to tons of data – something our customers (who are smaller businesses) typically cannot get their hands on. We took the data that we already had, analyzed it and turned it into actionable information that we could provide to our business partners. We are using the data to help our business partners better serve their clients and understand where there might be opportunities for growth. With this free data, they are now able to identify the highest potential sales opportunities, which, ultimately, help grow their business.

For many shoppers, the first thing they do before visiting a new merchant is check their reviews online and then look for discounts. How can you capitalize on this trend? You can offer coupons on your Web site or social media platforms. Consider developing a loyalty program that gives your most frequent customers discounts and other perks. Also think about other resources you could provide your customers. Perhaps you have a friend or employee who is an expert in an adjacent area who can offer free classes at your business. A bakery, for example, could offer free cake-decorating classes or even jam-preserving classes (adjacent, but still relevant). If you have a manufacturing plant, you could offer your customers discounted shipping during the off-season. If you own a consulting service, consider extending a free hour of advisory services for a particular specialty of your firm.

Show your customers that their feedback really matters. At Sage, we’re constantly reviewing our processes to determine the best ways to deliver a memorable customer experience – one of which is acting on customer feedback. One example can be seen with Sage BusinessCare, a service plan with different levels that provides our customers direction and support based upon how they are using our software. Our customers had provided feedback to us, indicating that the lowest level of Sage BusinessCare, the bronze level, was not providing much value to those who subscribed. We listened to them and, as a result, eliminated the bronze level. This is just one example, but you can use this lesson to do the same with your customers. Take “listening” a step further by backing it up with some action. Your customers will remember and value your business more than the competition.

Posted in Business-To-Business Research, Customer Satisfaction, Marketing Best Practices | Comment

Ancient Greeks and the focus group

Posted on February 19, 2013 by Rob Stone

Editor’s note: Rob Stone is CEO of Market Strategies International, a Livonia, Mich., research firm. This is an edited version of a post that originally appeared here under the same title.

I’ve never had anyone wear a toga to one of my focus groups, though a bearded man once walked in wearing a floral-print sarong and introduced himself as “the premier ass-waxer” in the city. He went by the name of Bambi. And, yes, he was a great respondent.

Now that I have your attention, I’ll ask for a bit of patience. There is a connection between togas and focus groups but it’s a meandering path. Last summer, Market Strategies launched a center of excellence for qualitative research. The effort has entailed hiring more than a dozen senior researchers from a variety of backgrounds, including brand consulting, advertising and behavioral sciences. A few of my peers have asked what prompted such a large investment in qualitative methods, especially at a time when so much of the research industry is moving toward “a world without questions,” where we rely on methods like big data, social media monitoring and neuroscience to find answers without ever engaging a respondent in a dialogue. All of those methods have a home at Market Strategies and merit their own investments – but none is about to replace qualitative research.

Our mission is to help clients make confident business decisions – but confidence is a subjective and variable trait. For some people, confidence comes from a robust sample and precise measurements. For many senior decision-makers, however, we continue to find that the voice of the customer – the literal voice of the customer – is irreplaceable. Quantitative research does numerous things that qualitative methods can’t begin to duplicate, but before an exec is willing to place a major bet on a new product or reposition a brand, it’s rare that confidence comes without actually listening to potential customers in real time. Conviction comes most powerfully from the evidence we gather with our own senses and in-person methods, in particular, remain unparalleled for generating conviction.

And that’s where the Greeks come in. Or Aristotle, anyway. In his Rhetoric, Aristotle notes that there are three principal components of any argument:

  • Logos (which is Greek for “word”), which refers to the logical claims of an argument – its clarity, its evidence, its internal logic.
  • Pathos (Greek for “experience”… or, curiously, “suffering” – but I’ll leave that for another blog), which is a speaker’s ability to evoke an emotional response in the audience.
  • Ethos (Greek for “character”), which refers to a speaker’s credibility – how the perception of the speaker’s character affects the perception of the message itself.

 

Survey research is a brilliant tool for isolating the kernel that underlies logos – for stripping away the elements of persuasion that go beyond the pure logical appeal of an argument. But the lived experience of hearing that argument come from a flesh-and-blood person – a person you feel you can take the measure of (ethos) and whose passion you feel as you listen to them convey their own experience through stories (pathos) – carries a rhetorical weight that top-two-box scores struggle to match.

Any seasoned qualitative researcher has seen the way that research stakeholders will continue to refer to notable respondents by name, in some cases years after the research is done. Because of the rhetorical power of human speech – the simultaneous experience of logos, ethos and pathos – it’s no surprise that the highest-stakes projects are the ones that always require video reels. As Aristotle knew, great rhetoric compels conviction. And conviction is the catalyst for business action.

This may not be fair (at least, not in a Vulcan outcomes analysis) and it may not even be uniformly good. We’ve all seen a stakeholder get carried away by one particularly credible respondent who made a brilliantly impassioned argument, even though it contradicted the general trend of the findings. But, used responsibly, qualitative research is a powerful tool for helping us create the conviction that leads to confident business decisions, particularly when experienced in person. Through his thoughtful demeanor and articulate responses, “Bambi” deployed all the classical tools of Aristotelian rhetoric and – despite the sarong – helped move a Fortune 500 company’s strategy forward as a result.

Posted in Focus Groups, Qualitative Research | Comment

Qual & Quant: A love story

Posted on February 14, 2013 by Adam Rossow

Editor’s note: Adam Rossow is partner and head of marketing at iModerate, a Denver research firm. This is an edited version of a post that appeared here under the same title.

In honor of Valentine’s Day, we wanted to celebrate the love of research’s most revered couple – quant and qual. I was recently lucky enough to sit down with this perfect pair and talk with them about how they met, what they love about each other and what makes their relationship so special.

Adam: Guys, thanks so much for taking the time to chat with me today. We all know you two are head over heels for each other. Do you remember when you first knew it was meant to be?

Quant: Of course. I was at a little restaurant outside of Houston. I had just finished up getting data from 500 consumers about this silly restaurant ad. I was annoyed with the results, as everything was still a little unsettled in my mind. And I hate grey areas. Anyway, through the door walks this beauty that looked so confident, so insightful…

Qual: I had just finished up some blue-sky work with teens and I needed a stiff drink. Out of the corner of my eye I saw him. Handsome. And polished from head-to-toe. But he looked a little lost. I was feeling daring, so I pulled up a chair and introduced myself.

Quant: Sure did. And we sat there chatting until they kicked us out. Turned out we were both in the research game. Me, I was into the hard stats. She was more, you know, “touchy feely.”

Qual: You will have to excuse him; he was never good with words. I think you meant to say enlightening and directional, honey.

Quant: Exactly. And that was it; from that moment on we were inseparable. You could say that she “completed me.”

Qual: Oy.

Adam: Great story guys. So Quant, tell me, what do you love most about Qual?

Quant: Easy. She is always helping me discover more and better understand and appreciate the things that I have a tough time seeing on my own. When I’m struggling, she’s always there with a story or anecdote that makes me feel more confident and secure.

Adam: Wonderful. And you Qual?

Qual: Let’s just say that before I met Quant, I really never had my feet on the ground. Sure, I was having fun exploring all that was out there, but I needed some stability, something to rely on. The ambiguity I knew before Quant wasn’t as freeing as one might think. He was the structure I needed and, just as I gave him a different perspective, he did the same for me.

Adam: So what makes your relationship so special?

Quant: Well, it’s going to sound a little black-and-white…

Qual: Shocker.

Quant: …but it’s because we make the best team. Her strengths are my weaknesses and vice versa. Together, I feel like we can solve anything, like there’s no question or issue can’t be answered or tackled.

Qual: Exactly. In the words of one of my favorite authors, Louisa May Alcott, “It takes two flints to make a fire.”

Quant: Yup, it’s 50-50 partnership and the result is a perfect 100.

Adam: So what do you think your relationship has done for research?

Quant: I think the two of us coming together has benefited the industry in many ways. With us both on a study, there’s no missing piece. I provide the stats and Qual provides the story behind them. There’s no wondering what one of my numbers meant or why a consumer said what they said. I don’t have to go back and do more studies and ask more questions. It’s all there, completely comprehensive. Also, on my own, things tend to get a little flat and dry for respondents, but when she is involved, everything is just more engaging.

Qual: That’s so sweet of you to say.

Adam: You guys are awesome. Thanks for taking the time to chat with me. Happy Valentine’s Day!

Quant: Thanks Adam, and for everyone looking for more information about what Qual and I can do for research when teamed up together, here’s a white paper [automatic download] you might enjoy.

Qual: Did you really end our love story interview with a free white paper download offer? I guess nothing is perfect.

Quant: Sorry dear.

Posted in Focus Groups, Market Research Humor, Qualitative Research, Quantitative Research, Uncategorized | Comment

Five ways to fix focus groups

Posted on February 11, 2013 by Tim Coffey

Editor’s note: Tim Coffey is president of VictusVeritas Insights, a Cincinnati research firm. This is an edited version of an article that originally appeared in the firm’s e-newsletter under the same title.

Nobody seems to like focus groups. And yet, this research method is still one of the most popular approaches for many companies. So, why is it that there is a familiar groan of “We just don’t get any real insights from focus groups”?

Is the problem with the moderators? Dull respondents? Poor facilities? Bad food? Too many M&Ms? Let’s get real. There are strengths and weaknesses inherent to the methodology and appropriate and ina158800305ppropriate execution that, together, often lead to less-than-completely-satisfying results.

So what are five ways to fix focus groups, to get better, more insightful results?

1. Focus your focus group. If this seems too simplistic to you, then you are probably a marketer and not a market researcher. I am qualified to poke at you because I’ve been both, so I understand the various pressures and behaviors that both roles must navigate. Having said that, focus is the single most important thing you can do to improve the results you get with focus groups.

What I mean when I say focus is, limit the number of learning objectives to three or four areas. Don’t think of learning objectives as questions but rather, “What do I need to learn from consumers in order to achieve my business goals?” Be strategic. Too often, market research managers and moderators are confronted with a laundry list of specific questions that can mask the real priorities and constrain them from digging deeper or using alternative approaches to get to real learning.

If you do limit your learning objectives, the promise is that the moderator will be able to design a discussion guide that utilizes multiple approaches or activities to get at each objective, which leads to deeper learning and insight against your priorities. I understand that there are lots of questions to be answered and not enough time or money to address them all. So get what you need, versus getting nothing. Everybody is happy.

2. Talk to insightful consumers. Of course, we must recruit consumers who fit the parameters that make sense for the objectives of the groups. But, does it make sense to work with consumers who are dull as dishwater and as introspective as a rock? Truth is, there are people who are much more observant and articulate and representative of where attitudes and behaviors are going versus where they’ve been. One truly insightful consumer can unlock understanding and learning more than a roomful of “representative” consumers. There are ways to recruit for this type of consumer that go beyond the typical “articulation and creativity” questions that are sometimes used in the industry. You can get people who are both consumers and are insightful, interesting and inspiring . . . people you will remember.

3. Quantitative and qualitative. Market research vendors often define themselves as either/or, with focus groups falling squarely into the qualitative. From a technical standpoint, what we are really talking about is the projectability of data or findings from our sample to a broader population. Let’s not go back to statistics class here but suffice to say that focus groups do not produce projectable data, thus they are qualitative. What they should do is to inspire deeper understanding and/or hypotheses and/or models about the potential way people think or behave or react.

The challenge is that sometimes we mix and match quantitative questions into a qualitative process. For instance, if you are asking how many, how much, how often, rank-order, what percent, etc., then you have ventured into questions begging for a quantitative response. This is dangerous territory.

So what do you do? We often combine mini-quantitative surveys with our focus groups so that we get the best of both worlds. This can be a very cost-effective way to maximize the learning you get from your focus groups. Smaller online survey sample sizes can produce representative data – albeit with lower confidence levels –  that help to quantify key questions and keep the focus groups free to do what they do best.

4. Don’t ask direct questions. If you think I am simply being provocative here, then we haven’t worked together yet. The minute you begin to ask consumers direct questions in a focus group is the minute they begin to mislead you. First of all, if you really need the answer to a direct question, see #3 above. Secondly, respondents in focus groups are not likely to share anything with you that could potentially make them look bad or that might be embarrassing. So, they either tell you what they think you want to hear or just make something up (i.e., lie). In either case, you are not getting the truth.

A better approach is to frame questions in a way that allows consumers to project their true feelings without the “cost” of social embarrassment. We do this in many different ways, including: metaphorical construction, visual association, persona creation and more. This takes a little more work before and after but the results are definitely worthwhile, as you can consistently deliver deeper, more insightful, more innovative findings than with ordinary approaches.

5. Demand insights not toplines. It should not be sufficient to accept a topline summary from your research consultant, just because several people were in the backroom taking notes and writing instant summaries for their boss. Focus groups are not a spectator sport.

A great report/presentation takes a little more time, to allow for the processing of all of the discussions and materials. And, most importantly, it takes some high-level thinking to distill and create true insights that address your challenges, to see the unseen. It should be a report/presentation that is inspiring, provocative and worth keeping and sharing. It should make a difference.

We view our reports/presentations as our “product.” We don’t write or produce them on airplanes. Each one is a creative culmination of our obsession to discover meaningful insights that make a difference.

* * *

Follow these five ways to fix focus groups and I guarantee you will not be disappointed. Some of them are easier than others but I assure you that your effort to affect change will be worth it!

Posted in Focus Groups, Market Research Best Practices, Qualitative Research, Research Recruiting | 3 Comments

Gen Y to brands: On Facebook, let us do the talking

Posted on February 6, 2013 by Quirk's Staff

Editor’s note: This entry was adapted from a press release from research firm InSites Consulting.

A research finding from InSites Consulting indicates that a mere 4 percent of all Americans aged 15 to 25 think that a brand page on Facebook is a credible source of information about the product. In other words, such pages are no more credible to youngsters than advertising or than what a competitor would say about the brand.

Does this imply that most companies overinvest in their presence on social media? “I don’t think so,” says Joeri Van den Bergh, a Gen Y analyst at InSites Consulting and author of How Cool Brands Stay Hot. “It is mainly a good indication of the fact that Generation Y is very much aware of a company’s marketing strategy.”

As youngsters attach a lot of importance to the opinion of their friends and of other users of a product or brand, companies should let those groups do the talking. When brands really use their social media socially by allowing feedback and conversations by regular consumers on their pages rather than by filling them themselves, that’s when they really become decent and useful marketing instruments. It is the only medium which allows open dialogue at no great expense. However, many companies keep using their pages too commercially and hope that that’s the way to get youngsters to think the brand is so cool that they will ‘Like’ anything that’s posted on the page.

But that’s not how it’s done, Van den Bergh says. “It’s all about creating compelling content together; stuff that is worth sharing in conversations with your friends.”

Twenty-two percent of U.S. youngsters indicate that what regular consumers write on online forums and blogs is credible, as is what they are told by their friends about a brand or product (14 percent) and the opinion of other brand users (20 percent). This is their top-three of most reliable sources.

Yet, while youngsters prize corporate reliability and honesty and most say they themselves are honest and reliable –  86 percent of U.S. youngsters claim both qualities – one out of every four American youngsters sometimes presents a less-than-complete version of themselves to others. This happens when they are with people who enjoy a higher social status (26 percent), when flirting (17 percent) or when with their in-laws (19 percent). About 25 percent do not shy away from occasionally being less honest with a teacher or boss. And 28 percent of the youth also confess to presenting themselves more positively than reality when looking for a new job.

“That is also an aspect of the Millennial generation,” says Van den Bergh. “Being loyal to yourself is their definition of authenticity. They expect that same honesty from the brands that they think are cool and that they buy. One out of every three U.S. youngsters thinks authenticity is one of the main positive brand characteristics.”

The data come from a global study organized by InSites Consulting among 4,065 respondents aged 15 to 25 (Generation Y) in 16 countries: the U.S., Brazil, Russia, India, China, the U.K., Germany, France, Sweden, Denmark, Poland, Romania, Italy, Spain, the Netherlands and Belgium. The sample is representative for the Gen Y population of each country.

Posted in Brand and Image Research, Marketing Best Practices, Social Media and Marketing Research | 1 Comment

When developing new products, why is it important to start with good ideas?

Posted on January 31, 2013 by Shari Morwood

Editor’s note: Shari Morwood is an innovation process facilitator at Minneapolis-based research firm Ideas To Go. This is an edited version of a post that originally appeared here under the same title.

How important is it to start with a good idea? Companies invest a lot in developing new products and finding ways to talk about them. So coming up with good ideas early in the process really is critical. But can you tell good ideas from bad all the way back at the fuzzy front-end of innovation?

I recently had a conversation near the campus of the University of Colorado at Boulder with Dr. Laura Kornish on this very subject. The value of a good idea is a topic she is very interested in – she actively researches innovation and new products, credibility of information and search problems.

Morwood: How did you get interested in this topic?

Kornish: It started when I met my co-author Karl Ulrich at a conference. Karl is vice dean of innovation and CIBC Professor of Entrepreneurship and e-Commerce at the Wharton School. He has authored several books, including Innovation Tournaments as well as his most recent [free] e-book Design: Creation of Artifacts in Society. He and I – and many other professors – teach product development courses where students generate and develop ideas for new products. I told him that I was hearing a lot of the same ideas in class over and over again: the high-heel shoe with the removable heel; the hands-free toilet; an electronic gadget to help find lost keys. In our first paper together, we measured how much repetition there was in sets of ideas generated by our students. We found that there was some strict repetition but not much. But there was a lot of similarity in clusters of ideas – nuanced or not-so-nuanced variations on a theme.

We wondered what it meant when an idea was generated more often. Did that mean the idea was obvious and maybe not such a good idea? Or did that mean the idea was popular and had a lot of market potential? We used the Darwinator system [an online idea management tool] to gather people’s ratings on the ideas. We found support for the hypothesis that ideas with more similar ideas were also rated higher: those ideas were more popular in both generation and in rating.

The next step was to ask whether those higher ratings really meant anything, in terms of dollars-and-cents in market outcomes. For our current study, we are working with a set of ideas that were actually commercialized, so we can see if there is a connection between how good the initial idea seems and whether or not the product based on that idea sells. A logical place to start with that was to ask product development and marketing professionals to rate the ideas, so we would have an informed opinion about which ideas were good and which were not.

That seems like a great way to go since experts and consultants are hired every day to help companies pick winning ideas. How did things turn out?

Worse than we expected. The experts did not agree and when we don’t have agreement, we don’t expect the results to be predictive. And they were not.

So what did you do next?

We decided to ask consumers what was a good idea. And that turned out to be a very good thing to do. We were able to ask a larger panel of consumers about the ideas – about 100 for each idea – compared to the panel of experts – only seven, which is actually a lot of experts. We see some crowd wisdom: the consumers’ ratings are predictive. We find that the ideas rated more highly by consumers are more likely to be commercially successful in the marketplace.

Our philosophy at Ideas To Go is that it’s important to start with the very best ideas you can to create something that wins in the marketplace. What have you found in your work?

I notice that it’s hard for companies to think about “investment” in idea generation. Ideas seem cheap and easy to come by. In a sense, people have no respect for ideas because they always think “there’s more where those came from.” But we find that the ideas do matter. On average, better ideas will lead to more successful products. So you need discipline in the idea-generation process to come up with really good ideas. The more ideas you have, the better the chance that you will find outstanding ones. Think about it in terms of height: the three tallest people out of a group of 1,000 will probably be taller than the three tallest people out of a group of 100. It’s statistical logic – the bigger the set of ideas you draw from, the better the bests ones will be.

We see people that are very wedded to the ideas they generate and perhaps a little afraid to push out too far. What have you seen in this regard?

Yes, people have a rebuttal to the above stated statistical logic and it’s that the ideas they generate are not random, therefore they believe their first ideas will be the best. But that’s not the case. A big mistake companies make in new product invention is that they don’t explore broadly enough at the beginning. It’s cheap to try out ideas and eliminate them early. It’s costly to do it later. They don’t take the time to develop a range of ideas. Getting good ideas is a process that requires discipline. Falling in love with one of your ideas or pushing through something the boss likes means there isn’t enough exploration around the possibilities.

When you get a lot of ideas, what’s the best next step?

We’ve found the best thing to do is screen them through consumers early. Consumers can help sort the dogs from the winners early. We see that better ideas create better products and that asking consumers is how to tell which ideas are better.

Posted in Brainstorming Research, Concept Research, Product Research | Comment

Tipping the (survey) scales: How to set the multicultural record straight

Posted on January 29, 2013 by Carlos E. Garcia

Editor’s note: Carlos E. Garcia is senior vice president, multicultural research, in the Burbank, Calif., office of research firm GfK.

Marketers want to be able to see into the future. That is, they want to make informed decisions today that will play out nicely in the marketplace tomorrow. A standard approach is to establish trend lines that are founded upon historical data and use current data to direct that arrow from the past into the future. But we have to ask: Is this approach still reliable, given our current environment of dramatic demographic changes and technology-driven paradigm shifts?

My specific area of concern is the rise of the multicultural markets (I’ll leave it to the digerati to take on the tech shifts). The multicultural markets have been an important part of our national economy for some years; but there was, understandably, some lag time between the demographic shifts and this fact sinking into the daily practices of the marketing establishment. It seems now the tipping point has been reached and the multicultural markets are now being included in total market studies.

Fine – accepted and acknowledged. But what about the trend-line foundational research that was conducted in the past? Remember those arrows that start out in the past and head to the future? Are they aimed in the right direction and thus pointed toward a bright future or will they veer off in a suboptimal direction?

Well, we can’t go backwards in time to fix old trackers, A&Us and segmentations. But at the very least marketers can get a clearer picture of the present.

The argument that multicultural populations must be included in numbers proportionate to their presence in the population is clear and hard to refute. The number of Spanish-speakers is clearly large enough to require that this be a part of your sampling plan as well. But is that enough?

To get things right when it comes to today’s unprecedented population shifts requires a probing look at how we do things in marketing and marketing research. Specifically:

  • Are your questionnaires written with multiple cultures in mind or are they simply translated?
  • Are your scales adapted to result in readable results across the cultural groups?
  • Do your attribute statements reflect the lives and lifestyles of the whole market or are they simply the usual suspects from days gone by?
  • Will your analysis be done with enough cultural sensitivity and savvy to find the insights you need to make full use of the data?

 

Coming into meetings fighting for the multicultural markets is tough. No one wants to change their tried-and-true trackers, their familiar set of standard measures, their usual scales, their usual way of doing things. But if the 2012 elections should teach us anything, it is that the usual way of doing things won’t work going forward.

The reality we now face includes a few facts:

  • Different cultures can interpret questions differently – so, while you may be think you know what you are asking, you might be mistaken unless your research team includes professionals with multicultural experience.
  • If you ask what a person of a different culture considers to be an unanswerable question, the data you get back will either be obviously indecipherable or, worse, misleading.
  • Different cultures use scales in different ways, so your goal of data comparability is undermined specifically by mandating use of the same scales across cultures. Techniques exist to handle this problem.
  • Different cultures face different socioeconomic and cultural realities, so the danger of asking condescending or confusing questions is high without the above-mentioned experience.
  • Getting your contemporary research right will help you see the differences between the cultures. This can help you find algorithms that would help you understand how to “back-cast” to adjust the tail of your arrow, so that your arrowhead really is pointed in the right direction.

 

It takes vision, wisdom and leadership to be willing to make the changes that will lead your organization in the right direction, to be able to confront the future armed with precise and nuanced data that reflects our evolving reality. Choose your research approaches and partners well, and you will be more than halfway there.

Posted in Ethnic/Multicultural Research, Market Research Best Practices, Online Surveys and Research, Quantitative Research | 4 Comments

How to craft surveys that maximize data integrity

Posted on January 22, 2013 by Roxanne Salen

Editor’s note: Roxanne Salen is an account executive at Denver research firm The Praxi Group.

When it comes to quantitative data integrity, much of the quality is derived from designing a survey that is engaging and simple for respondents to complete. All too often, in efforts to extract as much information as possible from one survey, data integrity becomes an afterthought. By crafting a respondent-friendly survey, you enhance the quality of the data that will ultimately support major business decisions for your company. Here are a few tips to uphold the quality of data you collect:

Reasonable questionnaire length
Oftentimes the demands of stakeholders drive researchers to try and gather answers to every outstanding business question in one very long survey. However, a long survey is detrimental to respondent engagement and can reduce (or even eliminate) thoughtful responses. Consider breaking the topic into two smaller studies. If that’s not an option, go over the survey with a fine-tooth comb to make sure you’re only asking the most actionable questions for your research team and stakeholders. Or break up the monotony of questions by creating engaging question formats such as a drag-and-drop task or sliding-scale graphics.

Answerable questions
Design questions that allow respondents to accurately recall their experience. When a question is based too heavily on recall, your answers (and data) may be skewed.

Use consumer-friendly language
Remove industry-speak from the questionnaire. Use clear and simple terms that reflect the way your customers think about and use your products.

Give explicit instructions for grid-format questions
Always indicate if the respondent should select an answer from a row or a column, as well as if they should select a single response or all that apply. Again, the more clarity your survey provides, the easier it will be for your respondent to provide an accurate response.

Logical order
Questions should be asked in the order that the respondent would have experienced the interaction or event; ask for the occasion trigger first and then in chronological order to ensure accurate recall of perceptions regarding each component.

Craft questions that are not open to too much interpretation
Consider that each respondent holds unique viewpoints. A question that uses vague language or even one that inadvertently triggers multiple emotional responses could result in an answer that does not meet the original goal of the question crafted by the researcher. Reread your question and ask yourself if a respondent could interpret it differently than you are intending.

Market research lends itself to the support of many company and product decisions. However, the data is only as good as the responses it is comprised of; be mindful of the respondent experience and you’ll maximize your data integrity.

Posted in Consumer Research, Interviewing, Market Research Best Practices, Quantitative Research | Comment

Do we need to rethink market research debriefs?

Posted on January 18, 2013 by Edward Appleton

Editor’s note: Edward Appleton is a client-side senior consumer insights manager based in Europe.

Are debriefs actually detrimental to insights? This is a question grounded in experience, not one designed simply to provoke debate.

A few months ago, I debriefed a multi-country study to a broad group of stakeholders. An interesting discussion was sparked and some actions agreed upon, including a deep-dive on selected topics. Fast-forward to last week’s meeting: a much smaller circle of participants, a narrower focus, fewer issues.

What happened? I was surprised. The room (in this case a virtual one) was full of viewpoints that hadn’t been expressed previously and differing data sets had been prepared that threw light on the same issue from a different angle.

It was a fruitful discussion and one that got me thinking. If the second meeting hadn’t happened, would potential barriers to actionability have been articulated and discussed? Would much, in fact, have remained below the surface?

How often is this true for “the debrief,” I wonder? Is a large, formal meeting the right way to encourage open, frank discussion about something or is it actually like poison to insights?

Here’s my take:

1. Insights thrive in one-on-one situations.

Insights have the potential to create waves organizationally – and require us as professionals to be bold and on occasion outspoken. It’s our remit. The same dynamic isn’t true for many other parts of the organization.

If the insight message is about change of any kind, then one-on-ones are almost certainly likely to be more effective than large meetings, including a research debrief.

This is a resource challenge that we need to rise to.

2. The “before” and “after” are critical moments in sharing and developing insights.

Perhaps we need a flow of smaller, informal meetings, using technology wherever possible to manage cost, rather than just aiming for one formal debrief.

The context of a formal debrief presentation no doubt has its advantages: focus; all key stakeholders present; saves time and money. However, there is to me a downside that this format brings with it: Speaking out in front of many others, sometimes in a language foreign to you, requires confidence, not to say courage.

Sustainable buy-in invariably emerges through collaboration and dialogue – an iterative process.

3. Insights gather weight through a synthesis process.

Piecing together various data sets – some of which may well be contradictory – and coming up with a cogent storyline is something that needs time and invariably works better when more than one perspective is brought to bear.

On that note: when was the last time when you created the actionability slides jointly with your key stakeholders?

However much you may try to do all this testing and reshaping prior to a formal recommendation and presentation, it’s difficult to build in all existing knowledge, especially if senior management is extremely busy. Add to this the fact that we exist in an environment of rapid change. Today’s insight can be challenged by a competitive innovation that looks to disrupt the market.

I don’t think the answer to the above is simply to replace a debrief with a workshop. I think we need to rethink how we go about the process of informing decision makers about insights – more frequently, less formally.

4. Debriefs can encourage conservative thinking.

Larger meetings are not usually great places to find out how tolerant of mistakes or different viewpoints an organization or department is. This can lead to debriefs pitching on the side of caution – avoid errors at all cost; do or say nothing controversial.

Good research to me means doing two things well: minimizing risk and helping identify the upsides. If we err to the former, we do ourselves a disservice. We need to get more comfortable about making bold recommendations with less-than-perfect vision.

5. Relationships nurture insights.

Powerful insights invariably emerge in a dialogue based on mutual respect and trust. If you know someone well, you move in synch with their style, their way of speaking; you interpret quicker, waste less time, trust them more. This is true both for client-side and agency relationships.

Relationship-building isn’t easy. It takes time and requires ongoing effort, not to say investment. It’s something both agencies and client MRs need to actively place a value on, as it’s an intangible that can easily be lost sight of.

* * *

In the hope that this blog is a form of sharing, I’d make a case for going beyond storytelling, visualization, infographics or whatever else we have in our debrief toolkit. Insights become powerful and are more likely to be acted upon through an ongoing dialogue with people you feel comfortable with, often in one-on-ones. One single presentation will seldom be enough. The project may be over formally but business challenges constantly change.

So, do we need to rethink the MR debrief? I’d say yes. Curious, as ever, as to others’ views.

Posted in Market Research Best Practices, Research Industry Trends, The Business of Research | 6 Comments