We identify new market and product opportunities to help companies grow.
Call us: 312.276.5140 Email: info@ericksonresearch.com

Erickson Research Blog

All posts in Qualitative Research

When It Comes to Your Company, What’s Worth More to You — Any Feedback or Honest Feedback?


Whether you’re working on a new initiative or simply haven’t heard the voice of your customers (or employees) lately — now is the time to check in.

And that’s where in-depth interviews (IDIs) can be invaluable.

IDIs helped our client avoid a near disaster

Recently, we conducted IDIs for a large manufacturer looking to increase sales in emerging industries. We talked to their field salespeople and customers to gather opinions on what the most lucrative industries might be.

Through these conversations, we uncovered something important and unexpected. They revealed a dire need for improved customer service as well as more support for key sales people. This was something the moderator skillfully honed in on by probing particular comments and allowing respondents to feel uninhibited in their feedback.

This would have been nearly impossible to achieve if the company had conducted the interviews themselves since many people expressed they had not felt comfortable talking directly to company employees.

It became clear that crucial needs were not being met. A couple of big accounts were considering changing suppliers. The company had gotten out of touch with their market. And luckily, we brought them right back.

In this case, IDIs were a true life saver. We gave the manufacturer very specific feedback on critical issues they could quickly act upon.

Why use third-party professionals? (READ: Not your internal staff)

  • Respondents need to feel safe and comfortable giving their honest opinion. They’re much more likely to be open with an objective outsider who won’t take negative feedback personally. Or — in the case of employees – doesn’t influence their job security.
  • A skilled moderator can relate to all types of people in a likeable, genuine way. That means adjusting their approach and vernacular to fit the respondent and subject matter — whether it’s Jane the full time mom, Ron the factory foreman or Dave the C-level executive.
  • Special care is required for people who are frustrated and just want to vent as well as those who are excessively chatty. A seasoned interviewer keeps respondents on topic and extracts valuable information, yet doesn’t stifle their need for expression. This is an acquired skill that can only come from conducting a lot of interviews with a lot of people.

Without question, a worthwhile investment

Yes, bringing in outside consultants is more costly than having your intern call a few customers. But in reality it’s a bargain… if compared to the sales lost when a big customer drops you– or the sleep you’ll lose when senior management wants to know why you didn’t see it coming.

Social media mining can replace focus groups

Perhaps the most written about topic in research these days is social media.  It’s hard to look at a conference agenda, industry magazine, or blog without seeing someone’s opinion about using social media in research.  What has been in much shorter supply is any sort of evidence of social media’s usefulness – or lack thereof – as a research tool.

I confess, I’m just as guilty as anyone when it comes to pontificating about social media in research.  It’s been the subject of some of my most popular blog posts.

A big part of the debate has been whether and how social media mining might replace other qualitative methods like focus groups.  After all, at first glance, an online community looks a lot like the ultimate focus group.

But is it?

Being researchers, we thought the obvious way to find out if social media mining could be used like focus groups was to conduct an experiment. So that’s what we did.

Earlier this year, we did parallel studies to compare the information we could get from a traditional focus group to what could be found mining data from online forums. We conducted 3 focus groups in Chicago.  We asked people to tell us about their relationships with and opinions about their banks and credit card companies.  In parallel to the focus groups, we selected 4 online personal finance forums to review.

What we found

There was a great deal of consistency between what was heard in the live focus groups and what we read on the Internet forums.  The primary conclusions that any researcher would draw from each approach were the same.  The alignment wasn’t perfect – but good enough to make social media mining a viable alternative to focus groups in this scenario.

Our main takeaways were the same, but we did see some differences among minor points.  Many of these difference simply reflect natural differences in the make-up of the “participants.”  In fact, the differences between the groups and the forums weren’t noticeably bigger than differences between the individual focus groups.

Some differences resulted from methodological differences between the two approaches.  The two biggest methodological differences were:

  1. Interaction. We interacted with the participants in the focus groups, but not on the forums. So, we could steer the conversation in groups with probes, drawing out more detail in some cases.
  2. Motivation.  People came to our groups because we invited them and we paid them (just like every other focus group).  People came to the forums because they had something to say or were seeking information.

What does this mean?

It means that, in some cases, social media mining can be an effective substitute for traditional focus groups.  This test project has shown social media mining will get essentially the same information as focus groups, but faster and more cost effectively.

I say social media mining can work “in some cases” because, like every other research methodology, it has to be applied intelligently.  Using social media mining instead of focus groups can work in situations where you are trying to get a general feel for the nature and range of opinion in the market, as we were in this scenario.

Social media mining seems to be best suited to exploratory research. This is the type of qualitative you might do at the beginning of a big qual/quant project where the goal of the qualitative is to sketch the broad outlines of the situation, frame the issues, or suggest hypotheses for quantitative testing.

This approach isn’t going to work for explanatory research. Qualitative projects where the primary goal is to understand “why” requires a two-way dialog so the interviewer can probe answers.   This is more often the type of qualitative you might do on its own or as a follow up to quantitative to understand context.

The end of the world as we know it?

So, should you stop hiring qualitative researchers and simply subscribe to one of the social media mining tools out there?

That would be a mistake. There isn’t an automated tool that can do this work.

The mining tools out there can help to identify forums where relevant discussions are happening, but you need a real live qualitative researcher reviewing, assembling, and interpreting what’s there.  The value is in the analysis, and if the person doing it isn’t experienced, well, garbage in, garbage out.

Is that it?

There’s a whole lot more information available about this test, which will be in a white paper that will be published on this site shortly.

How does social media mining compare to focus groups?

On Wednesday, May 26th, I’ll be one of the speakers at Peanut Labs‘ webinar on qualitative research using social media.  I’ll be publicly unveiling some exciting research we’ve done comparing traditional focus groups to social media mining.

Lots of people, including me, have been up on our soapboxes for a while now spouting off on all sides of the social media in research debate.  The problem with the debate so far is that it’s been long on opinion and short on evidence.

That’s about to change.

Thinking about how to use social media in your research?  Register for the webinar here.

My portion of the webinar is only 7 minutes, so this will be a high level overview of the results.  I’ll post a more detailed description on this blog next week.  You can get the RSS feed here.

Page 2 of 3123