It is very tempting to conduct all your research online. After all, it’s fast, cheap, and flexible. It would also be a huge mistake.
This isn’t going to be some rant about the evils of online research. On the contrary, I’m a big proponent of online research and think that the medium has created countless new possibilities for us to better understand markets. That doesn’t mean that online – or any other data collection method – is suitable for every situation.
When deciding on a medium for data collection, you need to weigh several criteria, including:
- Ability to reach the proper market
- Suitability for capturing the data you need
The first question you should ask yourself is “Can I reach the people I need to talk to using this medium?” Take a good look at the characteristics of the people you want to survey and select the data collection method that will reach the best representative sample of these people.
Looking for online shoppers? Web-based surveys are the way to go. Need to reach a very targeted geography? Phone or mail are probably your best bet. Surveying shoppers of a particular store or attendees of a particular event? Intercepts are a reasonable consideration.
Even if a method makes sense based on the nature of the respondents you are looking for, you need to be able to reach them. It’s still easier to obtain lists for members of many niche groups from phone panels than from online panels. This is partly because of the decades long head start people had in compiling phone lists and partly because of the fiercely protective culture that has evolved around email contact.
Many studies can be easily done using several different methods. To narrow down your choices, you need to think a little more deeply about the audience. Ask yourself if there are less obvious characteristics that could impact your data. Is the mindset of someone who spends a lot of time online different from someone who doesn’t? How about people who won’t answer the phone compared to people who will? Absolutely. Does it matter for you particular project? That’s the question you need to answer.
You also need to think about what you will be asking people to do and what, if any, stimuli they will need to react to. If survey respondents need to see something, phone is out of the question. Will they need to watch a commercial or movie clip? See a demonstration? Online is the only practical choice. Will they need to touch or taste something? You’ll need to conduct the interviews in person.
Another consideration is the complexity of the survey. Anything where you need to have numerous paths based on answers or need to be able to reference pervious answers suggests a computer controlled method – online or phone. Lots of scales in the survey? You have more options to reduce the respondent’s pain online than by phone.
Finally, you need to consider the cost and timing implications of alternative methods. In most situations, online will be the cheapest and fastest. It also tends to be the least sensitive to sample size. Phone can be fast, but will be expensive. Mail and intercept approaches are usually both expensive and slow.
I do strongly caution you against placing cost and timing considerations above the other issues discussed here. What does it matter that the research was cheap and fast if the data turns out to be misleading or useless?