‘People don’t like to admit that they don’t know something. They don’t like to embarass themselves, either. So, if we start asking questions that trigger these feelings, we shouldn’t be surprised if the answers we get aren’t worth much.
The textbook examples of this problem tell us not to write survey questions that:
- Ask people to do complex calculations or follow intricate processes
- Could expose character flaws or weaknesses when answered
- Probe for sensitive information about money, health, sex, or family
Its obvious enough that if we give people the choice between telling us whether they are dumb or lazy, we’re not going to get the information we want – and we’re not going to make many friends.
There is a much more insideous form of this problem in market research. As marketers, we are always hungry to better understand what goes on the mind of our customers. Often, we forget that our customers don’t care about our products nearly as much as we do.
This problem of asking unanswerable questions shows up time and time again in concept tests of all stripes – products, names, ads. It stems from marketers desire to test many alternative executions. Alternatives that have huge, life-altering consequences for the brand”s position, or perhaps the agency”s contract or the marketing director’s job.
A complete spectrum of versions are carefully crafted to be tested. Once the research comes back, all the arguments about the exact formula to win the hearts and minds of the consumer will be settled. After all, what could be better than laying out several distinct options and seeing what resonates best?
There’s the rub. More often than not, what the marketers see as distinct differences all look the same to most consumers. The new product or campaign is the most important thing the marketing team’s life. To the team, the difference between using Pantone 485 and Pantone 486 for the label background is night and day.
Unfortunately, our great new widget just isn’t that important to consumers. They’re certainly not involved enough to ponder slight differences and craft a considered opinion on the relative merits.
The lesson here is to ask questions that people can answer. Don’t try and tease out minor differences because most of your market doesn”t care enough about your product to see them as differences.
Instead – if you are going to test alternatives – test alternatives. Test “tastes great” against “less filling,” not “world’s fastest car” against “fastest car in the world.” When you ask a question that can be answered, you’ll get good information back.