BrainJuicer, (no, I didn’t make that name up) presented a methodology based on the wisdom of crowds at the AMA Executive Insights Conference in San Diego recently. They were promoting the approach as a better way to screen concepts than the traditional monadic test. The approach built upon work discussed in Wisdom of Crowds, by James Surowiecki.
The idea is based on old research showing that a random crowd could zero in on the correct answer as well as a group of experts. Ironically, the original experiment was done to prove the point that there was no wisdom in crowds in order to argue against the big mistake of democracy, letting the “unwashed masses”, make decisions. The original experiment involved guessing the weight of an ox that had been slaughtered and dressed for sale as food. The task was to guess the weight of the ox before it was killed. A random crowd of passers-by at a fair collectively guessed within a few pounds, which was just as accurate as a group of farmers and butchers, “experts” who theoretically should have been more knowledgeable than the general public.
Projecting forward to modern marketing, the idea is that any random crowd can screen concepts as well as a carefully chosen (and sometimes hard to reach and expensive) target market.
Faster, cheaper, just as good concept screening.
BrainJuicer has been working with this idea for several years. In fact, they’ve been recognized by ESOMAR for their work. You can download a very well done white paper from their site.
The idea is to ask a random group what they think of several ideas. The question is asked in the context of ideas they would invest in versus ideas that they wouldn’t. BrainJuicer validated their results with parallel monadic tests.
They found that both approaches identified the same top concepts in the same rank order, but the crowd was much better at identifying the “bad” ideas, eliminating the common problem with monadic testing, that mediocre and bad ideas were still rated in the upper-middle portion of the scale by respondents.
They’ve repeated the experiments in several categories, and obtained similar results.
The bottom line?
It appears possible to get reliable, rapid feedback from a general consumer sample to screen concepts intended for niche audiences. Not only does the approach save potentially huge amounts of money compared to recruiting large numbers of niche respondents for a monadic test, it appears to better filter out the weak ideas.
One of the very few times I’ve ever actually seen faster, cheaper, AND better.