It is one thing to believe something, and an entirely different animal to put that belief into an articulate argument for that belief. This quote by Vignelli explains the true issues of trusting focus groups and market testing.
“I don’t believe in market research. I don’t believe in marketing the way it’s done in America. The American way of marketing is to answer to the wants of the customer instead of answering to the needs of the customer. The purpose of marketing should be to find needs — not to find wants.
People do not know what they want. They barely know what they need, but they definitely do not know what they want. They’re conditioned by the limited imagination of what is possible. … Most of the time, focus groups are built on the pressure of ignorance.” via BrainPickings
Whenever someone asks me to do a focus group, I usually begin with asking that person what they want the focus group to answer. It is usually quite easy to guess the normal responses. Especially if the product or service is entirely new.
It’s not that the focus group isn’t observant or brilliant, they quite often are. The problem stems from them not having enough time with the product or service to really give us the important information. And sadly we can’t observe a tester for weeks.
Don’t confused with testing for quality assurance purposes, I’ve never seen a project without a few rough corners left, and that sort of testing is essential.