Making sentiment surveys practical

While big data analytics has quickly innovated to reflect the A/B testing revolution in product design and development, survey methods have failed to keep up. As a result, signals that companies can track at scale, such as clicks and time-on-site, have become the most important decision-making inputs for both tactical and strategic decisions. Along the way, the sentiments of the people who use our products have been marginalized. The potential for better, more holistic decision-making that leverages both behavioral and sentiment data is largely unrealized.

To design better products, we need to be able to precisely track what our users think, but also understand more about why they think it. One driving force behind the lack of innovation in sentiment data collection is the belief that it isn’t practical or scalable. Judd Antin presents tested strategies that are both. These strategies build from several key principles:

• Organizations often feel the need to adopt one survey approach and try to adapt it for every decision-making situation. This rarely works. Judd presents a typology of decisions and strategies which survey data can inform and relate it to specific ideas for developing relevant survey programs.
• Judd makes an argument for composite metrics that can roll up into simple metrics that are easy to track but also decompose to help diagnose observed movement in the metric.
• Judd suggests a bottom-up approach to developing surveys, which starts with the words that users themselves use when they describe their feelings about the product. This method moves from qualitative inquiry to pilot scale development to statistical evaluation in order to hone in on the right questions.