Data Skeptic Takes on Surveys and Market Research

Our new season, Data Skeptic: Surveys began in early January. When Sir Francis Galton published the results of a weight-guessing contest, he started a deep and rich area of statistics. If guessing the weight of a cow or the number of jelly beans in a jar, you're best guess is to average all the guesses. Wisdom is often found in the crowd, but on other occasions the crowd is puzzlingly bereft of wisdom. Part of our coverage will explore both cases.

The most notable way to measure the voice of the people is through public opinion polling. More specifically, through surveys. Surveys can be a powerful measurement tool. Many organizations make major decisions based on survey results. Yet, there are also many things to be skeptical about surveys. Improper methodology, bad survey design, bad sampling, and other mistakes both intentional and unintentional can often completely invalidate any conclusions drawn from a survey. Specific use cases of surveys, both successes and cautionary tales, are rich topics from which we'll share interviews

Many people are skeptical of survey results, and for good reason. Under this topic, we have the opportunity to explore statistical topics such as sampling, bias, stratification, confidence intervals, and other topics. These methodologies are applied differently in the world of big data in which solutions like the bonferoni correction and false discovery rates are more useful for large scale inference.

The process by which surveys are conducted can be a complex one. Many novel technologies have been developed for conducting surveys, and we plan to explore engineering stories about how survey platforms operate. No doubt interesting topics about data privacy, look-a-like modeling, and survey accuracy will find there way into this new season.

Alongside our typical content, this season will also introduce a new segment in which we share polling results each week summarizing the responses to our ongoing polls that we plan to conduct for the extend of the season.

As I write this we're in the midst of researching the topic. I'm incredibly excited to be sharing this new season with the world in 2023 and I can't wait to find my own way of measuring what the listeners think about it.