All About Surveys

Surveys might seem simple, but good data collection requires nuance, rigor, and constant vigilance. This season digs into the science behind asking questions: designing unbiased instruments, avoiding misleading phrasing, and detecting fraud or bot activity in online panels. With insights from practitioners and researchers, it uncovers the hidden complexities behind the polls and studies we often take at face value.

Episodes

  • The NLP Community Metasurvey — Julian Michael, a postdoc at the Center for Data Science, New York University, joins us today. Julian’s conversation with Kyle was centered on the NLP community metasurvey: a survey aimed at understan
  • Skeptical Survey Interpretation — Kyle shares his own perspectives on challenges getting insight from surveys. The discussion ranges from commentary on the market research industry to specific advice for detecting disingenuous or frau
  • The Gallup Poll — Jeff Jones, a Senior Editor at Gallup, joins us today. His conversation with Kyle spanned a range of topics on Gallup’s poll creation process. He discussed how Gallup generates unbiased questionnaires
  • Inclusive Study Group Formation at Scale — Gireeja Ranade, a University of California at Berkeley professor, speaks with us today. She presented her study on implementing inclusive study groups at scale and shared the observed student performa
  • The PhilPapers Survey — Today, we are joined by David Bourget. David is an Associate Professor in Philosophy at Western University in London, Ontario. David is also the co-director of the PhilPapers Foundation and Director o
  • Non-Response Bias — Today’s show focused on an essential part of surveys — missing values. This is typically caused by a low response rate or non-response from respondents. Yajuan Si is a Research Associate Professor at
  • Measuring Trust in Robots with Likert Scales — We are joined by two guests today, Mariah, a Ph.D. student in the CORE Robotics Lab at Georgia Tech, and Matthew Gombolay, the Director of the CORE Robotics Lab. They both discuss practices for measur
  • CAREER Prediction — Ever wondered what your next career would be? Today, Keyon Vafa, a computer science Ph.D. student at Columbia University, joins us to discuss his latest research on developing a machine-learning model
  • The Panel Study of Income Dynamics — Noura Insolera, a Research Investigator with the Panel Study of Income Dynamics (PSID), joins us to share how PSID conducts longitudinal household surveys. She also shared some interesting findings fr
  • Survey Design Working Session — Susan Gerbic joins Kyle to review some of the surveys Data Skeptic has launch, draft a new survey about podcast listening habits, and then review the results of that survey. You can see those results
  • Bot Detection and Dyadic Surveys — The use of social bots to fill out online surveys is becoming prevalent. Today, we speak with Sara Bybee, a postdoctoral research scholar at the University of Utah. Sara shares from her research, how
  • Reproducible ESP Testing — Our guest today is Zoltán Kekecs, a Ph.D. holder in Behavioural Science. Zoltán highlights the problem of low replicability in journal papers and illustrates how researchers can better ensure complete
  • A Survey of Data Science Methodologies — On the show, Iñigo Martinez, a Ph.D. student at the University of Navarra shares his survey results which investigated how data practitioners perform data science projects. He revealed the methodologi
  • Opinion Dynamics Models — On the show today, Dino Carpentras, a post-doctoral researcher at the Computational Social Science group at ETH Zürich joins us to discuss how opinion dynamics models are built and validated. He expla
  • Casual Affective Triggers — Crafting survey questions is one thing but getting your audience to fill it is yet another. On the show today, we speak with Alexander Nolte, an Associate Professor at the University of Tartu. Alexand
  • Conversational Surveys — Traditional surveys have straight-jacket questions to be answered, thus restricting the information that can be gotten. Today, Ziang Xiao, a Postdoc Researcher in the FATE group at Microsoft Research
  • Do Results Generalize for Privacy and Security Surveys — Today, Jenny Tang, a Ph.D. student of societal computing at Carnegie Mellon University discusses her work on the generalization of privacy and security surveys on platforms such as Amazon MTurk and Pr
  • 4 out of 5 Data Scientists Agree — This episode kicks off the new season of the show, **Data Skeptic: Surveys**. Linhda rejoins the show for a conversation with Kyle about her experience taking surveys and what questions she has for th

All About Surveys