C-SoDA Lecture Series

Google map

"Dealing with Cheating in Online Surveys"

Abstract: In unsupervised online surveys, respondents can easily look up the answers to questions with correct answers, threatening the validity of a wide range of survey measures. Studying the case of political knowledge, I show that this form of "cheating" does not necessarily have a large effect on measured knowledge. Instead, its effect depends on the probability of cheating and the success rate among those who cheat relative to their true level of knowledge. When cheating is a threat, a large majority of it can either be deterred using pledges or detected using page-switching paradata. "Catch" or "trap" questions have some good properties but can also backfire by causing respondents to cheat more on subsequent questions. Based on these findings, I propose a general framework for approaching the problem of cheating in online surveys in any given substantive area.

Bio: Matt Graham is an Assistant Professor of Political Science at Temple University. He studies public opinion and democratic accountability. Much of his research focuses on partisan belief differences, support for democratic values, political knowledge, and how beliefs and attitudes change in response to information. At Temple, Matt teaches quantitative methods courses (a blend of statistics and data science) at the undergraduate and graduate levels.