Microworkers as research participants: Does underpaying Turkers lead to cognitive dissonance?

Bingjie Liu, S. Shyam Sundar

Research output: Contribution to journalArticlepeer-review

15 Scopus citations


Social science researchers increasingly rely on microworkers to serve as study participants, paying them very little compared to participants recruited from other venues. This has raised ethical concerns and questioned the validity of research based on microworkers. Informed by cognitive dissonance theory, we conducted two between-subjects experiments to examine the effects of underpaying Amazon Mechanical Turk workers (Turkers) on their perceptions and their actual performance on criteria crucial to online social science research. Data show that underpaid Turkers experienced ‘cognitive dissonance’ such that those paid as low as $0.25 rated their participation as more important than those who were paid higher. Perceived importance was associated with other positive perceptions and demand characteristics. Nevertheless, underpaying Turkers increased dropout rate, reduced their level of effort in answering open-ended questions and undermined perceived agency. We discuss the ethical and practical implications of underpaying microworkers.

Original languageEnglish (US)
Pages (from-to)61-69
Number of pages9
JournalComputers in Human Behavior
StatePublished - Nov 2018

All Science Journal Classification (ASJC) codes

  • Arts and Humanities (miscellaneous)
  • Human-Computer Interaction
  • General Psychology


Dive into the research topics of 'Microworkers as research participants: Does underpaying Turkers lead to cognitive dissonance?'. Together they form a unique fingerprint.

Cite this