Microworkers as research participants: Does underpaying Turkers lead to cognitive dissonance?

Social science researchers increasingly rely on microworkers to serve as study participants, paying them very little compared to participants recruited from other venues. This has raised ethical concerns and questioned the validity of research based on microworkers. Informed by cognitive dissonance theory, we conducted two between-subjects experiments to examine the effects of underpaying Amazon Mechanical Turk workers (Turkers) on their perceptions and their actual performance on criteria crucial to online social science research. Data show that underpaid Turkers experienced ‘cognitive dissonance’ such that those paid as low as $0.25 rated their participation as more important than those who were paid higher. Perceived importance was associated with other positive perceptions and demand characteristics. Nevertheless, underpaying Turkers increased dropout rate, reduced their level of effort in answering open-ended questions and undermined perceived agency. We discuss the ethical and practical implications of underpaying microworkers.