Trust it or not: Effects of machine-learningwarnings in helping individuals mitigate misinformation

Haeseung Seo, Aiping Xiong, Dongwon Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

35 Scopus citations


Despite increased interests in the study of fake news, how to aid users' decision in handling suspicious or false information has not been well understood. To obtain a better understanding on the impact of warnings on individuals' fake news decisions, we conducted two online experiments, evaluating the effect of three warnings (i.e., one Fact-Checking and two Machine-Learning based) against a control condition, respectively. Each experiment consisted of three phases examining participants' recognition, detection, and sharing of fake news, respectively. In Experiment 1, relative to the control condition, participants' detection of both fake and real news was better when the Fact-Checking warning but not the two Machine-Learning warnings were presented with fake news. Postsession questionnaire results revealed that participants showed more trust for the Fact-Checking warning. In Experiment 2, we proposed a Machine-Learning-Graph warning that contains the detailed results of machine-learning based detection and removed the source within each news headline to test its impact on individuals' fake news detection with warnings. We did not replicate the effect of the Fact-Checking warning obtained in Experiment 1, but the Machine-Learning-Graph warning increased participants' sensitivity in differentiating fake news from real ones. Although the best performance was obtained with the Machine-Learning-Graph warning, participants trusted it less than the Fact-Checking warning. Therefore, our study results indicate that a transparent machine learning warning is critical to improving individuals' fake news detection but not necessarily increase their trust on the model.

Original languageEnglish (US)
Title of host publicationWebSci 2019 - Proceedings of the 11th ACM Conference on Web Science
PublisherAssociation for Computing Machinery
Number of pages10
ISBN (Electronic)9781450362023
StatePublished - Jun 26 2019
Event11th ACM Conference on Web Science, WebSci 2019 - Boston, United States
Duration: Jun 30 2019Jul 3 2019

Publication series

NameWebSci 2019 - Proceedings of the 11th ACM Conference on Web Science


Conference11th ACM Conference on Web Science, WebSci 2019
Country/TerritoryUnited States

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications


Dive into the research topics of 'Trust it or not: Effects of machine-learningwarnings in helping individuals mitigate misinformation'. Together they form a unique fingerprint.

Cite this