TY - GEN
T1 - A reverse turing test for detecting machine-made texts
AU - Shao, Jialin
AU - Uchendu, Adaku
AU - Lee, Dongwon
PY - 2019/6/26
Y1 - 2019/6/26
N2 - As AI technologies rapidly advance, the artifacts created by machines will become prevalent. As recent incidents by the Deepfake illustrate, then, being able to differentiate man-made vs. machinemade artifacts, especially in social media space, becomes more important. In this preliminary work, in this regard, we formulate such a classification task as the Reverse Turing Test (RTT) and investigate on the contemporary status to be able to classify man-made vs. machine-made texts. Studying real-life machine-made texts in three domains of financial earning reports, research articles, and chatbot dialogues, we found that the classification of man-made vs. machine-made texts can be done at least as accurate as 0.84 in F1 score. We also found some differences between man-made and machine-made in sentiment, readability, and textual features, which can help differentiate them.
AB - As AI technologies rapidly advance, the artifacts created by machines will become prevalent. As recent incidents by the Deepfake illustrate, then, being able to differentiate man-made vs. machinemade artifacts, especially in social media space, becomes more important. In this preliminary work, in this regard, we formulate such a classification task as the Reverse Turing Test (RTT) and investigate on the contemporary status to be able to classify man-made vs. machine-made texts. Studying real-life machine-made texts in three domains of financial earning reports, research articles, and chatbot dialogues, we found that the classification of man-made vs. machine-made texts can be done at least as accurate as 0.84 in F1 score. We also found some differences between man-made and machine-made in sentiment, readability, and textual features, which can help differentiate them.
UR - http://www.scopus.com/inward/record.url?scp=85069537165&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85069537165&partnerID=8YFLogxK
U2 - 10.1145/3292522.3326042
DO - 10.1145/3292522.3326042
M3 - Conference contribution
T3 - WebSci 2019 - Proceedings of the 11th ACM Conference on Web Science
SP - 275
EP - 279
BT - WebSci 2019 - Proceedings of the 11th ACM Conference on Web Science
PB - Association for Computing Machinery, Inc
T2 - 11th ACM Conference on Web Science, WebSci 2019
Y2 - 30 June 2019 through 3 July 2019
ER -