Online hate ratings vary by extremes: A statistical analysis

Joni Salminen, Hind Almerekhi, Ahmed Mohamed Kamel, Soon Gyo Jung, Bernard J. Jansen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Scopus citations

Abstract

Analyzing 5,665 crowd ratings on 1,133 social media comments, we find that individuals tend to agree on the extremes of a hate rating scale more than in the middle when evaluating the hatefulness of online comments. The agreement is higher for less hateful comments and lowest on moderately hateful comments. The results have implications for researchers developing machine learning models for online hate processing, as the extreme classes are likely to require fewer annotations for reaching statistical stability. Our findings suggest that the models developed in this domain should consider the distributions of hate ratings rather than average hate scores.

Original languageEnglish (US)
Title of host publicationCHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages213-217
Number of pages5
ISBN (Electronic)9781450360258
DOIs
StatePublished - Mar 8 2019
Event4th ACM SIGIR Conference on Information Interaction and Retrieval, CHIIR 2019 - Glasgow, United Kingdom
Duration: Mar 10 2019Mar 14 2019

Publication series

NameCHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval

Conference

Conference4th ACM SIGIR Conference on Information Interaction and Retrieval, CHIIR 2019
Country/TerritoryUnited Kingdom
CityGlasgow
Period3/10/193/14/19

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Information Systems

Fingerprint

Dive into the research topics of 'Online hate ratings vary by extremes: A statistical analysis'. Together they form a unique fingerprint.

Cite this