The benefits of a model of annotation

Rebecca J. Passonneau, Bob Carpenter

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (SciVal)

Abstract

This paper presents a case study of a difficult and important categorical annotation task (word sense) to demonstrate a probabilistic annotation model applied to crowdsourced data. It is argued that standard (chance-adjusted) agreement levels are neither necessary nor sufficient to ensure high quality gold standard labels. Compared to conventional agreement measures, application of an annotation model to instances with crowdsourced labels yields higher quality labels at lower cost.

Original languageEnglish (US)
Title of host publicationLAW 2013 and ID 2013 - 7th Linguistic Annotation Workshop and Interoperability with Discourse, Proceedings of the Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages187-195
Number of pages9
ISBN (Electronic)9781937284589
StatePublished - 2020
Event7th Linguistic Annotation and Interoperability with Discourse Workshop, LAW 2013 and ID 2013 - Sofia, Bulgaria
Duration: Aug 8 2013Aug 9 2013

Publication series

NameLAW 2013 and ID 2013 - 7th Linguistic Annotation Workshop and Interoperability with Discourse, Proceedings of the Workshop

Conference

Conference7th Linguistic Annotation and Interoperability with Discourse Workshop, LAW 2013 and ID 2013
Country/TerritoryBulgaria
CitySofia
Period8/8/138/9/13

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'The benefits of a model of annotation'. Together they form a unique fingerprint.

Cite this