TY - JOUR
T1 - Crowdsourcing the measurement of interstate conflict
AU - D'Orazio, Vito
AU - Kenwick, Michael
AU - Lane, Matthew
AU - Palmer, Glenn
AU - Reitter, David
N1 - Publisher Copyright:
©2016 D'Orazio et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
PY - 2016/6
Y1 - 2016/6
N2 - Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.
AB - Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.
UR - http://www.scopus.com/inward/record.url?scp=84976271826&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84976271826&partnerID=8YFLogxK
U2 - 10.1371/journal.pone.0156527
DO - 10.1371/journal.pone.0156527
M3 - Article
C2 - 27310427
AN - SCOPUS:84976271826
SN - 1932-6203
VL - 11
JO - PloS one
JF - PloS one
IS - 6
M1 - e0156527
ER -