TY - GEN
T1 - PEAK
T2 - 30th AAAI Conference on Artificial Intelligence, AAAI 2016
AU - Yang, Qian
AU - Passonneau, Rebecca J.
AU - De Melo, Gerard
N1 - Publisher Copyright:
© Copyright 2016, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
PY - 2016
Y1 - 2016
N2 - Evaluating the selection of content in a summary is important both for human-written summaries, which can be a useful pedagogical tool for reading and writing skills, and machinegenerated summaries, which are increasingly being deployed in information management. The pyramid method assesses a summary by aggregating content units from the summaries of a wise crowd (a form of crowdsourcing). It has proven highly reliable but has largely depended on manual annotation. We propose PEAK, the first method to automatically assess summary content using the pyramid method that also generates the pyramid content models. PEAK relies on open information extraction and graph algorithms. The resulting scores correlate well with manually derived pyramid scores on both human and machine summaries, opening up the possibility of wide-spread use in numerous applications.
AB - Evaluating the selection of content in a summary is important both for human-written summaries, which can be a useful pedagogical tool for reading and writing skills, and machinegenerated summaries, which are increasingly being deployed in information management. The pyramid method assesses a summary by aggregating content units from the summaries of a wise crowd (a form of crowdsourcing). It has proven highly reliable but has largely depended on manual annotation. We propose PEAK, the first method to automatically assess summary content using the pyramid method that also generates the pyramid content models. PEAK relies on open information extraction and graph algorithms. The resulting scores correlate well with manually derived pyramid scores on both human and machine summaries, opening up the possibility of wide-spread use in numerous applications.
UR - http://www.scopus.com/inward/record.url?scp=85007237288&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85007237288&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85007237288
T3 - 30th AAAI Conference on Artificial Intelligence, AAAI 2016
SP - 2673
EP - 2679
BT - 30th AAAI Conference on Artificial Intelligence, AAAI 2016
PB - AAAI press
Y2 - 12 February 2016 through 17 February 2016
ER -