TY - GEN
T1 - Do summaries help? A task-based evaluation of multi-document summarization
AU - McKeown, Kathleen
AU - Passonneau, Rebecca J.
AU - Elson, David K.
AU - Nenkova, Ani
AU - Hirschberg, Julia
PY - 2005
Y1 - 2005
N2 - We describe a task-based evaluation to determine whether multi-document summaries measurably improve user performance whe using online news browsing systems for directed research. We evaluated the multi-document summaries generated by Newsblaster, a robust news browsing system that clusters online news articles and summarizes multiple articles on each event. Four groups of subjects were asked to perform the same time-restricted fact-gathering tasks, reading news under different conditions: no summaries at all, single sentence summaries drawn from one of the articles, Newsblaster multi-document summaries, and human summaries. Our results show that, in comparison to source documents only, the quality of reports assembled using Newsblaster summaries was significantly better and user satisfaction was higher with both Newsblaster and human summaries.
AB - We describe a task-based evaluation to determine whether multi-document summaries measurably improve user performance whe using online news browsing systems for directed research. We evaluated the multi-document summaries generated by Newsblaster, a robust news browsing system that clusters online news articles and summarizes multiple articles on each event. Four groups of subjects were asked to perform the same time-restricted fact-gathering tasks, reading news under different conditions: no summaries at all, single sentence summaries drawn from one of the articles, Newsblaster multi-document summaries, and human summaries. Our results show that, in comparison to source documents only, the quality of reports assembled using Newsblaster summaries was significantly better and user satisfaction was higher with both Newsblaster and human summaries.
UR - http://www.scopus.com/inward/record.url?scp=80052119797&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=80052119797&partnerID=8YFLogxK
U2 - 10.1145/1076034.1076072
DO - 10.1145/1076034.1076072
M3 - Conference contribution
AN - SCOPUS:80052119797
SN - 1595930345
SN - 9781595930347
T3 - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
SP - 210
EP - 217
BT - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
T2 - 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005
Y2 - 15 August 2005 through 19 August 2005
ER -