TY - JOUR
T1 - FeTaQA
T2 - Free-form Table Question Answering
AU - Nan, Linyong
AU - Hsieh, Chiachun
AU - Mao, Ziming
AU - Lin, Xi Victoria
AU - Verma, Neha
AU - Zhang, Rui
AU - Kryściński, Wojciech
AU - Schoelkopf, Nick
AU - Kong, Riley
AU - Tang, Xiangru
AU - Mutuma, Mutethia
AU - Rosand, Ben
AU - Trindade, Isabel
AU - Bandaru, Renusree
AU - Cunningham, Jacob
AU - Xiong, Caiming
AU - Radev, Dragomir
N1 - Publisher Copyright:
© 2022 Association for Computational Linguistics. Distributed under a CC-BY 4.0 license.
PY - 2022/1/28
Y1 - 2022/1/28
N2 - Existing table question answering datasets contain abundant factual questions that primarily evaluate a QA system’s comprehension of query and tabular data. However, restricted by their short-form answers, these datasets fail to include question–answer interactions that represent more advanced and naturally occurring information needs: questions that ask for reasoning and integration of information pieces retrieved from a structured knowledge source. To complement the existing datasets and to reveal the challenging nature of the table-based question answering task, we introduce FeTaQA, a new dataset with 10K Wikipedia-based {table, question, free-form answer, supporting table cells} pairs. FeTaQA is collected from noteworthy descriptions of Wikipedia tables that contain information people tend to seek; generation of these descriptions requires advanced processing that humans perform on a daily basis: Understand the question and table, retrieve, integrate, infer, and conduct text planning and surface realization to generate an answer. We provide two benchmark methods for the proposed task: a pipeline method based on semantic parsing-based QA systems and an end-to-end method based on large pretrained text generation models, and show that FeTaQA poses a challenge for both methods.
AB - Existing table question answering datasets contain abundant factual questions that primarily evaluate a QA system’s comprehension of query and tabular data. However, restricted by their short-form answers, these datasets fail to include question–answer interactions that represent more advanced and naturally occurring information needs: questions that ask for reasoning and integration of information pieces retrieved from a structured knowledge source. To complement the existing datasets and to reveal the challenging nature of the table-based question answering task, we introduce FeTaQA, a new dataset with 10K Wikipedia-based {table, question, free-form answer, supporting table cells} pairs. FeTaQA is collected from noteworthy descriptions of Wikipedia tables that contain information people tend to seek; generation of these descriptions requires advanced processing that humans perform on a daily basis: Understand the question and table, retrieve, integrate, infer, and conduct text planning and surface realization to generate an answer. We provide two benchmark methods for the proposed task: a pipeline method based on semantic parsing-based QA systems and an end-to-end method based on large pretrained text generation models, and show that FeTaQA poses a challenge for both methods.
UR - http://www.scopus.com/inward/record.url?scp=85123835799&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85123835799&partnerID=8YFLogxK
U2 - 10.1162/tacl_a_00446
DO - 10.1162/tacl_a_00446
M3 - Article
AN - SCOPUS:85123835799
SN - 2307-387X
VL - 10
SP - 35
EP - 49
JO - Transactions of the Association for Computational Linguistics
JF - Transactions of the Association for Computational Linguistics
ER -