TY - GEN
T1 - UNIFIEDSKG
T2 - 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
AU - Xie, Tianbao
AU - Wu, Chen Henry
AU - Shi, Peng
AU - Zhong, Ruiqi
AU - Scholak, Torsten
AU - Yasunaga, Michihiro
AU - Wu, Chien Sheng
AU - Zhong, Ming
AU - Yin, Pengcheng
AU - Wang, Sida I.
AU - Zhong, Victor
AU - Wang, Bailin
AU - Li, Chengzu
AU - Boyle, Connor
AU - Ni, Ansong
AU - Yao, Ziyu
AU - Radev, Dragomir
AU - Xiong, Caiming
AU - Kong, Lingpeng
AU - Zhang, Rui
AU - Smith, Noah A.
AU - Zettlemoyer, Luke
AU - Yu, Tao
N1 - Publisher Copyright:
© 2022 Association for Computational Linguistics.
PY - 2022
Y1 - 2022
N2 - Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UNIFIEDSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UNIFIEDSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UNIFIEDSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UNIFIEDSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UNIFIEDSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg.
AB - Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UNIFIEDSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UNIFIEDSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UNIFIEDSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UNIFIEDSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UNIFIEDSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg.
UR - https://www.scopus.com/pages/publications/85143086823
UR - https://www.scopus.com/inward/citedby.url?scp=85143086823&partnerID=8YFLogxK
U2 - 10.18653/v1/2022.emnlp-main.39
DO - 10.18653/v1/2022.emnlp-main.39
M3 - Conference contribution
AN - SCOPUS:85143086823
T3 - Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
SP - 602
EP - 631
BT - Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
A2 - Goldberg, Yoav
A2 - Kozareva, Zornitsa
A2 - Zhang, Yue
PB - Association for Computational Linguistics (ACL)
Y2 - 7 December 2022 through 11 December 2022
ER -