TY - JOUR
T1 - SentiMedQAer
T2 - A Transfer Learning-Based Sentiment-Aware Model for Biomedical Question Answering
AU - Zhu, Xian
AU - Chen, Yuanyuan
AU - Gu, Yueming
AU - Xiao, Zhifeng
N1 - Publisher Copyright:
Copyright © 2022 Zhu, Chen, Gu and Xiao.
PY - 2022/3/10
Y1 - 2022/3/10
N2 - Recent advances have witnessed a trending application of transfer learning in a broad spectrum of natural language processing (NLP) tasks, including question answering (QA). Transfer learning allows a model to inherit domain knowledge obtained from an existing model that has been sufficiently pre-trained. In the biomedical field, most QA datasets are limited by insufficient training examples and the presence of factoid questions. This study proposes a transfer learning-based sentiment-aware model, named SentiMedQAer, for biomedical QA. The proposed method consists of a learning pipeline that utilizes BioBERT to encode text tokens with contextual and domain-specific embeddings, fine-tunes Text-to-Text Transfer Transformer (T5), and RoBERTa models to integrate sentiment information into the model, and trains an XGBoost classifier to output a confidence score to determine the final answer to the question. We validate SentiMedQAer on PubMedQA, a biomedical QA dataset with reasoning-required yes/no questions. Results show that our method outperforms the SOTA by 15.83% and a single human annotator by 5.91%.
AB - Recent advances have witnessed a trending application of transfer learning in a broad spectrum of natural language processing (NLP) tasks, including question answering (QA). Transfer learning allows a model to inherit domain knowledge obtained from an existing model that has been sufficiently pre-trained. In the biomedical field, most QA datasets are limited by insufficient training examples and the presence of factoid questions. This study proposes a transfer learning-based sentiment-aware model, named SentiMedQAer, for biomedical QA. The proposed method consists of a learning pipeline that utilizes BioBERT to encode text tokens with contextual and domain-specific embeddings, fine-tunes Text-to-Text Transfer Transformer (T5), and RoBERTa models to integrate sentiment information into the model, and trains an XGBoost classifier to output a confidence score to determine the final answer to the question. We validate SentiMedQAer on PubMedQA, a biomedical QA dataset with reasoning-required yes/no questions. Results show that our method outperforms the SOTA by 15.83% and a single human annotator by 5.91%.
UR - http://www.scopus.com/inward/record.url?scp=85127602109&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85127602109&partnerID=8YFLogxK
U2 - 10.3389/fnbot.2022.773329
DO - 10.3389/fnbot.2022.773329
M3 - Article
C2 - 35360832
AN - SCOPUS:85127602109
SN - 1662-5218
VL - 16
JO - Frontiers in Neurorobotics
JF - Frontiers in Neurorobotics
M1 - 773329
ER -