Empirical evaluation of multi-task learning in deep neural networks for natural language processing

Jianquan Li, Xiaokang Liu, Wenpeng Yin, Min Yang, Liqun Ma, Yaohong Jin

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

Multi-task learning (MTL) aims at boosting the overall performance of each individual task by leveraging useful information contained in multiple-related tasks. It has shown great success in natural language processing (NLP). Currently, a number of MTL architectures and learning mechanisms have been proposed for various NLP tasks, including exploring linguistic hierarchies, orthogonality constraints, adversarial learning, gate mechanism, and label embedding. However, there is no systematic exploration and comparison of different MTL architectures and learning mechanisms for their strong performance in-depth. In this paper, we conduct a thorough examination of five typical MTL methods with deep learning architectures for a broad range of representative NLP tasks. Our primary goal is to understand the merits and demerits of existing MTL methods in NLP tasks, thus devising new hybrid architectures intended to combine their strengths. Following the empirical evaluation, we offer our insights and conclusions regarding the MTL methods we have considered.

Original languageEnglish (US)
Pages (from-to)4417-4428
Number of pages12
JournalNeural Computing and Applications
Volume33
Issue number9
DOIs
StatePublished - May 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Empirical evaluation of multi-task learning in deep neural networks for natural language processing'. Together they form a unique fingerprint.

Cite this