Multi-task Learning with Multi-head Attention for Multi-choice Reading Comprehension
By Hui Wan. 2020
13 citations recorded by the source corpus.
Multiple-choice Machine Reading Comprehension (MRC) is an important and challenging Natural Language Understanding (NLU) task, in which a machine must choose the answer to a question from a set of choices, with the question placed in context of text passages or dialog. In the last a couple of years the NLU field has been revolutionized with the advent of models based on the Transformer architecture, which are pretrained on massive amounts of unsupervised data and then fine-tuned for various supervised learning NLU tasks. Transformer models have come to dominate a wide variety of leader-boards in the NLU field; in the area of MRC, the current state-of-the-art model on the DREAM dataset (see[Sunet al., 2019]) fine tunes Albert, a large pretrained Transformer-based model, and addition-ally combines it with an extra layer of multi-head attention between context and question-answer[Zhuet al., 2020].The purpose of this note is to document a new state-of-the-art result in the DREAM task, which is accomplished by, additionally, performing multi-task learning on two MRC multi-choice reading comprehension tasks (RACE and DREAM).
Subfields: Artificial Intelligence, Model Architectures, Question Answering & Reading Comprehension
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Model Architectures
In Question Answering & Reading Comprehension
Citations in the corpus
- Task-specific Objectives of Pre-trained Language Models for Dialogue Adaptation (2020, methodology): “2020), DUMA (Zhu, Zhao, and Li 2020) and DUMA+Multi-Task Learning (Wan 2020), which have complex matching networks for predicting answers.”