Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning
By Yufan Jiang, Shuangzhi Wu, Jing Gong, Yahui Cheng, Peng Meng, Weiliang Lin, Zhibo Chen, Mu Li. 2020
11 citations recorded by the source corpus.
Multi-choice Machine Reading Comprehension (MMRC) aims to select the correct answer from a set of options based on a given passage and question. Due to task specific of MMRC, it is non-trivial to transfer knowledge from other MRC tasks such as SQuAD, Dream. In this paper, we simply reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct. Then select the option with the highest confidence score. We construct our model upon ALBERT-xxlarge model and estimate it on the RACE dataset. During training, We adopt AutoML strategy to tune better parameters. Experimental results show that the single-choice is better than multi-choice. In addition, by transferring knowledge from other kinds of MRC tasks, our model achieves a new state-of-the-art results in both single and ensemble settings.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohammad Shoeybi
- Bryan Catanzaro
- Yuxiong He
- Samyam Rajbhandari
- M. Patwary
- Jared Casper
- Vijay Anand Korthikanti
- Xia Song
In Computer Vision and Pattern Recognition
- Bryan Catanzaro
- Mohammad Shoeybi
- Jared Casper
- Vijay Anand Korthikanti
- Samyam Rajbhandari
- M. Patwary
- Yuxiong He
- Michael Houston
In Natural Language Processing
- Mohammad Shoeybi
- M. Patwary
- Shaden Smith
- Yuxiong He
- Bryan Catanzaro
- Patrick LeGresley
- Samyam Rajbhandari
- Reza Yazdani Aminabadi
Citations in the corpus
- Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, methodology): “For RACE dataset, one of the best supervised models to date is an ALBERT ensemble [24].”
- UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training (2022, unknown): “…et al., 2020) 92.0 ELECTRA,142M (Zhang et al., 2020) 91.0 ERNIE3,10B (Fedus et al., 2021) 58.9 Masque (Nishida et al., 2019) 90.7 ALBERT, 31M (Jiang et al., 2020) 86.0 UnifiedQA-ft,11B (Khashabi et al., 2020) 86.4 UnifiedQA-ft,11B (Khashabi et al., 2020) 75.0 UnifiedQA-ft,11B (Khashabi et…”