Multi-range Reasoning for Machine Comprehension

By Yi Tay, Anh Tuan Luu, Siu Cheung Hui. 2018

36 citations recorded by the source corpus.

We propose MRU (Multi-Range Reasoning Units), a new fast compositional encoder for machine comprehension (MC). Our proposed MRU encoders are characterized by multi-ranged gating, executing a series of parameterized contract-and-expand layers for learning gating vectors that benefit from long and short-term dependencies. The aims of our approach are as follows: (1) learning representations that are concurrently aware of long and short-term context, (2) modeling relationships between intra-document blocks and (3) fast and efficient sequence encoding. We show that our proposed encoder demonstrates promising results both as a standalone encoder and as well as a complementary building block. We conduct extensive experiments on three challenging MC datasets, namely RACE, SearchQA and NarrativeQA, achieving highly competitive performance on all. On the RACE benchmark, our model outperforms DFN (Dynamic Fusion Networks) by 1.5%-6% without using any recurrent or convolution layers. Similarly, we achieve competitive performance relative to AMANDA on the SearchQA benchmark and BiDAF on the NarrativeQA benchmark without using any LSTM/GRU layers. Finally, incorporating MRU encoders with standard BiLSTM architectures further improves performance, achieving state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Mohit Bansal
3. Furu Wei
4. Yejin Choi
5. Yi Tay
6. Luke S. Zettlemoyer
7. Weizhu Chen
8. Percy Liang

In Computer Vision and Pattern Recognition

1. Jianfeng Gao
2. Mohit Bansal
3. Furu Wei
4. Luc Van Gool
5. Jie Zhou
6. Dacheng Tao
7. Trevor Darrell
8. Yi Tay

In Natural Language Processing

1. Furu Wei
2. Mohit Bansal
3. Jianfeng Gao
4. Yi Tay
5. Dong Yu
6. Weizhu Chen
7. Yejin Choi
8. M. Zhou

Citations in the corpus

• Option Comparison Network for Multiple-choice Reading Comprehension (2019, background): “…models have been proposed for MCRC and achieve promising re-sults in recent years (Chen et al., 2016; Yin et al., 2016; Trischler et al., 2016; Dhingra et al., 2017; Tay et al., 2018; Parikh et al., 2018; Zhu et al., 2018; Wang et al., 2018; Xu et al., 2017; Sun et al., 2018; Zhang et al., 2019).”
• DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension (2019, background): “However, world knowledge is rarely considered in state-of-the-art reading comprehension models (Tay et al., 2018; Wang et al., 2018b).”
• Improving Machine Reading Comprehension with General Reading Strategies (2018, methodology): “Researchers develop a variety of methods with attention mechanisms (Chen et al., 2016; Dhingra et al., 2017; Xu et al., 2018; Tay et al., 2018; Tang et al., 2019) for improvement such as adding an elimination module (Parikh et al., 2018) or applying hierarchical attention strategies (Zhu et al.,…”
• Beyond English-Only Reading Comprehension: Experiments in Zero-shot Multilingual Transfer for Bulgarian (2019, background): “Most previous work (Pan et al., 2018; Radford et al., 2018; Tay et al., 2018; Sun et al., 2019b) was monolingual, and a relevant context for each question was available a priori.”
• Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering (2018, methodology): “baselines: Bi-attn (MRU) (Tay et al., 2018) and Hierarchical Co-Matching (Wang et al.”
• Evidence Extraction for Machine Reading Comprehension with Deep Probabilistic Logic (2018, unknown): “…Fusion Networks (ensemble) (Yichong Xu, 2018) 55.6 49.4 51.2 BiAttention(MRU, single) (Tay et al., 2018) 57.7 47.4 50.4 BiAttention(MRU, ensemble) (Tay et al., 2018) 60.2 50.3 53.3 Co-matching (Wang et al., 2018) 55.8 48.2 50.4 Fined-Tuned Transformer (FT) (Radford et al., 2018) 62.9 57.4…”
• Multi-Matching Network for Multiple Choice Reading Comprehension (2019, methodology): “• BiAttention (MRU) (Tay, Tuan, and Hui 2018) adopts bidirectional attention to obtain matching representation among sequences encoded by Multi-range Reasoning Units (MRU).”
• Densely Connected Attention Propagation for Reading Comprehension (2018, background): “We also compare with the recent BiAttention + MRU model [Tay et al., 2018].”
• Attention-Guided Answer Distillation for Machine Reading Comprehension (2018, methodology): “Following Tay et al. (2018), we compete on the summary setting to compare with reported baselines.”
• Cut to the Chase: A Context Zoom-in Network for Reading Comprehension (2018, result): “We compare our model against reported models in Koˇcisk`y et al. (2017) (Seq2Seq, ASR, BiDAF) and the Multi-range Reasoning Unit (MRU) in Tay et al. (2018).”