ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions

By Soham Parikh, Ananya B. Sai, Preksha Nema, Mitesh M. Khapra. 2018

31 citations recorded by the source corpus.

The task of Reading Comprehension with Multiple Choice Questions, requires a human (or machine) to read a given {passage, question} pair and select one of the n given options. The current state of the art model for this task first computes a question-aware representation for the passage and then selects the option which has the maximum similarity with this representation. However, when humans perform this task they do not just focus on option selection but use a combination of elimination and selection. Specifically, a human would first try to eliminate the most irrelevant option and then read the passage again in the light of this new information (and perhaps ignore portions corresponding to the eliminated option). This process could be repeated multiple times till the reader is finally ready to select the correct option. We propose ElimiNet, a neural network-based model which tries to mimic this process. Specifically, it has gates which decide whether an option can be eliminated given the {passage, question} pair and if so it tries to make the passage representation orthogonal to this eliminated option (akin to ignoring portions of the passage corresponding to the eliminated option). The model makes multiple rounds of partial elimination to refine the passage representation and finally uses a selection module to pick the best option. We evaluate our model on the recently released large scale RACE dataset and show that it outperforms the current state of the art model on 7 out of the 13 question types in this dataset. Further, we show that taking an ensemble of our elimination-selection based method with a selection based method gives us an improvement of 3.1% over the best-reported performance on this dataset.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Option Comparison Network for Multiple-choice Reading Comprehension (2019, background): “…models have been proposed for MCRC and achieve promising re-sults in recent years (Chen et al., 2016; Yin et al., 2016; Trischler et al., 2016; Dhingra et al., 2017; Tay et al., 2018; Parikh et al., 2018; Zhu et al., 2018; Wang et al., 2018; Xu et al., 2017; Sun et al., 2018; Zhang et al., 2019).”
A Co-Matching Model for Multi-choice Reading Comprehension (2018, background): “• ElimiNet (Soham et al., 2017) tries to ﬁrst eliminate the most irrelevant choices and then select the best answer.”
Improving Machine Reading Comprehension with General Reading Strategies (2018, methodology): “…of methods with attention mechanisms (Chen et al., 2016; Dhingra et al., 2017; Xu et al., 2018; Tay et al., 2018; Tang et al., 2019) for improvement such as adding an elimination module (Parikh et al., 2018) or applying hierarchical attention strategies (Zhu et al., 2018; Wang et al., 2018b).”
Multi-Matching Network for Multiple Choice Reading Comprehension (2019, background): “• ElimiNet (Parikh et al. 2018) tries to eliminate candidate answers in a multi-hop manner, which is built on top of gated attention layer(s).”
Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions (2018, methodology): “The word embeddings are initialized by the pre-trained GloVe word vectors (Common Crawl, 6B tokens, 100-dimension) (Pennington, Socher, and Manning 2014), and keep ﬁxed during training.”
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation (2021, background): “Knowledge of the question type can help in the development of question answering systems (Parikh et al., 2019) as well as for assessing performance on individual splits.”