Reinforced Mnemonic Reader for Machine Comprehension

By Minghao Hu, Yuxing Peng, Xipeng Qiu. 2017

69 citations recorded by the source corpus.

In this paper, we introduce the Reinforced Mnemonic Reader for machine comprehension (MC) task, which aims to answer a query about a given context document. We propose several novel mechanisms that address critical problems in MC that are not adequately solved by previous works, such as enhancing the capacity of encoder, modeling long-term dependencies of contexts, refining the predicted answer span, and directly optimizing the evaluation metric. Extensive experiments on TriviaQA and Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

U-Net: Machine Reading Comprehension with Unanswerable Questions (2018, methodology): “…Currently, end-to-end neural network models have achieved great successes for machine reading comprehension (Seo et al. 2016; Kumar et al. 2015; Sukhbaatar et al. 2015; Cui et al. 2016; Xiong, Zhong, and Socher 2016; Dhingra et al. 2016; Shen et al. 2016; Hu et al. 2017; Wang, Yan, and Wu 2018).”
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, methodology): “A great number of end-to-end neural network models have been proposed to tackle these challenges, including BiDAF (Seo et al., 2016), r-net (Wang et al., 2017), DCN (Xiong et al., 2016), ReasoNet (Shen et al., 2017b), Document Reader (Chen et al., 2017), Interactive AoA Reader (Cui et al., 2017) and”
Know What You Don’t Know: Unanswerable Questions for SQuAD (2018, background): “In turn, these datasets have spurred a diverse array of model architecture improvements (Seo et al., 2016; Hu et al., 2017; Wang et al., 2017; Clark and Gard-ner, 2017; Huang et al., 2018).”
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension (2017, background): “, 2016) D D Mnemonic Reader (Hu et al., 2017) D D D R-net (Wang et al.”
MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension (2017, methodology): “Many works are based on the task of machine reading comprehension, and attention mechanism have been particularly successful(Xiong et al., 2016; Cui et al., 2016; Wang et al., 2016; Seo et al., 2016; Hu et al., 2017; Shen et al., 2016; Wang et al., 2017).”
Phase Conductor on Multi-layered Attentions for Machine Comprehension (2017, methodology): “Both Pan et al. (2017) and Hu et al. (2017) employs the structure of multi-hops or iterative aligner to repeatedly fuse the passage representation with the question representation as well as the passage representation itself.”
DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding (2017, methodology): “…dependency context-aware sequence representation, structured attention (Kim et al. 2017; Liu and Lapata 2017) and self-alignment attention (Hu, Peng, and Qiu 2017) are proposed, using cosine similarity between each pair of element form input sequence to mitigate the weakness of LSTM on…”
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling (2018, background): “token2token self-attention (Hu et al., 2017; Vaswani et al., 2017; Shen et al., 2017) produces context-aware representations by exploring the dependency between two tokens x i and x j from the same sequence x .”
A Multi-Stage Memory Augmented Neural Network for Machine Reading Comprehension (2018, background): “…2017b) 76.83 84.40 O O FusionNet (Huang et al., 2017b) 75.97 83.90 O O RaSoR + TR (Salant and Berant, 2017) 75.79 83.26 O - Conducter-net (Liu et al., 2017a) 74.41 82.74 O O Reinforced Mnemonic Reader (Hu et al., 2017) 73.20 81.80 O O BiDAF + Self-attention (Clark and Gardner, 2017) 72.14 81.05 -”
SpeechBERT: An Audio-and-Text Jointly Learned Language Model for End-to-End Spoken Question Answering (2019, unknown): “nd SpokenS-test, respectively. MODEL SQuAD-dev SpokenS-test (GT text) (ASR trans.) (train on plain text) EM F1 EM F1 BiDAF [26] 58.4 69.9 37.02 50.9 R-NET [27] 66.34 76.20 44.75 58.68 Mnemonic Reader [28] 64.00 73.35 40.36 52.87 Dr.QA [13] 62.84 73.74 41.16 54.51 FusionNet [29] 70.47 79.51 46.51 60.”
Dynamic Integration of Background Knowledge in Neural NLU Systems (2017, background): “For instance, the system of Hu et al. (2017) uses a complex multi-hop attention mechanism to achieve their results.”
Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading (2018, methodology): “Recently, a large body of work have been proposed to tackle the SQuAD dataset only e.g. in (Dhingra et al., 2016; Wang et al., 2017b; Xiong et al., 2017; Liu et al., 2017a; Hu et al., 2017; Huang et al., 2017; Liu et al., 2017b).”
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension (2018, background): “Massive reading comprehension question answering datasets (Hermann et al., 2015; Hill et al., 2016; Dhingra et al., 2017; Dunn et al., 2017) have driven a large number of successful neural approaches (Kadlec et al., 2016; Hu et al., 2017, inter alia ).”