Learning Natural Language Inference with LSTM

By Shuohang Wang, Jing Jiang. 2015

458 citations recorded by the source corpus.

Natural language inference (NLI) is a fundamentally important task in natural language processing that has many applications. The recently released Stanford Natural Language Inference (SNLI) corpus has made it possible to develop and evaluate learning-centered methods such as deep neural networks for natural language inference (NLI). In this paper, we propose a special long short-term memory (LSTM) architecture for NLI. Our model builds on top of a recently proposed neural attention model for NLI but is based on a significantly different idea. Instead of deriving sentence embeddings for the premise and the hypothesis to be used for classification, our solution uses a match-LSTM to perform word-by-word matching of the hypothesis with the premise. This LSTM is able to place more emphasis on important word-level matching results. In particular, we observe that this LSTM remembers important mismatches that are critical for predicting the contradiction or the neutral relationship label. On the SNLI corpus, our model achieves an accuracy of 86.1%, outperforming the state of the art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Structured Attention Networks (2017, neutral): “5 Matching LSTMs (Wang & Jiang, 2016) 86 .”
Long Short-Term Memory-Networks for Machine Reading (2016, methodology): “Recent approaches use two sequential LSTMs to encode the premise and the hypothesis respectively, and apply neural attention to reason about their logical relationship (Rocktäschel et al., 2016; Wang and Jiang, 2016).”
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “…deal of progress on NLU, serving as a standard benchmark for sentence understanding and spurring work on core representation learning techniques for NLU such as attention (Wang and Jiang, 2016), memory (Munkhdalai and Yu, 2017), and the use of parse structure (Mou et al., 2016b; ar X iv :1 70 4.”
A Decomposable Attention Model for Natural Language Inference (2016, methodology): “The neural counterpart to alignment, attention (Bahdanau et al., 2015), which is a key part of our approach, was originally proposed and has been predominantly used in conjunction with LSTMs (Rockt¨aschel et al., 2016; Wang and Jiang, 2015) and to a lesser extent with CNNs (Yin et al., 2016).”
NewsQA: A Machine Comprehension Dataset (2016, methodology): “The first neural model is the match-LSTM (mLSTM) system of Wang & Jiang (2016b).”
Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering (2017, neutral): “The match-LSTM (Wang & Jiang, 2016) model is one way to achieve the above effect in entailment.”
Gated Self-Matching Networks for Reading Comprehension and Question Answering (2017, methodology): “Wang and Jiang (2016b) build question-aware passage representation with match-LSTM (Wang and Jiang, 2016a), and predict answer boundaries in the passage with pointer networks (Vinyals et al., 2015).”
Dynamic Coattention Networks For Question Answering (2016, methodology): “For SQuAD, Wang & Jiang (2016b) proposed an end-to-end neural network model that consists of a Match-LSTM encoder, originally introduced in Wang & Jiang (2016a), and a pointer network decoder (Vinyals et al., 2015); Yu et al. (2016) introduced a dynamic chunk reader, a neural reading comprehension…”
Enhanced LSTM for Natural Language Inference (2016, background): “The model of Munkhdalai and Yu (2016b) extends the framework of Wang and Jiang (2016) to a full n-ary tree model and achieves further improvement.”
Machine Comprehension Using Match-LSTM and Answer Pointer (2016, methodology): “In a recent work on learning natural language inference, we proposed a match-LSTM model for predicting textual entailment (Wang & Jiang, 2016).”
A Fast Unified Model for Parsing and Sentence Understanding (2016, background): “ Google Faculty Research Award, the Stanford Data Science Initiative, and the National Science Foundation under grant nos. BCS 1456077 and IIS 4Attention-based models like Rocktaschel et al. (2016),¨ Wang and Jiang (2016), and the unpublished Cheng et al. (2016) have shown accuracies as high as 86.3”
Natural Language Inference over Interaction Space (2017, methodology): “Rockt¨aschel et al. (2015) aligns each sentence word-by-word with attention on top of LSTMs. Wang & Jiang (2015) enforces cross sentence attention word-by-word matching with the proprosed mL-STM model.”
Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment (2018, methodology): “…important, because the SNLI corpus is widely used as training data by many NN models for RTE (Bowman et al., 2015; Rockt¨aschel et al., 2015; Yin et al., 2016; Mou et al., 2016a; Wang and Jiang, 2016; Liu et al., 2016a; Liu et al., 2016b; Cheng et al., 2016; Parikh et al., 2016; Sha et al., 2016).”
R3: Reinforced Reader-Ranker for Open-Domain Question Answering (2017, methodology): “In this section, we ﬁrst review the Match-LSTM (Wang and Jiang, 2016) which provides input for both the Reader and Ranker.”
End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension (2016, methodology): “We propose a novel attention mechanism inspired by word-by-word style attention methods (Rocktäschel et al. 2015; Wang and Jiang 2015; Santos et al. 2016).”
A Co-Matching Model for Multi-choice Reading Comprehension (2018, methodology): “Inspired by some previous work (Wang and Jiang, 2016; Trischler et al., 2016), we ﬁrst use bi-directional LSTMs (Hochre-iter and Schmidhuber, 1997) to pre-process the sequences as follows: where H p ∈ R l × P , H q ∈ R l × Q and H a ∈ R l × A are the sequences of hidden states generated by the…”