Shortcut-Stacked Sentence Encoders for Multi-Domain Inference

By Yixin Nie, Mohit Bansal. 2017

129 citations recorded by the source corpus.

We present a simple sequential sentence encoder for multi-domain natural language inference. Our encoder is based on stacked bidirectional LSTM-RNNs with shortcut connections and fine-tuning of word embeddings. The overall supervised model uses the above encoder to encode two input sentences into two vectors, and then uses a classifier over the vector combination to label the relationship between these two sentences as that of entailment, contradiction, or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements over existing encoders on matched and mismatched multi-domain natural language inference (top single-model result in the EMNLP RepEval 2017 Shared Task (Nangia et al., 2017)). Moreover, they achieve the new state-of-the-art encoding result on the original SNLI dataset (Bowman et al., 2015).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “Recently, several hard test sets have been made available for revealing the biases NLI models learn from their training datasets (Nie and Bansal, 2017; McCoy et al., 2019; Gururangan et al., 2018; Naik et al., 2018).”
e-SNLI: Natural Language Inference with Natural Language Explanations (2018, background): “Glockner et al. show that models achieving high accuracies on SNLI, such as [21, 22, 6], show dramatically reduced performance on this simpler dataset, while the model of Chen et al. [5] is more robust due to incorporating exter-nal knowledge.”
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences (2018, methodology): “Sentence-encoding models (e.g. Bowman et al., 2015, 2016; Nie and Bansal, 2017; Shen et al., 2018) encode the premise and hypothesis individually, while attention-based models align words in the premise with similar words in the hypothesis, encoding the two sentences together (e.”
Stress Test Evaluation for Natural Language Inference (2018, background): “on MultiNLI (Nie and Bansal, 2017), and create a typology of phenomena that it finds challenging (§2).”
Visual Entailment: A Novel Task for Fine-Grained Image Understanding (2019, methodology): “Since then, several neural-network based methods have been proposed over SNLI that either use sentence encoding models to individually encode hypothesis and premise or attention based models that encode the sentences together and align similar words in hypothesis and premise [8, 50, 62, 59].”
Combining Fact Extraction and Verification with Neural Semantic Matching Networks (2018, background): “2015) (SNLI) and the Multi-Genre Natural Language Inference (Williams, Nangia, and Bowman 2018) (Multi-NLI) have promoted the development of many different neural NLI models (Nie and Bansal 2017; Conneau et al. 2017; Parikh et al. 2016; Chen et al. 2017b; Gong, Luo, and Zhang 2017; Ghaeini et al. 20”
EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference (2019, methodology): “, 2018) spurred the development of many neural models (Parikh et al., 2016; Nie and Bansal, 2017; Conneau et al., 2017; Balazs et al., 2017; Chen et al., 2017a; Radford et al., 2018; Devlin et al., 2018).”
Teaching Syntax by Adversarial Distraction (2018, background): “(Conneau et al., 2017; Nie and Bansal, 2017), but most common models apply attention between tokens in the premise and hypothesis.”
Star-Transformer (2019, unknown): “…83.2 Tree-based CNN (Mou et al., 2016) 82.1 4096D BiLSTM-max (Conneau et al., 2017) 84.5 300D DiSAN (Shen et al., 2018a) 85.6 Residual encoders (Nie and Bansal, 2017) 86.0 Gumbel TreeLSTM (Choi et al., 2018) 86.0 Reinforced self-att (Shen et al., 2018b) 86.3 2400D Multiple DSA (Yoon et al.,…”
Analyzing Compositionality-Sensitivity of NLI Models (2018, methodology): “…development of many different neural NLI models, including encoding and co-attention models us-ing both sequential and recursive representations (Nie and Bansal 2017; Choi, Yoo, and Lee 2017; Chen et al. 2017; Gong, Luo, and Zhang 2017; Ghaeini et al. 2018; Parikh et al. 2016; Wang, Hamza, and…”
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (2019, background): “…to train complicated neural models which have achieved the state-of-the-art results (Bowman et al., 2015; Parikh et al., 2016; Sha et al., 2016; Chen et al., 2017b,c; Munkhdalai and Yu, 2017; Nie and Bansal, 2017; Conneau et al., 2017; Gong et al., 2018; Tay et al., 2018; Ghaeini et al., 2018).”
On the Use of Word Embeddings Alone to Represent Natural Language Sequences (2018, methodology): “Notably, on SNLI dataset, we observe that SWEM-max performs the best among all SWEM variants, consistent with the ﬁndings in Nie & Bansal (2017); Conneau et al. (2017) that max-pooling over BiLSTM hidden units outperforms average pooling operation on SNLI dataset.”
Learning to Compose Task-Specific Tree Structures (2017, methodology): “For a model to correctly predict the relationship between two sentences, it should en-2 (Munkhdalai and Yu, 2017a) 84.6 3.0M 26 † 600D Gumbel Tree-LSTM (Ours) 86.0 10.3M 3.4 600D (300+300) Gated-Attention BiLSTM (Chen et al., 2017) 85.5 11.6M 8.5 † 512–1024–2048D Shortcut-Stacked BiLSTM (Nie and Ban”