DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference

By Reza Ghaeini, Sadid A. Hasan, Vivek Datla, Joey Liu, Kathy Lee, Ashequl Qadir, Yuan Ling, Aaditya (Adi) Prakash, Xiaoli Z. Fern, Oladimeji Farri. 2018

105 citations recorded by the source corpus.

We present a novel deep learning architecture to address the natural language inference (NLI) task. Existing approaches mostly rely on simple reading mechanisms for independent encoding of the premise and hypothesis. Instead, we propose a novel dependent reading bidirectional LSTM network (DR-BiLSTM) to efficiently model the relationship between a premise and a hypothesis during encoding and inference. We also introduce a sophisticated ensemble strategy to combine our proposed models, which noticeably improves final predictions. Finally, we demonstrate how the results can be improved further with an additional preprocessing step. Our evaluation shows that DR-BiLSTM obtains the best single model and ensemble model results achieving the new state-of-the-art scores on the Stanford NLI dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “The most successful approaches make use of attentional models that match and align words in the premise to those in the hypothesis (Tay et al., 2017; Peters et al., 2018; Ghaeini et al., 2018; Chen et al., 2017; Wang et al., 2017b; McCann et al., 2017), but recent non-attentional models designed to ”
Stress Test Evaluation for Natural Language Inference (2018, background): “Spelling errors occur often in MultiNLI data, due to involvement of Turkers and noisy source text (Ghaeini et al., 2018), which is problematic as some NLI systems rely heavily on word embeddings.”
Analyzing Compositionality-Sensitivity of NLI Models (2018, background): “…models us-ing both sequential and recursive representations (Nie and Bansal 2017; Choi, Yoo, and Lee 2017; Chen et al. 2017; Gong, Luo, and Zhang 2017; Ghaeini et al. 2018; Parikh et al. 2016; Wang, Hamza, and Florian 2017), all achieving near human-level performance on standard datasets.”
Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020, methodology): “Neural models proposed for NLI have been shown to be effective (Parikh et al., 2016; Chen et al., 2017d,e; Ghaeini et al., 2018; Peters et al., 2018), including models incorporating external knowledge (Chen et al., 2017b; Yang et al., 2019).”
Fine-grained Fact Verification with Kernel Graph Attention Network (2019, background): “Many fact verification systems leverage Natural Language Inference (NLI) techniques (Chen et al., 2017b; Ghaeini et al., 2018; Parikh et al., 2016; Radford et al., 2018; Peters et al., 2018; Li et al., 2019) to verify the claim.”
Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference (2018, methodology): “Several pieces of prior work in NLI have attempted to visualize the attention layer to provide some understanding of their models (Ghaeini et al., 2018b; Parikh et al., 2016).”
Kernel Graph Attention Network for Fact Verification (2019, background): “Many fact veriﬁcation systems leverage the NLI techniques (Chen et al., 2017b; Ghaeini et al., 2018; Parikh et al., 2016; Radford et al., 2018; Peters et al., 2018; Soleimani et al., 2019) to verify the claim: the NLI task is to classify the relationship between a pair of premise and hypothesis as…”
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (2019, background): “…to train complicated neural models which have achieved the state-of-the-art results (Bowman et al., 2015; Parikh et al., 2016; Sha et al., 2016; Chen et al., 2017b,c; Munkhdalai and Yu, 2017; Nie and Bansal, 2017; Conneau et al., 2017; Gong et al., 2018; Tay et al., 2018; Ghaeini et al., 2018).”