Textual Entailment with Structured Attentions and Composition

By Kai Zhao, Liang Huang, Mingbo Ma. 2016

29 citations recorded by the source corpus.

Deep learning techniques are increasingly popular in the textual entailment task, overcoming the fragility of traditional discrete models with hard alignments and logics. In particular, the recently proposed attention models (Rocktäschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art accuracy by computing soft word alignments between the premise and hypothesis sentences. However, there remains a major limitation: this line of work completely ignores syntax and recursion, which is helpful in many traditional efforts. We show that it is beneficial to extend the attention model to tree nodes between premise and hypothesis. More importantly, this subtree-level attention reveals information about entailment relation. We study the recursive composition of this subtree-level entailment relation, which can be viewed as a soft version of the Natural Logic framework (MacCartney and Manning, 2009). Experiments show that our structured attention and entailment composition model can correctly identify and infer entailment relations from the bottom up, and bring significant improvements in accuracy.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Structured Attention Networks (2017, background): “8 Attention over constituency tree nodes (Zhao et al., 2016) 87 .”
SciTaiL: A Textual Entailment Dataset from Science Question Answering (2018, neutral): “This idea is similar to the approach of Zhao, Huang, and Ma (2016), who compute entailment probabilities on each node in a binarized tree from the premise and hypothesis.”
Improving Natural Language Inference Using External Knowledge in the Science Questions Domain (2018, background): “Few prior approaches have exploited syntactic structures in the form of graphs for textual entailment (Zhao, Huang, and Ma 2016; Khot, Sabharwal, and Clark 2018).”
DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference (2018, background): “Various deep learning models have been proposed that achieve successful results for this task (Gong et al., 2017; Wang et al., 2017; Chen et al., 2017; Yu and Munkhdalai, 2017a; Parikh et al., 2016; Zhao et al., 2016; Sha et al., 2016).”
Leveraging Term Banks for Answering Complex Questions: A Case for Sparse Vectors (2017, background): “One possibility is to use a supervised deep learning approach with an attention model to focus on rare events (Li et al., 2016; Zhao et al., 2017).”