A Fast Unified Model for Parsing and Sentence Understanding

By Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, Christopher Potts. 2016

323 citations recorded by the source corpus.

Tree-structured neural networks exploit valuable syntactic parse information as they interpret the meanings of sentences. However, they suer from two key technical problems that make them slow and unwieldyforlarge-scaleNLPtasks: theyusually operate on parsed sentences and they do not directly support batched computation. We address these issues by introducingtheStack-augmentedParser-Interpreter NeuralNetwork(SPINN),whichcombines parsing and interpretation within a single tree-sequence hybrid model by integrating tree-structured sentence interpretation into the linear sequential structure of a shiftreduceparser. Ourmodelsupportsbatched computation for a speedup of up to 25◊ over other tree-structured models, and its integrated parser can operate on unparsed data with little loss in accuracy. We evaluate it on the Stanford NLI entailment task and show that it significantly outperforms other sentence-encoding models.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Structured Attention Networks (2017, neutral): “1 Stack-Augmented Parser-Interpreter Neural Net (Bowman et al., 2016) 83 .”
Recurrent Neural Network Grammars (2016, neutral): “, 2013b), and sentence representation (Socher et al., 2011; Bowman et al., 2006).”
A Decomposable Attention Model for Natural Language Inference (2016, background): “300D LSTM RNN encoders (Bowman et al., 2016) 83.”
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, methodology): “SPINN: The Stack-augmented ParserInterpreter Neural Network (SPINN; Bowman et al., 2016) is a tree-based model that creates sentence representations by composing together the phrases of the sentence in accordance with a syntactic parse.”
Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (2018, neutral): “The addition of constituency parsing improves performance on sentence relatedness (SICK-R) and entailment (SICK-E) consistent with observations made by Bowman et al. (2016).”
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences (2018, methodology): “Sentence-encoding models (e.g. Bowman et al., 2015, 2016; Nie and Bansal, 2017; Shen et al., 2018) encode the premise and hypothesis individually, while attention-based models align words in the premise with similar words in the hypothesis, encoding the two sentences together (e.g. Rockt¨aschel et…”
Enhanced LSTM for Natural Language Inference (2016, methodology): “(2) 300D LSTM encoders (Bowman et al., 2016) 3.”
Natural Language Inference over Interaction Space (2017, methodology): “…representation for each sentence and classiﬁes the relation by using the concatenation of two vector representation along with their absolute element-wise difference and element-wise product(Bowman et al., 2016; Vendrov et al., 2015; Mou et al., 2015; Liu et al., 2016; Munkhdalai & Yu, 2016).”
Assessing Composition in Sentence Vector Representations (2018, methodology): “As part of this effort, we will expand to more comprehensive testing of a diverse range of sentence embedding systems (Bowman et al., 2016; Subramanian et al., 2018).”
Charagram: Embedding Words and Sentences via Character n-grams (2016, background): “, 2015) to functions with rich recursive structure (Socher et al., 2011; Tai et al., 2015; Bowman et al., 2016).”
Shortcut-Stacked Sentence Encoders for Multi-Domain Inference (2017, methodology): “Moreover, common sentence encoders can again be classiﬁed into tree-based encoders such as SPINN in Bowman et al. (2016) which we mentioned before, or sequential encoders such as the biLSTM model by Bowman et al. (2015).”
Good-Enough Compositional Data Augmentation (2019, background): “…or loss functions themselves—regularization (Oh et al., 2017), posterior regularization (Ganchev et al., 2010; Hu et al., 2018), explicit stacks (Grefenstette et al., 2015) and composition operators (Bowman et al., 2016); these existing proposals tend to be task-and architecture-speciﬁc.”