A Minimal Span-Based Neural Constituency Parser

By Mitchell Stern, Jacob Andreas, Daniel Klein. 2017

203 citations recorded by the source corpus.

In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans. We show that this model is not only compatible with classical dynamic programming techniques, but also admits a novel greedy top-down inference algorithm based on recursive partitioning of the input. We demonstrate empirically that both prediction schemes are competitive with recent work, and when combined with basic extensions to the scoring model are capable of achieving state-of-the-art single-model performance on the Penn Treebank (91.79 F1) and strong performance on the French Treebank (82.23 F1).

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, methodology): “For each pair of captions, we use a constituency parser (Stern et al., 2017) to split the second sentence into noun and verb phrases (Figure 1).”
AllenNLP: A Deep Semantic Natural Language Processing Platform (2018, methodology): “The AllenNLP constituency parser is an implementation of a minimal neural model for constituency parsing based on an independent scoring of labels and spans (Stern et al., 2017).”
Constituency Parsing with a Self-Attentive Encoder (2018, methodology): “Stern et al. (2017a) define → y k and ← y k in terms of the output of the forward and backward portions, respectively, of their BiLSTM encoder; we instead construct each of → y k and ← y k by splitting in half2 the outputs yk from Section 2.2.”
Unsupervised Question Answering by Cloze Translation (2019, methodology): “Most relevant to us is unsupervised NMT (Con-neau et al., 2017; Lample et al., 2017, 2018; Artetxe et al., 2018) and style transfer (Subrama-nian et al., 2018).”
Value-Agnostic Conversational Semantic Parsing (2021, result): “Similar to Stern et al. (2017) and Kurib-ayashi et al. (2019), we assume that the score for a span factorizes, and deﬁne the embedding of each span value as the concatenation of the contextual embeddings of the ﬁrst and last tokens of the span, ˜ v = [ h k start utt ; h k end utt ] .”
Span-based Semantic Parsing for Compositional Generalization (2020, methodology): “Span-based parsing had success in both syntactic (Stern et al., 2017; Kitaev and Klein, 2018) and semantic parsing (Pasupat et al., 2019).”
What’s Going On in Neural Constituency Parsers? An Analysis (2018, methodology): “We then follow past work in dependency parsing (Wang and Chang, 2016) and constituency parsing (Cross and Huang, 2016b; Stern et al., 2017) in representing the span ( i, j ) by the concatenation of the corresponding forward and backward span differences: See Figure 1 for an illustration.”
Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples (2018, methodology): “Stern et al. (2017a) demonstrated that a parser with minimal dependence between the decisions that produce a parse can achieve state-of-the-art performance.”
Unsupervised Recurrent Neural Network Grammars (2019, methodology): “6 , 7 The parameterization of span scores is similar to recent works (Wang and Chang, 2016; Stern et al., 2017; Kitaev and Klein, 2018): we add position embeddings to word embeddings and run a bidirectional LSTM over the input representations to obtain the forward The score s ij ∈ R for a…”
An Empirical Study of Building a Strong Baseline for Constituency Parsing (2018, unknown): “7 (Stern et al., 2017a) Top-down 91.”
Visually Grounded Compound PCFGs (2020, methodology): “To realize this desideratum, we could follow the inside algorithm and recursively compose span representations (Le and Zuidema, 2015; Stern et al., 2017; Drozdov et al., 2019), which is, however, time-and memory-inefﬁcient in practice.”
SpanNER: Named Entity Re-/Recognition as Span Prediction (2021, methodology): “, 2020) as well as its re-ranking (Collins and Koo, 2005; Huang, 2008) as a span prediction (Stern et al., 2017) problem, which is be unified and parameterized with the same form.”
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2020, methodology): “Our formulation for encoding spans is similar to that used in other neural span-factored models (Stern et al., 2017; Lee et al., 2017).”
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (2018, methodology): “…performance of similar span embedding methods for coreference (Lee et al., 2017), suggesting that this style of models could be used for other span-span relation tasks, such as syntactic parsing (Stern et al., 2017), relation extraction (Miwa and Bansal, 2016), and QA-SRL (FitzGerald et al., 2018).”
Learning Syntax from Naturally-Occurring Bracketings (2021, methodology): “6This is inspired by span-based supervised constituencyparsing methods (Stern et al., 2017), which in turn was based on Wang and Chang (2016).”
Position-Aware Tagging for Aspect Sentiment Triplet Extraction (2020, methodology): “Motivated by (Wang and Chang, 2016; Stern et al., 2017), we calculate the segment representation g a,b for an opinion span with boundaries of a and b (both inclusive) as follows: where”
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank (2019, methodology): “We use CKY-style algorithm (Stern et al., 2017a; Gaddy et al., 2018) to obtain the tree ˆ T in O ( n 3 ) time complexity.”
Compound Probabilistic Context-Free Grammars for Grammar Induction (2019, methodology): “We use a CRF parser whose span score parameterization is similar similar to recent works (Wang and Chang, 2016; Stern et al., 2017; Kitaev and Klein, 2018): position embeddings are added to word embeddings, and a bidirectional LSTM with 256 hidden dimensions is run over the input representations to…”
Cross-Domain Generalization of Neural Constituency Parsers (2019, background): “Is the success of neural constituency parsers (Henderson 2004; Vinyals et al. 2015; Dyer et al. 2016; Cross and Huang 2016; Choe and Charniak 2016; Stern et al. 2017; Liu and Zhang 2017; Ki-taev and Klein 2018, inter alia ) similarly transferable to out-of-domain treebanks?”