What’s Going On in Neural Constituency Parsers? An Analysis

By David Gaddy, Mitchell Stern, Daniel Klein. 2018

76 citations recorded by the source corpus.

A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity. The goal of this work is to analyze the extent to which information provided directly by the model structure in classical systems is still being captured by neural methods. To this end, we propose a high-performance neural model (92.08 F1 on PTB) that is representative of recent work and perform a series of investigative experiments. We find that our model implicitly learns to encode much of the same information that was explicitly provided by grammars and lexicons in the past, indicating that this scaffolding can largely be subsumed by powerful general-purpose neural machinery.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Dissecting Contextual Word Embeddings: Architecture and Representation (2018, background): “Concurrent with our work, Khandelwal et al. (2018) studied the role of context in inﬂuencing language model predictions, Gaddy et al. (2018) analyzed neural constituency parsers, Blevins et al. (2018) explored whether RNNs trained with several different objectives can learn hierarchical syntax, and…”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “, 2016) NMT word embeddings synonyms, morphological features Nearest neighbors (Gaddy et al., 2018) Parser word embeddings Word features (shape, etc.”
Linguistic Knowledge and Transferability of Contextual Representations (2019, background): “, 2016a, inter alia), ablating components of the model and dataset (Kuncoro et al., 2017; Gaddy et al., 2018; Khandelwal et al., 2018), or interpreting attention mechanisms (Bahdanau et al.”
Constituency Parsing with a Self-Attentive Encoder (2018, methodology): “(2017a) with additional modifications from Gaddy et al. (2018). Their parser is architecturally streamlined yet achieves the highest performance among discriminative single-system parsers trained on WSJ data only, which is why we”