Inducing History Representations for Broad Coverage Statistical Parsing

By James Henderson. 2003

105 citations recorded by the source corpus.

We present a neural network method for inducing representations of parse histories and using these history representations to estimate the probabilities needed by a statistical left-corner parser. The resulting statistical parser achieves performance (89.1% F-measure) on the Penn Treebank which is only 0.6% below the best current parser for this task, despite using a smaller vocabulary size and less prior linguistic knowledge. Crucial to this success is the use of structurally determined soft biases in inducing the representation of the parse history, and no use of hard independence assumptions.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Recurrent Neural Network Grammars (2016, methodology): “The discriminative model also lets us use ancestor sampling to obtain samples of parse trees for sentences, and this is used to solve a second practical challenge with RNNGs: approximating the marginal likelihood and MAP tree of a sentence under the generative model.”
Grammar as a Foreign Language (2014, background): “Early neural network approaches to parsing, for example by Henderson (2003; 2004) also relied on strong linguistic insights.”
Globally Normalized Transition-Based Neural Networks (2016, methodology): “Our model conﬁguration is basically the same as the one originally proposed by Chen and Manning (2014) and then re-ﬁned by Weiss et al. (2015).”
Discriminative Training of a Neural Network Statistical Parser (2004, methodology): “This paper outlines each of these components, but more details can be found in (Henderson, 2003b), and, for the discriminative model, in (Henderson, 2003a).”
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better (2018, positive): “As RNNGs build phrase-structure trees through top-down operations, we propose extensions to the structure-building sequences and model architecture that enable left-corner (Henderson, 2003, 2004) and bottom-up (Chelba and Jelinek, 2000; Emami and Jelinek, 2005) generation orders (§4).”
Neural CRF Parsing (2015, methodology): “…CRFs. Prior work on parsing using neural network models has often sidestepped the problem of structured inference by making sequential decisions (Henderson, 2003; Chen and Manning, 2014; Tsuboi, 2014) or by doing reranking (Socher et al., 2013; Le and Zuidema, 2014); by contrast, our framework…”
Robust Parsing of the Proposition Bank (2006, methodology): “We present work to test the hypothesis that a current statistical parser (Henderson, 2003) can output richer information robustly, that is without any significant degradation of the parser’s accuracy on the original parsing task, by explicitly modelling semantic role labels as the interface between ”
A Neural Syntactic Language Model (2005, methodology): “An improved recurrent neural network has been used by Henderson (2000, 2003) to model a left-corner parser and has achieved state-of-the-art results.”
Probabilistic CFG with Latent Annotations (2005, result): “…generalization of non-terminal symbols of a PCFG, but they did not report the results of parsing with the obtained PCFG. Henderson’s parsing model (Henderson, 2003) has a similar motivation as ours in that a derivation history of a parse tree is compactly represented by induced hidden variables…”
The Unstoppable Rise of Computational Linguistics in Deep Learning (2020, methodology): “on language modelling. Bengio et al. (2001, 2003) used an MLP to estimate the parameters of an n-gram language model, and showed PTB Constituents model LP LR F1 Costa et al. (2001) PoS 57.8 64.9 61.1 Henderson (2003) PoS 83.3 84.3 83.8 Henderson (2003) 88.8 89.5 89.1 Henderson (2004) 89.8 90.4 90.1 ”
A Latent Variable Model for Generative Dependency Parsing (2007, methodology): “Instead we use a pruning strategy similar to that de scribed in [17], where it was applied to a considerably harder search problem: cons tituent parsing with a left-corner parsing order.”
Improving Neural Parsing by Disentangling Model Combination and Reranking Effects (2017, methodology): “decoding procedures developed for other generative models (Henderson, 2003; Titov and Henderson, 2010; Buys and Blunsom, 2015) and can be viewed as a simplified version of the procedure used in the generative top-down parsers of Roark (2001) and Charniak (2010).”
Constituent Parsing with Incremental Sigmoid Belief Networks (2007, methodology): “In this section we evaluate the two approximations to dynamic SBNs discussed in the previous section, the feed-forward method equivalent to the neural network of (Henderson, 2003) (NN method) and the mean ﬁeld method (MF method).”
Top-Down Nearly-Context-Sensitive Parsing (2010, methodology): “Similarly, consider the more recent work of Nivre(Nivre, 2003) and Henderson(Henderson, parse ( w where q ′ = q ∗ p ( x | t ) , r ′ = LAP( t ′ , w ′ ) , and ).”
Incremental Sigmoid Belief Networks for Grammar Learning (2010, methodology): “We replicated the same definition of derivation and the same pattern of interconnection between states as described in Henderson (2003).”