Learning Accurate, Compact, and Interpretable Tree Annotation

By Slav Petrov, Leon Barrett, Romain Thibaux, Daniel Klein. 2006

991 citations recorded by the source corpus.

We present an automatic approach to tree annotation in which basic nonterminal symbols are alternately split and merged to maximize the likelihood of a training treebank. Starting with a simple X-bar grammar, we learn a new grammar whose nonterminals are subsymbols of the original nonterminals. In contrast with previous work, we are able to split various terminals to different degrees, as appropriate to the actual complexity in the data. Our grammars automatically learn the kinds of linguistic distinctions exhibited in previous work on manual tree annotation. On the other hand, our grammars are much more compact and substantially more accurate than previous work on automatic annotation. Despite its simplicity, our best grammar achieves an F1 of 90.2% on the Penn Treebank, higher than fully lexicalized systems.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Attention is All you Need (2017, methodology): “In contrast to RNN sequence-to-sequence models [37], the Transformer outperforms the BerkeleyParser [29] even when training only on the WSJ training set of 40K sentences.”
Grammar as a Foreign Language (2014, methodology): “[12] partially alleviate the heavy reliance on manual modeling of linguistic structure by using latent variables to learn a more articulated model.”
Learning Dependency-Based Compositional Semantics (2011, methodology): “13 To perform POS tagging, we used the Berkeley Parser (Petrov et al. 2006), trained on the WSJ Treebank (Marcus, Marcinkiewicz, and Santorini 1993) and the Question Treebank (Judge, Cahill, and v.”
Recurrent Convolutional Neural Networks for Text Classification (2015, methodology): “We list two major meth-ods for comparison: the context-free grammar (CFG) produced by the Berkeley parser (Petrov et al. 2006)”
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints (2016, methodology): “We preprocess all data using the Berkeley Parser (Petrov et al., 2006), speciﬁcally the GPU-accelerated version of the parser from Hall et al. (2014), and the Berkeley Entity Resolution System (Durrett and Klein, 2014).”
Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation (T) (2015, neutral): “In standard SMT training, word aligments are automatically calculated from the statistics of a parallel corpus by using a probabilistic model and unsupervised machine learning techniques [21], [22].”
What Do Recurrent Neural Network Grammars Learn About Syntax? (2016, background): “…representative samples for NPs, VPs, and PPs. given bracketing structure, simple dimensionality reduction techniques could reveal conventional nonterminal categories with high accuracy; Petrov et al. (2006) also showed that latent variables can be used to recover ﬁne-grained nonterminal categories.”
EU-BRIDGE MT: Combined Machine Translation (2014, methodology): “The extraction is GHKM-based with syntactic target-side annotation from the German Berkeley Parser, but we strip off the syntactic labels.”
Self-Training PCFG Grammars with Latent Annotations Across Languages (2009, methodology): “We observe that many of the rule parameters of the grammar trained on WSJ training data alone have zero probabilities (rules with extremely low probabilities are also filtered to zero), as was also pointed out in (Petrov et al., 2006).”
Exploiting Source-side Monolingual Data in Neural Machine Translation (2016, methodology): “To pre-order the Chinese sentences using the syntax-based reordering method proposed by (Wang et al., 2007), we utilize the Berkeley parser (Petrov et al., 2006).”
11,001 New Features for Statistical Machine Translation (2009, methodology): “…our English training trees using expectation-maximization (Wang et al., 2007), and to get more speciﬁc translation rules, we relabel the trees with up to 4 specialized versions of each nonterminal symbol, again using expectation-maximization and the split / merge technique of Petrov et al. (2006).”
Edinburgh’s Syntax-Based Systems at WMT 2015 (2014, methodology): “To parse the English data, we used the Berkeley parser (Petrov et al., 2006; Petrov and Klein, 2007).”
Improved Inference for Unlexicalized Parsing (2007, methodology): “Most research on parsing has focused on English and parsing performance on other languages is generally signiﬁcantly lower.”
Edinburgh’s Statistical Machine Translation Systems for WMT16 (2016, methodology): “To parse the target-side of the training data, we used the Berkeley parser (Petrov et al., 2006; Petrov and Klein, 2007) for English, and the ParZu dependency parser (Sennrich et al., 2013) for German.”
Large-scale Exact Decoding: The IMS-TTT submission to WMT14 (2014, methodology): “The English half of the parallel data was parsed using Egret 9 which is a re-implementation of the Berkeley parser (Petrov et al., 2006).”
Products of Random Latent Variable Grammars (2010, methodology): “Here, we focus on the latent variable approach of Petrov et al. (2006), where an Expectation Maximization (EM) algorithm is used to induce a hierarchy of increasingly more reﬁned grammars.”