A Latent Variable Model for Generative Dependency Parsing

By Ivan Titov, James Henderson. 2007

106 citations recorded by the source corpus.

We propose a generative dependency parsing model which uses binary latent variables to induce conditioning features. To define this model we use a recently proposed class of Bayesian Networks for structured prediction, Incremental Sigmoid Belief Networks. We demonstrate that the proposed model achieves state-of-the-art results on three different languages. We also demonstrate that the features induced by the ISBN's latent variables are crucial to this success, and show that the proposed model is particularly good on long dependencies.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Recurrent Neural Network Grammars (2016, background): “While several transition-based neural models of syntactic generation exist (Henderson, 2003, 2004; Emami and Jelinek, 2005; Titov and Henderson, 2007; Buys and Blunsom, 2015b), these have relied on structure building operations based on parsing actions in shift-reduce and leftcorner parsers which op”
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations (2016, methodology): “(2016) and Vaswani et al. (2016) use a BiLSTM sequence tagging model to assign a CCG supertag for each token in the sentence.”
A Bayesian Model for Generative Transition-based Dependency Parsing (2015, background): “A previous generative transition-based dependency parser (Titov and Henderson, 2007) obtains competitive accuracies, but training and decoding is computationally very expensive.”
Top Accuracy and Fast Dependency Parsing is not a Contradiction (2010, methodology): “Titov and Henderson (2007) combined a transition based parsing algorithm, which used a beam search with a latent variable machine learning technique.”
A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies (2008, methodology): “The synchronous derivations described above are modelled with an Incremental Sigmoid Belief Network (ISBN) (Titov and Henderson, 2007a).”
Transition-based Dependency Parsing with Rich Non-local Features (2011, result): “Hence our work is in line with Titov and Henderson (2007) in using labeled transitions with global learning.”
The Inside-Outside Recursive Neural Network model for Dependency Parsing (2014, background): “Titov and Henderson (2007) were the ﬁrst proposing to use deep networks for dependency parsing.”
Structured Training for Neural Network Transition-Based Parsing (2015, background): “Finally, we also note that neural network representations have a long history in syntactic parsing (Henderson, 2004; Titov and Henderson, 2007; Titov and Henderson, 2010); however, like Chen and Manning (2014), our network avoids any recurrent structure so as to keep inference fast and e ﬃ cient…”
Generative Incremental Dependency Parsing with Neural Networks (2015, methodology): “Our model outperforms the generative model of Titov and Henderson (2007), which we retrained on our dataset, by despite that model being able to condition on arbitrary-sized contexts.”
Distributed Word Representation Learning for Cross-Lingual Dependency Parsing (2014, background): “From the perspective of applying deep networks in natural language processing systems, there are a number of works in the literature (Collobert and Weston, 2008; Collobert et al., 2011; Henderson, 2004; Socher et al., 2011; Titov and Henderson, 2010; Turian et al., 2010).”