A Fast and Accurate Dependency Parser using Neural Networks

By Danqi Chen, Christopher D. Manning. 2014

1,913 citations recorded by the source corpus.

Almost all current dependency parsers classify based on millions of sparse indicator features. Not only do these features generalize poorly, but the cost of feature computation restricts parsing speed significantly. In this work, we propose a novel way of learning a neural network classifier for use in a greedy, transition-based dependency parser. Because this classifier learns and uses just a small number of dense features, it can work very fast, while achieving an about 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets. Concretely, our parser is able to parse more than 1000 sentences per second at 92.2% unlabeled attachment score on the English Penn Treebank.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval (2015, methodology): “, 2010), parsing (Chen and Manning, 2014), sentiment (Socher et al.”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, methodology): “We use the Stanford Parser (Chen and Manning, 2014) distributed in Stanford CoreNLP version 3.9.1 to identify the relevant dependency arcs.”
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task (2016, methodology): “We use Stanford’s neural network dependency parser (Chen and Manning, 2014) to parse all our document and question text, and all other features can be extracted without additional tools.”
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks (2015, neutral): “…44.4 82.9 RNTN (Socher et al., 2013) 45.7 85.4 DCNN (Blunsom et al., 2014) 48.5 86.8 Paragraph-Vec (Le and Mikolov, 2014) 48.7 87.8 CNN-non-static (Kim, 2014) 48.0 87.2 CNN-multichannel (Kim, 2014) 47.4 88.1 DRNN (Irsoy and Cardie, 2014) 49 (Chen and Manning, 2014) and the Binary Tree-LSTM (Sec.”
Learning to Generate Reviews and Discovering Sentiment (2017, background): “These representations, learned by modeling word co-occurrences, increase the data efﬁciency and generalization capability of NLP systems (Pennington et al., 2014) (Chen & Manning, 2014).”
Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015, background): “, 2003), selectional preference acquisition (Van de Cruys, 2014), and dependency parsing (Chen and Manning, 2014).”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, methodology): “1 using the Stanford parser [7], resulting in each word in the question being assigned a grammatical entity such as nominal subject (“nsubj”), adverb modifier (“admod”), etc.”
A Fast Unified Model for Parsing and Sentence Understanding (2016, background): “There is a fairly long history of work on building neural network-based parsers that use the core operations and data structures from transition-based parsing, of which shift-reduce parsing is a variant (Henderson, 2004; Emami and Jelinek, 2005; Titov and Henderson, 2010; Chen and Manning, 2014; Buy”
Demographic Dialectal Variation in Social Media: A Case Study of African-American English (2016, methodology): “We test the Stanford CoreNLP neural network dependency parser (Chen and Manning, 2014) using the english SD model that outputs this formalism;13 its disparity is worse.”
Joint Topic-Semantic-aware Social Recommendation for Online Voting (2017, methodology): “Word2vec as well as associated Skip-Gram model are extensively used in document classification [15], dependency parser [4], and network embedding [8].”
Recent Trends in Deep Learning Based Natural Language Processing (2017, methodology): “[133] modeled the decision making at each time step with a neural network with one hidden layer.”
Understanding and Improving Layer Normalization (2019, methodology): “the validation set (1,700 sentences, 80,234 transition examples), and section 23 as the testing set (2,416 sentences, 113,368 transition examples). We implement a MLP-based parser following the work [Chen and Manning, 2014]. The dimension of the hidden state is 512, the batch size is 1;024, the drop”
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (2016, background): “…(NLP) systems, like syntactic parsing (Nivre and Scholz, 2004; McDonald et al., 2005; Koo and Collins, 2010; Ma and Zhao, 2012a; Ma and Zhao, 2012b; Chen and Manning, 2014; Ma and Hovy, 2015) and entity coreference resolution (Ng, 2010; Ma et al., 2016), are becoming more sophisticated, in part…”
Globally Normalized Transition-Based Neural Networks (2016, background): “Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2442–2452, Berlin, Germany, August 7-12, 2016. c©2016 Association for Computational Linguistics”