An Effective Neural Network Model for Graph-based Dependency Parsing

By Wenzhe Pei, Tao Ge, Baobao Chang. 2015

62 citations recorded by the source corpus.

Most existing graph-based parsing models rely on millions of hand-crafted features, which limits their generalization ability and slows down the parsing speed. In this paper, we propose a general and effective Neural Network model for graph-based dependency parsing. Our model can automatically learn high-order feature combinations using only atomic features by exploiting a novel activation function tanhcube. Moreover, we propose a simple yet effective way to utilize phrase-level information that is expensive to use in conventional graph-based parsers. Experiments on the English Penn Treebank show that parsers based on our model perform better than conventional graph-based parsers.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification (2015, background): “…different activation functions in the convolution layer, including: ReLU (as per the baseline conﬁguration), hyperbolic tangent (tanh), Sigmoid function (Maas et al., 2013), SoftPlus function (Dugas et al., 2001), Cube function (Chen and Manning, 2014), and tanh cube function (Pei et al., 2015).”
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations (2016, methodology): “For example, the work of (Chen and Manning, 2014) uses 18 different elements in its feature function, while the work of (Pei et al., 2015) uses 21 different elements.”
A Primer on Neural Network Models for Natural Language Processing (2015, methodology): “A hinge based approached was used by Pei et al (2015) for arc-factored dependency parsing, and the probabilistic approach by Durrett and Klein (Durrett & Klein, 2015) for a CRF constituency parser.”
Easy-First Dependency Parsing with Hierarchical Tree LSTMs (2016, background): “…representations, there are four dominant approaches: search based parsers that use local features that are fed to a neural-network classifier (Pei et al., 2015; Durrett and Klein, 2015); greedy transition based parsers that use local features that are fed into a neuralnetwork classifier…”
Deep Multitask Learning for Semantic Dependency Parsing (2017, methodology): “Local structure scoring was traditionally done with linear models over hand-engineered features, but lately, various forms of representation learning have been explored to learn feature combinations (Lei et al., 2014; Taub-Tabib et al., 2015; Pei et al., 2015, inter alia).”