Neural Probabilistic Model for Non-projective MST Parsing

By Xuezhe Ma, Eduard H. Hovy. 2017

45 citations recorded by the source corpus.

In this paper, we propose a probabilistic parsing model that defines a proper conditional probability distribution over non-projective dependency trees for a given sentence, using neural representations as inputs. The neural network architecture is based on bi-directional LSTMCNNs, which automatically benefits from both word- and character-level representations, by using a combination of bidirectional LSTMs and CNNs. On top of the neural network, we introduce a probabilistic structured layer, defining a conditional log-linear model over non-projective trees. By exploiting Kirchhoff’s Matrix-Tree Theorem (Tutte, 1984), the partition functions and marginals can be computed efficiently, leading to a straightforward end-to-end model training procedure via back-propagation. We evaluate our model on 17 different datasets, across 14 different languages. Our parser achieves state-of-the-art parsing performance on nine datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Eduard H. Hovy
3. Xuezhe Ma
4. Nanyun Peng
5. Zecong Hu
6. Jingzhou Liu
7. Mohit Bansal
8. Pengfei Liu

Citations in the corpus

• Semi-Supervised Sequence Modeling with Cross-View Training (2018, neutral): “(Chiu and Nichols, 2016) 91.62 ± 0.33 86.28 ± 0.26 ID-CNN-CRF (Strubell et al., 2017) 90.65 ± 0.15 86.84 ± 0.19 Tri-Trained LSTM (Lewis et al., 2016) 94.7 Shortcut LSTM (Wu et al., 2017) 95.08 97.53 JMT* (Hashimoto et al., 2017) 95.77 97.55 94.67 92.90 LM-LSTM-CNN-CRF (Liu et al., 2017) 95.96 ± 0.08”
• Stack-Pointer Networks for Dependency Parsing (2018, background): “, 2016) and the neural MST parser (NeuroMST) (Ma and Hovy, 2017) for comparison.”
• Deep Multitask Learning for Semantic Dependency Parsing (2017, unknown): “Our work is perhaps closest to those who used BiLSTMs to encode inputs (Kiperwasser and Goldberg, 2016; Kuncoro et al., 2016; Wang and Chang, 2016; Dozat and Manning, 2017; Ma and Hovy, 2016).”