LSTM CCG Parsing

By Michael Lewis, Kenton C. T. Lee, Luke S. Zettlemoyer. 2016

86 citations recorded by the source corpus.

We demonstrate that a state-of-the-art parser can be built using only a lexical tagging model and a deterministic grammar, with no explicit model of bi-lexical dependencies. Instead, all dependencies are implicitly encoded in an LSTM supertagger that assigns CCG lexical categories. The parser significantly outperforms all previously published CCG results, supports efficient and optimal A decoding, and benefits substantially from semisupervised tri-training. We give a detailed analysis, demonstrating that the parser can recover long-range dependencies with high accuracy and that the semi-supervised learning enables significant accuracy gains. By running the LSTM on a GPU, we are able to parse over 2600 sentences per second while improving state-of-the-art accuracy by 1.1 F1 in domain and up to 4.5 F1 out of domain.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Semi-Supervised Sequence Modeling with Cross-View Training (2018, neutral): “(Chiu and Nichols, 2016) 91.62 ± 0.33 86.28 ± 0.26 ID-CNN-CRF (Strubell et al., 2017) 90.65 ± 0.15 86.84 ± 0.19 Tri-Trained LSTM (Lewis et al., 2016) 94.7 Shortcut LSTM (Wu et al., 2017) 95.08 97.53 JMT* (Hashimoto et al., 2017) 95.77 97.55 94.67 92.90 LM-LSTM-CNN-CRF (Liu et al., 2017) 95.96 ± 0.08”
Targeted Syntactic Evaluation of Language Models (2018, methodology): “We combine language modeling with CCG supertagging, a task that predicts for each word in the sentence its CCG supertag (Bangalore and Joshi, 1999; Lewis et al., 2016).”
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations (2016, methodology): “Then, a feature func- 4While the BiLSTM computation is quite efficient as it is, as demonstrated by Lewis et al (2016), if using a GPU implementation the BiLSTM encoding can be efficiently performed over many of sentences in parallel, making its computation cost almost negligible. tion φ(c)…”
Human-in-the-Loop Parsing (2016, methodology): “Experimental Setup We use Lewis et al. (2016)’s state-of-the-art CCG parser for our baseline.”
In-Order Transition-based Constituent Parsing (2017, unknown): “Neural networks have achieved the state-of-the-art for parsing under various grammar formalisms, including dependency (Dozat and Manning, 2017) constituent (Dyer et al., 2016; Kuncoro et al., 2017) and CCG parsing (Xu, 2016; Lewis et al., 2016).”
Shortcut Sequence Tagging (2017, unknown): “Dev Test Clark and Curran (2007) 91.5 92.0 MLP (Lewis et al. (2014)) 91.3 91.6 Bi-LSTM (Lewis et al. (2016)) 94.1 94.3 Elman-RNN (Xu et al. (2015)) 93.1 93.0 Bi-RNN (Xu et al. (2016)) 93.49 93.52 Bi-LSTM (Vaswani et al. (2016)) 94.24 94.5 9-stacked Bi-LSTM (Wu et al. (2016b) different depths.”
A* CCG Parsing with a Supertag and Dependency Factored Model (2017, methodology): “Inspect-ing the details, our supertagger runs slower than those of neuralccg and EasySRL reimpl , while in A* search our parser processes over 7 times more sentences than neuralccg .”