Parsing as Language Modeling

By Do Kook Choe, Eugene Charniak. 2016

121 citations recorded by the source corpus.

We recast syntactic parsing as a language modeling problem and use recent advances in neural network language modeling to achieve a new state of the art for constituency Penn Treebank parsing — 93.8 F 1 on section 23, us-ing 2-21 as training, 24 as development, plus tri-training. When trees are converted to Stanford dependencies, UAS and LAS are 95.9% and 94.1%.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semi-Supervised Sequence Modeling with Cross-View Training (2018, methodology): “For dependency parsing, we omit results from Choe and Charniak (2016), Kuncoro et al. (2017), and Liu and Zhang (2017) because these train constituency parsers and convert the system outputs to dependency parses.”
Direct Output Connection for a High-Rank Language Model (2018, background): “Neural language models also positively influence syntactic parsing (Dyer et al., 2016; Choe and Charniak, 2016).”
The Marginal Value of Adaptive Gradient Methods in Machine Learning (2017, methodology): “Choe and Charniak [1] show that constituency parsing can be cast as a language modeling problem, with trees being represented by their depth-first traversals.”
What Do Recurrent Neural Network Grammars Learn About Syntax? (2016, background): “Vinyals et al. (2015) and Choe and Charniak (2016) must learn implicitly, plays a crucial role in the RNNG’s generalization success.”
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better (2018, methodology): “Rather surprisingly, syntactic LSTM language models without explicit composition have no advantage over sequential LSTMs that operate on word sequences, although these models can nevertheless be excellent predictors of phrase structures (Choe and Charniak, 2016).”
Multilingual Constituency Parsing with Self-Attention and Pre-Training (2018, neutral): “3 Choe and Charniak (2016) – – 93.”
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies (2019, methodology): “, 2016) and a incrementalized version of the Parsing-as-Language-Modeling configuration from Charniak et al. (2016). Models are tested on a diverse range of configurations for two classes of non-local grammatical dependencies in English—Negative Polarity licensing and Filler–Gap Dependencies.”
Cross-Domain Generalization of Neural Constituency Parsers (2019, background): “…additional ensembling or self-training techniques, which have also been shown to be compatible with neural constituency parsers (Dyer et al., 2016; Choe and Charniak, 2016; Fried et al., 2017; Kitaev et al., 2019) and may provide beneﬁts orthogonal to the pre-trained representations and…”
Learning Structured Natural Language Representations for Semantic Parsing (2017, background): “This property makes FunQL logical forms natural to be generated with recurrent neural networks (Vinyals et al., 2015; Choe and Charniak, 2016; Dyer et al., 2016).”
In-Order Transition-based Constituent Parsing (2017, unknown): “Furthermore, our ﬁnal model obtains a 93.6 F 1 with supervised reranking (Choe and Charniak, 2016) and a 94.2 F 1 with semi-supervised reranking, achieving the state-of-the-art results on constituent parsing on English benchmark.”
Structured Prediction as Translation between Augmented Natural Languages (2021, methodology): “Our approach is also related to previous works that use seq2seq approaches to perform parsing (Vinyals et al., 2015; Dyer et al., 2016; Choe & Charniak, 2016; Rongali et al., 2020), with the main difference that we propose a general framework that uses augmented natural languages as a way to unify…”
Improving Neural Parsing by Disentangling Model Combination and Reranking Effects (2017, background): “Recent work on neural constituency parsing (Dyer et al., 2016; Choe and Charniak, 2016) has found multiple cases where generative scoring models for which inference is complex outperform base models for which inference is simpler.”
A Survey of the Usages of Deep Learning for Natural Language Processing (2020, methodology): “Choe and Charniak [120] treated parsing as a language modeling problem and used an LSTM to assign probabilities to the parse trees, achieving state of the art.”
On the Practical Computational Power of Finite Precision RNNs for Language Recognition (2018, unknown): “For example, processing linearized parse trees (Vinyals et al., 2015; Choe and Charniak, 2016; Aharoni and Goldberg, 2017) requires counting brackets and nesting levels.”