Improving Neural Parsing by Disentangling Model Combination and Reranking Effects

By Daniel Fried, Mitchell Stern, Daniel Klein. 2017

40 citations recorded by the source corpus.

Recent work has proposed several generative neural models for constituency parsing that achieve state-of-the-art results. Since direct search in these generative models is difficult, they have primarily been used to rescore candidate outputs from base parsers in which decoding is more straightforward. We first present an algorithm for direct search in these generative models. We then demonstrate that the rescoring results are at least partly due to implicit model combination rather than reranking effects. Finally, we show that explicit model combination can improve performance even further, resulting in new state-of-the-art numbers on the PTB of 94.25 F1 when training only on gold data and 94.66 F1 when using external data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of the Usages of Deep Learning for Natural Language Processing (2020, background): “[121] wanted to determine whether the power of the models came from the reranking process or simply from the combined power of two models.”
An Empirical Study of Building a Strong Baseline for Constituency Parsing (2018, background): “In fact, RNNG and its extensions (Kuncoro et al., 2017; Fried et al., 2017) provide the current state-of-the-art performance.”
Global Reasoning over Database Structures for Text-to-SQL Parsing (2019, methodology): “To overcome this, we separately train a discriminative model (Collins and Koo, 2005; Ge and Mooney, 2006; Lu et al., 2008; Fried et al., 2017) to re-rank the top-K queries in the decoder’s output beam.”
Cross-Domain Generalization of Neural Constituency Parsers (2019, background): “…self-training techniques, which have also been shown to be compatible with neural constituency parsers (Dyer et al., 2016; Choe and Charniak, 2016; Fried et al., 2017; Kitaev et al., 2019) and may provide beneﬁts orthogonal to the pre-trained representations and structured models we analyze here.”
Natural Language Processing Advancements By Deep Learning: A Survey (2020, unknown): “Recurrent neural network grammars (RNNG) [83] 93.6 In-order traversal over syntactic trees + LSTM [84] 94.2 Model Combination and Reranking [85] 94.6 Self-Attentive Encoder [86] 95.1 phrasal elements and phrase-structure rules do not contribute to the process.”