Cross-Domain Generalization of Neural Constituency Parsers

By Daniel Fried, Nikita Kitaev, Daniel Klein. 2019

40 citations recorded by the source corpus.

Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing—but to what degree do they generalize to other domains? We present three results about the generalization of neural parsers in a zero-shot setting: training on trees from one corpus and evaluating on out-of-domain corpora. First, neural and non-neural parsers generalize comparably to new domains. Second, incorporating pre-trained encoder representations into neural parsers substantially improves their performance across all domains, but does not give a larger relative improvement for out-of-domain treebanks. Finally, despite the rich input representations they learn, neural parsers still benefit from structured output prediction of output trees, yielding higher exact match accuracy and stronger generalization both to larger text spans and to out-of-domain corpora. We analyze generalization on English and Chinese corpora, and in the process obtain state-of-the-art parsing results for the Brown, Genia, and English Web treebanks.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Pretrained Transformers Improve Out-of-Distribution Robustness (2020, background): “and past work show that increases in accuracy do not directly translate to reduced IID/OOD generalization gaps (Hendrycks and Dietterich, 2019; Fried et al., 2019).”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “…in NLP (Blitzer et al., 2006; Daumé III, 2007; Axelrod et al., 2011), and has recently been addressed for neural LMs and transfer learning (Fried et al., 2019; Oren et al., 2019; Gururangan et al., 2020), where pretrained LMs lead to substantial improvements and increased robustness…”
Pitfalls of Static Language Modelling (2021, background): “…primarily to domain shifts, has a long history in NLP (Blitzer et al., 2006; Daum´e III, 2007; Axelrod et al., 2011), and has recently been addressed in the context of neural LMs and transfer learning (Fried et al., 2019; Oren et al., 2019; Hendrycks et al., 2020; Gururangan et al., 2020).”