Reranking and Self-Training for Parser Adaptation

By David McClosky, Eugene Charniak, Mark Johnson. 2006

286 citations recorded by the source corpus.

Statistical parsers trained and tested on the Penn Wall Street Journal (WSJ) treebank have shown vast improvements over the last 10 years. Much of this improvement, however, is based upon an ever-increasing number of features to be trained on (typically) the WSJ treebank data. This has led to concern that such parsers may be too finely tuned to this corpus at the expense of portability to other genres. Such worries have merit. The standard "Charniak parser" checks in at a labeled precision-recall f-measure of 89.7% on the Penn WSJ test set, but only 82.9% on the test set from the Brown treebank corpus.This paper should allay these fears. In particular, we show that the reranking parser described in Charniak and Johnson (2005) improves performance of the parser on Brown to 85.2%. Furthermore, use of the self-training techniques described in (McClosky et al., 2006) raise this to 87.8% (an error reduction of 28%) again without any use of labeled Brown data. This is remarkable since training the parser and reranker on labeled Brown data achieves only 88.4%.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Marginalized Denoising Autoencoders for Domain Adaptation (2012, background): “Examples are computational biology (Liu et al., 2008), natural language processing (Daume III, 2007; McClosky et al., 2006) and computer vision (Saenko et al., 2010).”
Co-Training for Domain Adaptation (2011, background): “Domain adaptation is of practical importance in many areas of applied machine learning, ranging from computational biology [17] to natural language processing [11, 19] to computer vision [23].”
Analysis of Representations for Domain Adaptation (2006, background): “This is quite common in natural language processing, where we might be performing the same syntactic analysis task, such as tagging or parsing, but on domains with very different vocabularies [6, 11].”
Cross-Domain Generalization of Neural Constituency Parsers (2019, background): “treebanks still transfer to out-of-domain improvements (McClosky et al., 2006).”
Using monolingual source-language data to improve MT performance (2006, background): “Self-training has been explored in other areas of NLP, such as parsing [13] and speech recognition (see below).”
Semi-supervised model adaptation for statistical machine translation (2007, methodology): “In McClosky et al. (2006b), these methods are then used to adapt a parser trained on Wall Street Journal data, without using labeled data from the latter domain.”
Self-Training for Enhancement and Domain Adaptation of Statistical Parsers Trained on Small Datasets (2007, methodology): “For the Brown corpus, we based our division on (Bacchiani et al., 2006; McClosky et al., 2006b).”
UDALM: Unsupervised Domain Adaptation through Language Modeling (2021, methodology): “Pseudo-labeling techniques are semi-supervised algorithms that either use the same model (self-training) (Yarowsky, 1995; McClosky et al., 2006; Abney, 2007) or multiple ensembles of models (tri-training) (Zhou and Li, 2005; Søgaard, 2010) in order to produce pseudo-labels for the target unlabeled…”
Transfer learning using computational intelligence: A survey (2015, methodology): “Self-training methods have been applied to domain adaptation on Natural Language Processing (NLP) tasks including parsing [18-21]; part-of-speech tagging [22]; conversation summarization [23]; entity recognition [22, 24, 25]; sentiment classification [26]; spam detection [22]; cross-language…”
Improved Inference for Unlexicalized Parsing (2007, background): “It is interesting to note that these grammars capture many of the “structural zeros” described by Mohri and Roark (2006) and pruning rules with probability below e − 10 reduces the grammar size drastically without inﬂuenc-ing parsing performance.”
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, methodology): “Self-training (Yarowsky, 1995; McClosky et al., 2006b) is one of the earliest and simplest bootstrapping approaches.”
The Cascade Transformer: an Application for Efficient Answer Sentence Selection (2020, unknown): “Our apporach also shares some similarities with student-teacher (ST) approaches for self-training (Yarowsky, 1995; McClosky et al., 2006).”
Deep Contextualized Self-training for Low Resource Dependency Parsing (2019, background): “For constituency parsing, selftraining has shown to improve linear parsers both when large training data is available (McClosky et al., 2006a,b), and in the lightly supervised and the cross-domain setups (Reichart and Rappoport, 2007).”
Unsupervised Domain Adaptive Re-Identification: Theory and Practice (2018, unknown): “Moreover, since our guarantee is built on well extracted features from real images, the encoder, i.e. feature extractor, is trained via a novel self-training framework, which is originally proposed for NLP tasks [19, 20].”
Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples (2018, methodology): “Much of this work (McClosky et al., 2006b; Reichart and Rappoport, 2007; Sagae and Tsujii, 2007; Kawahara and Uchimoto, 2008; McClosky et al., 2010; Sagae, 2010; Baucom et al., 2013; Yu et al., 2015) has focused on how to best use co-training (Blum and Mitchell, 1998) or self-training to augment a s”
Iterative parameter mixing for distributed large-margin training of structured predictors for natural language processing (2015, background): “Distributed Training Studies McDonald et al. (2010) investigate the training of a nonprojective dependency parser for Czech (with about 70,000 training examples, Hajic et al.”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, methodology): “…2009) such as bootstrapping methods like self-training, co-training and tri-training or methods such as temporal ensembling (Charniak, 1997; McClosky et al., 2006; Blum and Mitchell, 1998; Steedman et al., 2003; Zhou and Li, 2005; Søgaard and Rishøj, 2010; Saito et al., 2017; Laine and…”