Supervised and unsupervised PCFG adaptation to novel domains

By Brian Roark, Michiel Bacchiani. 2003

110 citations recorded by the source corpus.

This paper investigates adapting a lexicalized probabilistic context-free grammar (PCFG) to a novel domain, using maximum a posteriori (MAP) estimation. The MAP framework is general enough to include some previous model adaptation approaches, such as corpus mixing in Gildea (2001), for example. Other approaches falling within this framework are more effective. In contrast to the results in Gildea (2001), we show F-measure parsing accuracy gains of as much as 2.5% for high accuracy lexicalized parsing through the use of out-of-domain treebanks, with the largest gains when the amount of indomain data is small. MAP adaptation can also be based on either supervised or unsupervised adaptation data. Even when no in-domain treebank is available, unsupervised techniques provide a substantial accuracy gain over unadapted grammars, as much as nearly 5% F-measure improvement.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Agnostic Federated Learning (2019, background): “…et al., 2018), speech recognition (Legetter and Woodland, 1995; Gauvain and Chin-Hui, 1994; Pietra et al., 1992; Rosenfeld, 1996; Jelinek, 1998; Roark and Bacchiani, 2003), and computer vision (Mart´ınez, 2002) A problem more closely related to that of federated learning is that of…”
Domain Adaptation: Learning Bounds and Algorithms (2009, methodology): “…Blitzer et al., 2007; Jiang & Zhai, 2007; Chelba & Acero, 2006; Daumé III & Marcu, 2006), speech processing (Legetter & Woodland, 1995; Gauvain & Chin-Hui, 1994; Pietra et al., 1992; Rosenfeld, 1996; Jelinek, 1998; Roark & Bacchiani, 2003), computer vision (Martı́nez, 2002), and many other areas.”
Domain Adaptation with Structural Correspondence Learning (2006, methodology): “In such cases, we must take steps to adapt a model trained on the source domain for use in the target domain (Roark and Bacchiani, 2003; Florian et al., 2004; Chelba and Acero, 2004; Ando, 2004; Lease and Char-niak, 2005; Daum´e III and Marcu, 2006).”
Multiple Source Adaptation and the Rényi Divergence (2009, background): “This problem arises in a variety of natu-ral language processing tasks such parsing, statistical language modeling, text classiﬁcation [15, 7, 16, 9, 10, 4, 6], or speech processing [11, 8, 14] and computer vision [13] tasks, as well as in many other applications.”
Transfer learning using computational intelligence: A survey (2015, methodology): “Self-training methods have been applied to domain adaptation on Natural Language Processing (NLP) tasks including parsing [18-21]; part-of-speech tagging [22]; conversation summarization [23]; entity recognition [22, 24, 25]; sentiment classification [26]; spam detection [22]; cross-language documen”
Model adaptation for spoken language understanding (2005, background): “Another study is about supervised and unsupervised adaptation of probabilistic context free grammars to a new domain using again MAP adaptation [8].”
Pivot Based Language Modeling for Improved Neural Domain Adaptation (2018, background): “(Roark and Bacchiani, 2003; Chelba and Acero, 2004; Daumé III and Marcu, 2006)).”
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy (2010, methodology): “In [24], Roark and Bacchiani used source-domain data to construct a Dirichlet prior for MAP estimation of the target domain.”
PADA: A Prompt-based Autoregressive Approach for Adaptation to Unseen Domains (2021, background): “…on various scenarios where the target domain is known at training time (e.g. through labeled or unlabeled data) but is yet under-represented (Roark and Bacchiani, 2003; Daumé III and Marcu, 2006; Jiang and Zhai, 2007; McClosky et al., 2010; Rush et al., 2012; Schn-a abel and Schütze, 2014).”
Spoken language understanding using the Hidden Vector State Model (2006, methodology): “In recent years, MAP adaptation has been successfully applied to n-gram language models (Bacchiani and Roark, 2003) and lexicalized PCFG models (Roark and Bacchiani, 2003).”
The CoNLL 2007 Shared Task on Dependency Parsing (2007, methodology): “This includes the work of Roark and Bacchiani (2003), Flo-rian et al. (2004), Chelba and Acero (2004), Daum´e and Marcu (2006), and Titov and Henderson (2006).”