Corpus Variation and Parser Performance

By Daniel Gildea. Conference on Empirical Methods in Natural Language Processing, 2001

327 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/ee7e21dd09949a5a53b39c13fca9cd3d55e2bc50

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Accurate Unlexicalized Parsing (2003, background): “More recently, Gildea (2001) discusses how taking the bilexical probabilities out of a good current lexicalized PCFG parser hurts performance hardly at all: by at most 0.5% for test text from the same domain as the training data, and not at all for test text from a different domain.”
The CoNLL 2007 Shared Task on Dependency Parsing (2007, background): “One well known characteristic of data-driven parsing systems is that they typically perform much worse on data that does not come from the training domain (Gildea, 2001).”
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy (2010, background): “Hwa [22] and Gildea [23] proved that simple techniques based on using adequately selected subsets of source-domain data and parameter pruning can improve the performance on unlabeled target data.”
Learning a Compositional Semantic Parser using an Existing Syntactic Parser (2009, background): “The performance of a syntactic parser trained only on the Wall Street Journal (WSJ) can degrade dramatically in new domains due to corpus variation (Gildea, 2001).”
QuestionBank: Creating a Corpus of Parse-Annotated Questions (2006, methodology): “Previous work on parser performance and domain variation by Gildea (2001) showed that by training a parser on the Penn-II Treebank and testing on the Brown corpus, parser accuracy drops by 5.7% compared to parsing the Wall Street Journal (WSJ) based Penn-II Treebank Section 23.”
Parsing the WSJ Using CCG and Log-Linear Models (2004, result): “Hockenmaier also found the dependencies to be very beneﬁcial — in contrast to recent results from the lexicalised PCFG parsing literature (Gildea, 2001) — but did not gain from the use of distance measures.”
Reranking and Self-Training for Parser Adaptation (2006, methodology): “We have already seen the results (Gildea, 2001) and (Bacchiani et al., 2006) achieve in Table 1.”
Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques (2002, background): “The lower reduction in error rate relative to the upper bound for DR evaluation on the Brown corpus can be attributed to a corpus effect that has also been observed by Gildea (2001) for training and testing PCFGs on the WSJ and Brown corpora.”
Unbounded Dependency Recovery for Parser Evaluation (2009, background): “Second, it is well known that the accuracy of parsers trained on the Penn Treebank degrades when they are applied to different genres and domains (Gildea, 2001).”
Supervised and unsupervised PCFG adaptation to novel domains (2003, methodology): “Johnson and Riezler (2000) looked at adding features to a maximum entropy model for stochastic unification-based grammars (SUBG), from corpora that are not annotated with the SUBG, but rather with simpler treebank annotations for which there are much larger treebanks.”
Effective Self-Training for Parsing (2006, background): “Gildea (2001) and Bacchiani et al. (2006) show that out-of-domain training data can improve parsing accuracy.”
Intricacies of Collins’ Parsing Model (2004, background): “However, Gildea (2001) (cid:4) (cid:230) (cid:4) (cid:230) longer had the top level of context that included the head word (he removed (cid:147)Back-off level 0(cid:148), as depicted in Figure 11).”
Domain Adaptation for Statistical Classifiers (2006, methodology): “Also in the parsing domain, Hwa (1999) and Gildea (2001) have shown that simple techniques based on using carefully chosen subsets of the data and parameter pruning can improve the performance of an adapted parser.”
Cross-Domain Generalization of Neural Constituency Parsers (2019, background): “…known to perform better in their training domains than on out-of-domain corpora, their out-of-domain performance degrades in well-understood ways (Gildea, 2001; Petrov and Klein, 2007), and improvements in performance on in-domain treebanks still transfer to out-of-domain improvements (McClosky…”
Solving logic puzzles: From robust processing to precise semantics (2004, result): “…set genre can degrade parse accuracy, and that small amounts of correct-genre data can be more important than large amounts of wrong-genre data (Gildea, 2001); we have found corroborating evidence in misparsings of noun phrases common in puzzle texts, such as “Sculptures C and E”, which do not…”