Wide-Coverage Efficient Statistical Parsing with CCG and Log-Linear Models

By Stephen Clark, James Richard Curran. Computational Linguistics, 2007

463 citations recorded by the source corpus.

This article describes a number of log-linear parsing models for an automatically extracted lexicalized grammar. The models are full parsing models in the sense that probabilities are defined for complete parses, rather than for independent events derived by decomposing the parse tree. Discriminative training is used to estimate the models, which requires incorrect parses for each sentence in the training data as well as the correct parse. The lexicalized grammar formalism used is Combinatory Categorial Grammar (CCG), and the grammar is automatically extracted from CCGbank, a CCG version of the Penn Treebank. The combination of discriminative training and an automatically extracted grammar leads to a significant memory requirement (up to 25 GB), which is satisfied using a parallel implementation of the BFGS optimization algorithm running on a Beowulf cluster. Dynamic programming over a packed chart, in combination with the parallel implementation, allows us to solve one of the largest-scale estimation problems in the statistical parsing literature in under three hours. A key component of the parsing system, for both training and testing, is a Maximum Entropy supertagger which assigns CCG lexical categories to words in a sentence. The supertagger makes the discriminative training feasible, and also leads to a highly efficient parser. Surprisingly, given CCG's spurious ambiguity, the parsing speeds are significantly higher than those reported for comparable parsers in the literature. We also extend the existing parsing techniques for CCG by developing a new model and efficient parsing algorithm which exploits all derivations, including CCG's nonstandard derivations. This model and parsing algorithm, when combined with normal-form constraints, give state-of-the-art accuracy for the recovery of predicate-argument dependencies from CCGbank. The parser is also evaluated on DepBank and compared against the RASP parser, outperforming RASP overall and on the majority of relation types. The evaluation on DepBank raises a number of issues regarding parser evaluation. This article provides a comprehensive blueprint for building a wide-coverage CCG parser. We demonstrate that both accurate and highly efficient parsing is possible with CCG.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2d45f21c9deb17987a6be71b3c9a2758791540a2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Lexical Generalization in CCG Grammar Induction for Semantic Parsing (2011, methodology): “This is a difficult learning problem, since the CCG analyses that 1A related tactic is commonly used in wide-coverage CCG parsers derived from treebanks, such as work by Hockenmaier and Steedman (2002) and Clark and Curran (2007).”
Cross-Caption Coreference Resolution for Automatic Image Understanding (2010, methodology): “Syntactic Role: We parsed our captions with the C&C parser (Clark and Curran, 2007), and record whether the word appears as a direct object of a verb, as the object of a preposition, as the subject of the sentence, or as a modiﬁer.”
Inducing Probabilistic CCG Grammars from Logical Form with Higher-Order Unification (2010, methodology): “Our approach discriminates between analyses using a log-linear CCG parsing model, similar to those used in previous work (Clark & Curran, 2003, 2007), but differing in that the syntactic parses are treated as a hidden variable during training, following the approach of Zettlemoyer & Collins (2005,…”
Unbounded Dependency Recovery for Parser Evaluation (2009, methodology): “The parser uses a log-linear probability model to select the highest-scoring derivation (Clark and Curran, 2007).”
Broad-coverage CCG Semantic Parsing with AMR (2015, methodology): “Given a sentence x , we use a weighted log-linear CCG (Lafferty et al., 2001; Clark and Curran, 2007) to rank the space of possible parses under the grammar Λ .”
A* CCG Parsing with a Supertag and Dependency Factored Model (2017, methodology): “In the CCG parsing literature, some work optimizes a dependency model, instead of supertags or a derivation (Clark and Curran, 2007; Xu et al., 2014).”
Scaling Semantic Parsers with On-the-Fly Ontology Matching (2013, methodology): “The first parsing stage uses a probabilistic combinatory categorial grammar (CCG) (Steedman, 2000; Clark and Curran, 2007) to map sentences to new, underspecified logical-form meaning representations containing generic logical constants that are not tied to any specific ontology.”
Syntactic Processing Using the Generalized Perceptron and Beam Search (2011, methodology): “Then in the subsequent sections we describe each task in turn, based on conference papers including Zhang and Clark (2007, 2008a, 2008b, 2009, 2010), presented in our single coherent framework.”
Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus (2011, unknown): “C&C (Clark and Curran, 2007) is a statistical CCG parser.”
Large-scale Semantic Parsing without Question-Answer Pairs (2014, methodology): “We convert the output of an open-domain combinatory categorial grammar (CCG) parser (Clark and Curran, 2007) into a graphical representation and subsequently map it onto Freebase.”
Bootstrapping Semantic Parsers from Conversations (2011, background): “A weighted linear CCG (Clark and Curran, 2007) provides a ranking on the space of possible parses under the grammar, which can be used to select the best logical form for a sentence.”
Weakly Supervised Training of Semantic Parsers (2012, background): “Similar rules for noun compounds are used in other CCG parsers (Clark and Curran, 2007).”
Large-scale Semantic Parsing via Schema Matching and Lexicon Extension (2013, background): “Inference (parsing) and parameter estimation are driven by standard dynamic programming algorithms (Clark and Curran, 2007), while lexicon induction is based on a novel search procedure through the space of possible higher-order logic unification operations that yield the desired logical form for a…”
Natural Language Inference with Monotonicity (2019, methodology): “We have tested two commonly used CCG parsers, C&C (Clark and Curran, 2007) and EasyCCG (Lewis and Steedman, 2014).”