Shallow Parsing with Conditional Random Fields
By Fei Sha, Fernando C Pereira. 2003
1,567 citations recorded by the source corpus.
Conditional random fields for sequence labeling offer advantages over both generative models like HMMs and classifiers applied at each sequence position. Among sequence labeling tasks in language processing, shallow parsing has received much attention, with the development of standard evaluation datasets and extensive comparison among methods. We show here how to train a conditional random field to achieve performance as good as any reported base noun-phrase chunking method on the CoNLL task, and better than any reported single model. Improved training methods based on modern optimization algorithms were critical in achieving these results. We present extensive comparisons between models and training methods that confirm and strengthen previous results on shallow parsing and training methods for maximum-entropy models.
Subfields: Artificial Intelligence, Natural Language Processing, Computational Linguistics & Semantics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Percy Liang
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Noah A. Smith
- Christopher D. Manning
- Quoc V. Le
In Natural Language Processing
- Christopher D. Manning
- Percy Liang
- Noah A. Smith
- Mohit Bansal
- Luke S. Zettlemoyer
- Mirella Lapata
- Graham Neubig
- Yoav Goldberg
In Computational Linguistics & Semantics
- Andrew McCallum
- Michael Collins
- Ivan Titov
- Phil Blunsom
- Trevor Cohn
- Ronan Collobert
- Mark W. Schmidt
- James Richard Curran
Citations in the corpus
- Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields (2017, methodology): “Deterministic (batch) methods such as L-BFGS (Sha and Pereira, 2003; Wallach, 2002) have linear convergence rate but the cost per iteration is large.”
- Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs (2016, methodology): “Text chunking, also known as shallow parsing (Sha & Pereira, 2003), consists in dividing the input text into syntactically related non-overlapping groups of words, called phrase or chunks.”
- Natural Language Processing (Almost) from Scratch (2011, methodology): “…Accuracy Shen et al. (2007) 97.33% Toutanova et al. (2003) 97.24% Giménez and Màrquez (2004) 97.16% (a) POS System F1 Shen and Sarkar (2005) 95.23% Sha and Pereira (2003) 94.29% Kudo and Matsumoto (2001) 93.91% (b) CHUNK System F1 Ando and Zhang (2005) 89.31% Florian et al. (2003) 88.76% Kudo…”
- Hybrid Deterministic-Stochastic Methods for Data Fitting (2011, methodology): “…CRF experiments, we use the noun-phrase chunking problem from the CoNLL-2000 Shared Task [31], where the goal is to assign each word in a sentence to one of 22 possible states, and we use approximately 1.6 million features to represent the presence of words and word combinations [31, 32].”
- Domain Adaptation with Structural Correspondence Learning (2006, background): “Discriminative tag-gers and chunkers have been the state-of-the-art for more than a decade (Ratnaparkhi, 1996; Sha and Pereira, 2003).”
- Kernel methods in machine learning (2007, neutral): “• For the parametric case, Lafferty et al. [90] have employed variants of improved iterative scaling (Darroch and Ratcliff [40] and Della Pietra [46]) to optimize equation (72), whereas Sha and Pereira [122] have investigated preconditioned conjugate gradient descent and limited memory quasi-Newton ”
- Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields (2015, methodology): “, 2003], semantic role labeling [Cohn and Blunsom, 2005], topic modeling [Zhu and Xing, 2010], information extraction [Peng and McCallum, 2006], shallow parsing [Sha and Pereira, 2003], named-entity recognition [Settles, 2004], as well as a host of other applications in natural language processing a”
- Algorithmic stemmers or morphological analysisq An evaluation (2009, background): “, 2001), shallow parsing (Sha and Pereira, 2003), and named-entity recognition in the biomedical domain (Settles, 2004).”
- Large-Scale Personalized Human Activity Recognition Using Online Multitask Learning (2013, neutral): “Table 1 summarizes prior work on activity recognition.”
- Optimizing Costly Functions with Simple Constraints: A Limited-Memory Projected Quasi-Newton Algorithm (2009, positive): “We follow Sha and Pereira (2003) and use a feature vector of size 1.8M, representing things such as “did word W occur at location i ”, for each English word W that occured three or more times in the training data.”
- Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets (2004, background): “Such models are well suited to sequence analysis, and CRFs in particular have been shown to be useful in part-of-speech tagging (Lafferty et al., 2001), shallow parsing (Sha and Pereira, 2003), and named entity recognition for newswire data (McCallum and Li, 2003).”
- Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition (2011, methodology): “It is a first-order conditional HMM trained with the perceptron algo-7 sourceforge.net/projects/supersensetag rithm (Collins, 2002), a discriminative model with excellent efficiency-performance trade-off (Sha and Pereira, 2003).”
- Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study (2005, methodology): “, using iterative scaling [2], quasi-Newton methods [3], etc.”
- Efficient multiple hyperparameter learning for log-linear models (2007, methodology): “While a number of efficient procedures exist for solving the optimization problem OPT1 [34, 11], little attention is usually given to choosing an appropriate regularization matrixC.”
- Structure learning in random fields for heart motion abnormality detection (2008, methodology): “If we use α = 1, this degenerates into the standard L1/Lasso regularizer R 1 (w ) = ||w ||1 (we refer to this as L 1 L 1 ).”
- Interactive Information Extraction with Constrained Conditional Random Fields (2004, neutral): “Maximum aposteriori training of these models is effi - ciently performed by hill-climbing methods such as conjugate gradient, or its improved second-order cousin, limitedmemory BFGS (Sha & Pereira 2003).”
- Hidden conditional random fields for phone classification (2005, methodology): “L-BFGS is a well-known low-memory quasi-Newton method which has been applied successfully to the estimation of CRF parameters [14].”