Efficient Training of Conditional Random Fields

By Hanna M. Wallach. 2002

178 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields (2017, methodology): “Deterministic (batch) methods such as L-BFGS (Sha and Pereira, 2003; Wallach, 2002) have linear convergence rate but the cost per iteration is large.”
Hybrid Deterministic-Stochastic Methods for Data Fitting (2011, methodology): “Several comparison studies indicate that these type of limited-memory quasi-Newton methods are among the most efficient deterministic methods available for solving large-scale logistic-regression and CRF problems [19, 36, 21, 32].”
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields (2015, methodology): “Below we state the convergence rates of diﬀerent methods for training CRFs, including the fastest known rates for deterministic algorithms (like L-BFGS and accelerated gradient) [Nesterov, 2004], stochastic algorithms (like [averaged] stochastic gradient and AdaGrad) [Ghadimi and Lan, 2012], online ”
Shallow Parsing with Conditional Random Fields (2003, methodology): “Concurrently with the present work, Wallach (2002) tested conjugate gradient and second-order methods for CRF training, showing significant training speed advantages over iterative scaling on a small shallow parsing problem.”
An Introduction to Conditional Random Fields for Relational Learning (2007, methodology): “When such second-order methods are used, gradient-based optimization is much faster than the original approaches based on iterative scaling in Laﬀerty et al. [2001], as shown experimentally by several authors [Sha and Pereira, 2003, Wallach, 2002, Malouf, 2002, Minka, 2003].”
Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm (2004, unknown): “This technique has been shown to be very effective in a variety of NLP tasks (Malouf, 2002; Wallach, 2002).”
Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data (2004, background): “L-BFGS has previously been shown to outperform other optimization algorithms for linear-chain CRFs (Sha and Pereira, 2003; Malouf, 2002; Wallach, 2002).”
Jointly Optimizing a Two-Step Conditional Random Field Model for Machine Transliteration and Its Fast Decoding Algorithm (2010, methodology): “CRF training is usually performed through the L-BFGS algorithm (Wallach, 2002) and decoding is performed by the Viterbi algorithm.”
Modeling Latent-Dynamic in Shallow Parsing: A Latent Conditional Model with Improved Inference (2008, methodology): “As for numerical optimization (Malouf 2002; Wallach 2002), we performed gradient decent with the Limited-Memory BFGS (L-BFGS) optimization technique (Nocedal & Wright 1999).”
An Introduction to Conditional Random Fields (2010, background): “[54], as shown experimentally by several authors [108, 132, 68, 79].”
Conditional random fields for activity recognition (2007, methodology): “In addition to first order methods, an approximate second order method, Limited Memory BFGS [7], has also been used successfully [16, 12].”
Conditional Random Fields: An Introduction (2004, methodology): “Instead, maximum likelihood parameters must be identified using an iterative technique such as iterative scaling [5, 1, 10] or gradient-based methods [15, 17].”
Bayesian reasoning and machine learning (2012, unknown): “This means that the storage of the Hessian is not feasible for Newton based training and either limited memory methods or conjugate gradient techniques are typically preferred[288].”