Accelerated training of conditional random fields with stochastic gradient methods

By Kevin P. Murphy. International Conference on Machine Learning, 2006

355 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/950e8d556e30d2a873172bfa90f4f36da5286c07

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Learning Graphical Model Parameters with Approximate Marginal Inference (2013, methodology): “This “surrogate likelihood” [17] approximation appears to be the most widely used loss in imaging problems, with marginals approximated by either mean ﬁeld [18], [19], TRW [20] or LBP [21], [22], [23], [24], [25].”
A Stochastic Quasi-Newton Method for Online Convex Optimization (2007, methodology): “We replicate two experiments by Vishwanathan et al. (2006) which apply 1-”
Parameter Learning and Convergent Inference for Dense Random Fields (2013, methodology): “One way to deal with this is to approximate the gradient by substituting approximate inference in place of exact inference (Wainwright et al., 2003; Sutton & McCallum, 2005; Kumar et al., 2005; Vishwanathan et al., 2006; Levin & Weiss, 2009; Pal et al., 2012).”
Practical Very Large Scale CRFs (2010, methodology): “…limited memory BFGS (L-BFGS) (Liu and Nocedal, 1989), which is used in the popular CRF++ (Kudo, 2005) and CRFsuite (Okazaki, 2007) packages; conjugate gradient (Nocedal and Wright, 2006) and Stochastic Gradient Descent (SGD) (Bottou, 2004; Vishwanathan et al., 2006), used in CRFsgd (Bottou, 2007).”
Training structural SVMs when exact inference is intractable (2008, methodology): “Therefore, the partition function is approximated (Culotta et al., 2007; He et al., 2004; Kumar & Hebert, 2003; Vishwanathan et al., 2006), or the model is simpli.ed to make the partition function tractable (Sutton & McCallum, 2005), or CRF max-likelihood training is replaced with Perce”
A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction (2010, methodology): “For the baselines, we use the code, features and optimal parameters of [23].”
A Dynamic Conditional Random Field Model for Joint Labeling of Object and Scene Classes (2008, background): “Finally, the edge potentials’ parameter sets ΘΨ and ΘΛ are learned jointly in a maximum likelihood setting with stochastic meta descent [23].”
Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy (2010, methodology): “The empirical improvements of the stochastic approach have been demonstrated in maximum entropy-based models (Vishwanathan et al., 2006) and other large-scale machine learning tasks.”
Slow Learners are Fast (2009, background): “This occurs, for instance, in the case of conditional random ﬁelds (Ratliﬀ et al., 2007; Vishwanathan et al., 2006), in planning (Ratliﬀ et al., 2006), and in ranking (Weimer et al., 2008).”
A Tutorial on Energy-Based Learning (2006, methodology): “This is because stochastic gradients can take advantage of the redundancy between the samples by updating the parameters on the basis of a single sample, whereas “batch” optimization methods waste considerable resources to compute exact descent directions, often nullifying the theoretical speed adva”
Contextual classification of lidar data and building object detection in urban areas (2014, methodology): “Although this technique does not ensure convergence to the global optimum, it has been shown to provide good results in Vishwanathan et al. (2006).”
Sample size selection in optimization methods for machine learning (2012, background): “On the other hand, batch (or mini-batch) algorithms can easily exploit parallelism in the function and gradient evaluation, and are able to yield high accuracy in the solution of the optimization problem [2,31,32], if so desired.”
Social Network Data Analytics (2011, background): “Fueled by technological advances and inspired by empirical analysis, the number of such problems and the diversity of domains from which they arise – biological [47, 86, 98, 114, 118], clinical [9, 30], ecological [14, 33, 68, 69, 97], engineering [1, 75, 122], linguistic [46], scienti c [29, 42, 71”
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields (2015, methodology): “Duetothe high cost of evaluating the CRF objective function on even a single training example, it is now common to train CRFs using stochastic gradient methods [Vishwanathan et al., 2006].”