A scalable modular convex solver for regularized risk minimization

By Choon Hui Teo, Alex Smola, S. V. N. Vishwanathan, Quoc V. Le. 2007

199 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Statistics and Probability

In Optimization & Training

Citations in the corpus

Barrier Frank-Wolfe for Marginal Inference (2015, methodology): “This is a generalization of the technique used in Teo et al. (2007) (also used in the context of Frank-Wolfe in the proof of Theorem C.4 in Lacoste-Julien et al. (2013)).”
Block-Coordinate Frank-Wolfe Optimization for Structural SVMs (2012, background): “This is a standard recurrence inequality which appeared for example in Joachims et al. (2009, Theorem 5, see their Equation (23)) or in the appendix of Teo et al. (2007).”
Minimizing finite sums with the stochastic average gradient (2013, background): “[61], and non-smooth loss functions (or regularizers) can also be put in this framework by using smooth approximations (for example, see [43]).”
Learning Optimized Risk Scores (2016, background): “Note that (i) and (ii) are well-known benefits of cutting plane algorithms for convex empirical risk minimization problems (see e.g., Teo et al., 2007).”
Optimization for machine learning (2010, background): “been extensively studied on a variety of real-life problems in domains ranging from the text classification, bioinformatics and computer vision to computer security systems (Teo et al., 2007; Franc and Sonnenburg, 2008, 2010; Teo et al., 2010).”
Large-scale distributed L-BFGS (2017, background): “Some works have explored the scaling of optimization algorithms to build largescale models with numerous parameters through distributed computing and parallelization [9, 18, 20, 21].”
Feature Weighting for Improved Classifier Robustness (2009, background): “4: while not converged do 5: Get current weight vector ¯ w 6: for each instance (x, y) ∈ D do 7: for i = 1 to d do 8: //Minimize the value of yw T x by scaling each 9: //feature xi of x by 1 or 1/si 10: Let u * = arg min u∈{1,s i } yiwixi/u 11: Let ¯ xi = xiu * 12: end for 13: end for 14: Let ¯ D be”
Tighter Bounds for Structured Estimation (2008, background): “This has two benefits: firstly, the problem has no local minima, and secondly, the optimization problem is continuous and piecewise differentiable, which allows for effective optimization [17] [19] [20].”
Scaling Distributed Machine Learning with the Parameter Server (2014, unknown): “[46] called distributed subgradient descent.”
Discriminative Models for Multi-Class Object Layout (2009, methodology): “We briefly outline our approach here using the notation from [21].”
Large Scale Distributed Deep Networks (2012, background): “In the convex case, distributing gradient computation [18] is the naturally first step, but sometimes suffers slowdowns due to synchronization issues.”
A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets (2012, methodology): “An extensive list of convex loss functions used in machine learning is given by Teo et al. [2007], and we can even include non-smooth loss functions (or regularizers) by using smooth approximations.”
StatSnowball: a statistical approach to extracting entity relationships (2009, methodology): “This l1-norm regularized MLE problem yields a sparse estimate by setting some components of w to exact zeros [24, 11] and has efficient solvers, such as the Orthant-Wise Limited-memory Quasi-Newton (OWL-QN) method [2].”