Trading convexity for scalability

By Ronan Collobert, Fabian H Sinz, Jason Aaron Edward Weston, L. Eon Bottou. 2006

407 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Randomized Prediction Games for Adversarial Machine Learning (2016, background): “This argument shares similarities with the idea of addressing nonconvex machine learning problems directly, without resorting to convex approximations [37], [38].”
Satisfying Real-world Goals with Dataset Constraints (2016, background): “We can, however, work around this difficulty by training a classifier that makes randomized predictions based on the ramp function [7]:”
The landscape of empirical risk for nonconvex losses (2016, background): “Second, non-convex models are often computationally more scalable, in that the prediction is only sensitive to examples that are close to the decision boundary, and hence uses a smaller number of support vectors [8].”
Fast and simple gradient-based optimization for semi-supervised support vector machines (2014, methodology): “Similar ideas led to the continuation framework [15], to deterministic annealing methods [16, 10], and to the use of the (constrained) concaveconvex procedure [17, 18, 19].”
Residual Weighted Learning for Estimating Individualized Treatment Rules (2015, background): “The ramp loss (Collobert et al. 2006) is also called the truncated hinge loss function (Wu and Liu 2007).”
Robust support vector machines based on the rescaled hinge loss function (2017, methodology): “However, both RSVM-Ramp and RSVM-RHHQs can improve the sparseness of SVM in the presence of outliers, which is partly because that they both place much less emphasis on outliers [5,12,14].”
Support Vector Machines with the Ramp Loss and the Hard Margin Loss (2011, methodology): “[11] use optimization methods for SVM with the ramp loss that do not guarantee global optimality.”
On handling indicator constraints in mixed integer programming (2016, methodology): “In this paper, we are interested in the SVM with the ramp loss function, see [12,29].”
Analysis of Learning from Positive and Unlabeled Data (2014, methodology): “For non-convex optimization with the ramp loss, see [12, 13].”
Active Learning as Non-Convex Optimization (2009, methodology): “We have also performed some experiments tuning to minimize error and found the non-convex losses did sometimes reach lower error rates, although the differences are not often dramatic, consistent with observations of Collobert et al. (2006).”
Derivative-Free Optimization via Classification (2016, methodology): “We finally study on a classification task with Ramp loss (Collobert et al. 2006).”
Privacy-Preserving Stochastic Gradual Learning (2018, background): “In the third set of baselines, we choose four diﬀerent losses: logistic loss (convex and smooth), hinge loss (convex and non-smooth), Gompertz loss [50] (non-convex and smooth), and ramp loss [51] (non-convex and non-smooth).”
Algorithms for Direct 0-1 Loss Optimization in Binary Classification (2013, methodology): “While other works have investigated classification methods that are robust to outliers, including tlogistic regression (Ding & Vishwanathan, 2010), robust truncated hinge loss (Wu & Liu, 2007), SavageBoost (Masnadi-Shirazi & Vasconcelos, 2008), and (Collobert et al., 2006) (which shows that a non-co”
Object Detection with Grammar Models (2011, background): “In this case L′ becomes a type of ramp loss [4, 6, 14].”
Supervised classification and mathematical optimization (2013, background): “The ramp loss has also been advocated in [213, 66] with the motivation of generating less support vectors than the SVM with hinge loss.”
Robust twin support vector machine for pattern classification (2013, methodology): “Bi and Zhang derived a general statistical formulation where unobserved input is modeled as a hidden mixture component [21]; Literatures [20,22–24] employed second order cone programming (SOCP) methods to handle the missing and uncertain data; literatures [25–27] construct the robust models by the r”