Boosting Algorithms as Gradient Descent in Function Space

By Llew Mason, Jonathan Baxter, Peter L. Bartlett, Marcus Frean. 2007

242 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization (2013, background): “For some examples which ﬁt our setting, one may refer the problems in references [1,11,12,15,16,21,22,23,32].”
Multi-task learning for boosting with application to web search ranking (2010, background): “In particular we will follow previous literature by [14] and ensure that our solver is in fact an instance of gradient boosting [13].”
SemiBoost: Boosting for Semi-Supervised Learning (2009, background): “Conventional derivation of boosting algorithms follows the function gradient approach (Mason et al., 1999), which can also be viewed as a relaxation approach to approximate the original objective function by a linear function.”
Multiple Instance Boosting for Object Detection (2005, methodology): “To do this, we created MILBoost, a new method for folding MIL into the AnyBoost [9] framework.”
A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization (2018, background): “Moreover, in various interesting problems such as deep learning, dictionary learning, training of neural networks, and classification tasks with nonconvex activation functions, [49, 48, 8, 23, 42, 73, 33], the loss function f is nonconvex, which represents another major challenge for stochastic opti”
MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation (2014, methodology): “Similar to MIL-Boost, the formulation of MILCut-Struct can also be optimized using gradient descent in the AnyBoost framework [25, 38].”
Multi-class AdaBoost ∗ (2009, background): “Another view on boosting, which is popular in the statistical community, regards AdaBoost as a functional gradient descent algorithm [6, 11, 17].”
Confidence-Rated Multiple Instance Boosting for Object Detection (2014, background): “The classifier ϕ is trained to achieve a low empirical error rate on the bags such that max j ( 2 · 1{ϕ(xij)≥0} − 1 ) = yi.”
An Invitation to Imitation (2015, background): “This procedure won’t converge in one iteration– in fact, in Figure 8 it runs over a rock! However, multiple iterations (Figure 9 can be shown to be a gradient boosting approach [Mason et al., 1999] to descending a convex loss function that upper bounds imitation loss.”
Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization (2016, background): “5) for solving the following nonconvex support vector machine (SVM) problem with a sigmoid loss function, which has been considered in [20, 35]: min x∈Rn f(x) := Eu,v[1− tanh(v〈x, u〉)] + λ‖x‖(2)2, (4.”
Contextual Object Detection Using Set-Based Classification (2012, methodology): “We use functional gradient descent interpretation of boosting, specifically MarginBoost framework [22], in developing our learning algorithm.”
Learning (predictive) risk scores in the presence of censoring due to interventions (2015, methodology): “We use an optimization procedure similar to that of gradient boosted regression (Mason et al, 1999; Friedman, 2001) to obtain the DSS function parameters.”