Stochastic gradient boosting

By Jerome H. Friedman. 2002

6,637 citations recorded by the source corpus.

Subfields: Computational Mechanics, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

In Optimization & Training

Citations in the corpus

Machine Learning Methods That Economists Should Know About (2019, methodology): “The algorithms then use stochastic gradient descent (Friedman [2002], Bottou [1998, 2012]) described in Section 2.”
Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking (2017, background): “We also set the maximum depth of the tree d to the total For GBDT , we use four values of the number K of base trees in the ensemble, with K = { 10 , 100 , 500 , 1,000 } in combination with the learning rates α , 0.001, 0.01, 0.05, 0.1, 1.”
Distilling Knowledge from Deep Networks with Applications to Healthcare Domain (2015, methodology): “Gradient boosting [15, 16] is a method which takes an ensemble of weak learners, usually decision trees, to optimize a differentiable loss function by stages.”
Interpretable classification models for recidivism prediction (2015, background): “t to our work, much of the research to date has focused on improving predictive accuracy by training powerful black-box models such as random forests (Breiman, 2001b) and stochastic gradient boosting Friedman (2002). Random forests (Breiman, 2001b), in particular, have been used for several criminol”
Mathematical optimization in classification and regression trees (2021, background): “Gradient Boosting Machines (GBMs) is also an iterative procedure based on combining weak tree learners but usually deeper than in AdaBoost.”
XGBoost: A Scalable Tree Boosting System (2016, methodology): “The first technique is shrinkage introduced by Friedman [11].”
Quantitative missense variant effect prediction using large-scale mutagenesis data (2017, positive): “Envision uses a stochastic gradient boosting learning algorithm, which excels at analyzing nonlinear interactions between features and has performed well in a myriad of regression tasks (Friedman, 2002).”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “blems related to overtting [35,68]. Mitchell et al. [51] provided a theoretical explanation for it. Boosting is one classical way to build an ensemble, e.g., AdaBoost [16] and Gradient Tree Boosting [17]. Stacking combines multiple models by learning a combiner and it applies to both tasks in superv”
Evidence for the Higgs-boson Yukawa coupling to tau leptons with the ATLAS detector (2015, neutral): “Boosting is a method which improves the performance and stability of decision trees and involves the combination of many trees into a single final discriminant [25, 26].”
Ecology of butterflies in Europe (2009, methodology): “xi × xj ) of the predictors are used to deal with departures from these assumptions (Guisan et al., 2002). There are many examples for multiple linear regression models regarding butterflies: Kuussaari et al. (1996) applied multiple linear regressions to model the effect of patch characteristics on ”
Machine Learning Based Intrusion Detection Systems for IoT Applications (2019, background): “While, in the case of AUC metric there are no pairs which are highly signiﬁcant, whilst GBM-CART and XGB-CART are the only less signiﬁcant pairs among all the classiﬁer pairs, every other pair is not signiﬁcant.”
Learning by Tracking: Siamese CNN for Robust Target Association (2016, methodology): “Finally, both sets of features are combined using a GB classifier [24] to produce the final prediction (see Fig.”
Random forests and stochastic gradient boosting for predicting tree canopy cover: Comparing tuning processes and model performance (2016, methodology): “SGB (Friedman et al. 2000; Friedman 2001, 2002) is now gaining recognition in ecological modeling.”
Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, neutral): “Friedman (2002) incorporated subsampling in each iteration and empirically showed signiﬁcant improvement in predictive accuracy.”
Predicting tree species presence and basal area in Utah: A comparison of stochastic gradient boosting, generalized additive models, and tree-based methods (2006, background): “Stochastic gradient boosting (Friedman, 2001, 2002) is related to both boosting and bagging.”
Machine learning based approaches for detecting COVID-19 using clinical text data (2020, methodology): “The randomly selected subsample is then used instead of the full sample to fit the base learner [22].”