Gradient boosting machines, a tutorial

By Alexey Natekin, Alois C. Knoll. Front. Neurorobot., 2013

3,482 citations recorded by the source corpus.

Gradient boosting machines are a family of powerful machine-learning techniques that have shown considerable success in a wide range of practical applications. They are highly customizable to the particular needs of the application, like being learned with respect to different loss functions. This article gives a tutorial introduction into the methodology of gradient boosting methods with a strong focus on machine learning aspects of modeling. A theoretical information is complemented with descriptive examples and illustrations which cover all the stages of the gradient boosting model design. Considerations on handling the model complexity are discussed. Three practical examples of gradient boosting applications are presented and comprehensively analyzed.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/eec471897375942fd690b736c2753bb19d907273

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation (2023, methodology): “The objective is to minimise this loss function by adding weak learners using gradient descent [119].”
Machine Learning for Neural Decoding (2017, background): “XGBoost (Extreme Gradient Boosting; Chen and Guestrin, 2016) is an implementation of gradient boosted trees (Natekin and Knoll, 2013; Chen, 2014).”
Breast Tumor Cellularity Assessment using Deep Neural Networks (2019, methodology): “instead of using the segmentation maps directly, we extract various features from them and use the gradient boosting trees (GBT) [28] to predict the cellularity score.”
Development and Evaluation of the Combined Machine Learning Models for the Prediction of Dam Inflow (2020, background): “The character of the loss function is automatically reﬂected in learning through the gradient [40].”
Compilation and physicochemical classification analysis of a diverse hERG inhibition database (2016, neutral): “GBM method relies on stepwise optimization (boosting) of an ensemble of weak predictors (decision trees) that may account for non-linear effects in the property variation [50], and for this reason it is preferred over traditional additive approaches, such as partial least squares (PLS).”
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, background): “Given the spatio-temporal characteristics of urban data, such as trafﬁc ﬂows, a line of studies integrated GNNs with various temporal learning methods to capture dynamics in both space and time dimension [3].”
A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects (2022, methodology): “can overfit and model noise if the input data is noisy [79], 450 [83].”
Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis (2018, background): “Gradient boosting models are being extensively used in machine learning due to their speed, accuracy, and robustness against overfitting [21].”
Emergency department triage prediction of clinical outcomes using machine learning models (2019, methodology): “In the gradient boosting decision tree models, we also computed the importance that is summed over iterations [32].”
Scene Classification via a Gradient Boosting Random Convolutional Network Framework (2016, background): “GBM is a useful practical tool for different predictive tasks, and it can consistently provide more accurate results than the conventional single machine learning models [14].”
Ensemble learning: A survey (2018, background): “It includes a learning procedure in which the goal is to construct the base learners so that they are maximally correlated with the negative gradient of the loss function, associated with the whole ensemble (Natekin & Knoll, 2013).”