Boosting Algorithms: Regularization, Prediction and Model Fitting. Rejoinder.

By Peter Buhlmann, Torsten Hothorn. 2007

999 citations recorded by the source corpus.

We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Fused Lasso Additive Model (2014, background): “Flexible additive modeling in high dimensions has been an active area of research in recent years (Bühlmann and Yu 2003; Sardy and Tseng 2004; Lin and Zhang 2006; Avalos, Grandvalet, and Ambroise 2007; Bühlmann and Hothorn 2007; Huang, Horowitz, and Wei 2010; Zhang, Cheng, and Liu 2011; Wood, Goude,”
CAM: Causal Additive Models, high-dimensional order search and penalized regression (2013, methodology): “We implement this with a boosting method for additive model fitting [2, 5], using the R-function gamboost from the package mboost [9].”
High-dimensional additive modeling (2008, methodology): “Our new approach with the sparsity and smoothness penalty is never worse and some-times substantially better than L 2 -boosting for generalized additive model ﬁtting [ 5, 7].”
Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, neutral): “From a broader point of view, discussions of consistency and convergence of general L2 boosting framework can be found in Bühlmann and Yu (2003), Zhang et al. (2005) and Bühlmann and Hothorn (2007).”
The functional linear array model (2015, methodology): “The model is represented as the sum of simple (penalized) regression models, the so-called base-learners, that fit the negative gradient in each step (Friedman, 2001; Bühlmann and Hothorn, 2007).”
Using methods from the data mining and machine learning literature for disease classification and prediction: A case study examining classification of heart failure sub-types (2013, methodology): “The reader is referred elsewhere for a more detailed discussion of the theoretical foundation of boosting and its relationship with established methods in statistics [18,19].”
Less than eight (and a half) misconceptions of spatial analysis (2012, methodology): “Methods accounting for non-stationarity and SA include, for example, wavelet-revised regressions (Carl & Kühn, 2010) or a specific non-stationary extension of boosted regression trees (Bühlmann & Hothorn, 2007; Hothorn et al., 2011).”
Building Predictive Models in R Using the caret Package (2008, methodology): “See B¨uhlmann and Hothorn (2007) for more details about AIC pruning.”
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence (2019, background): “Researchers in this area tend to follow a similar strategy to those who study bagging, relying on slight idealizations to derive convergence rates and other relevant theorems (Bühlmann and Hothorn 2007; Bühlmann and Yu 2003; Ehrlinger and Ishwaran 2012).”
High dimensional structured additive regression models: Bayesian regularization, smoothing and predictive performance (2011, background): “Other (non-Bayesian) approaches for high dimensional regression such as boosting (Bühlmann and Hothorn, 2007) or random forests (Breiman, 2001) have been developed in the statistical learning community.”
A survey on missing data in machine learning (2021, methodology): “Imputation is then made from a regression tree by identifying the terminal node to which a new subject belongs and sampling from the outcomes in that node [90].”
Ensemble Learning (2019, positive): “Bühlmann and Hothorn (2010) propose Twin Boosting, which involves a first round of classical boosting followed by a second round of boosting which is forced to resemble the one from the first round. The method has much better feature selection behaviour than boosting, particularly with respect to re”