Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost

By Ping Li. 2010

173 citations recorded by the source corpus.

Logitboost is an influential boosting algorithm for classification. In this paper, we develop robust logitboost to provide an explicit formulation of tree-split criterion for building weak learners (regression trees) for logitboost. This formulation leads to a numerically stable implementation of logitboost. We then propose abc-logitboost for multi-class classification, by combining robust logitboost with the prior work of abc-boost. Previously, abc-boost was implemented as abc-mart using the mart algorithm. Our extensive experiments on multi-class classification compare four algorithms: mart, abcmart, (robust) logitboost, and abc-logitboost, and demonstrate the superiority of abc-logitboost. Comparisons with other learning methods including SVM and deep learning are also available through prior publications.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

XGBoost: A Scalable Tree Boosting System (2016, background): “Tree boosting has been shown to give state-of-the-art results on many standard classification benchmarks [16].”
Yahoo! Learning to Rank Challenge Overview (2010, methodology): “Ping Li recently proposed Robust LogitBoost (Li, 2010) to provide a numerically stable implementation of the highly influential LogitBoost algorithm (Friedman et al.”
Efficient Second-Order Gradient Boosting for Conditional Random Fields (2015, methodology): “In the same spirit as LogitBoost for multi-class prediction [Friedman et al., 1998, Li, 2010], we consider the second order Taylor expansion of the negative log-likelihood l ( y , x , φ ): (4) The gradient G and Hessian H in Eq.”
XGBoost : Reliable Large-scale Tree Boosting System (2015, methodology): “Variants of the model have been applied to problems such as classification [5, 8] and ranking [4].”
LightGBM: A Highly Efficient Gradient Boosting Decision Tree (2017, background): “GBDT achieves state-of-the-art performances in many machine learning tasks, such as multi-class classification [2], click prediction [3], and learning to rank [4].”
Min-Max Kernels (2015, background): “…are all public (and mostly well-known), from various sources including the UCI repository, the LIBSVM web site, the book web site of [12], and the two papers [17, 18] which compared deep nets, boosting and trees, kernel SVMs, etc. (Also see http://hunch.net/?p=1467 for interesting discussions.)”