CatBoost: gradient boosting with categorical features support

By Anna Veronika Dorogush, Vasily Ershov, Andrey Gulin. 2018

1,928 citations recorded by the source corpus.

In this paper we present CatBoost, a new open-sourced gradient boosting library that successfully handles categorical features and outperforms existing publicly available implementations of gradient boosting in terms of quality on a set of popular publicly available datasets. The library has a GPU implementation of learning algorithm and a CPU implementation of scoring algorithm, which are significantly faster than other gradient boosting libraries on ensembles of similar sizes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FOSTER: Feature Boosting and Compression for Class-Incremental Learning (2022, background): “In recent years, gradient boosting [13] based algorithms [2,7,23] achieve excellent performance on various tasks.”
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (2021, neutral): “Commonly used tools such as XGBoost [4], CatBoost [10], and LightGBM [22] provide several beneﬁts such as interpretability, the ability to handle a variety of feature types including null values, as well as performance in both high and low data regimes.”
Tabular Data: Deep Learning is Not All You Need (2021, methodology): “Researchers and practitioners use several GBDT algorithms, including XGBoost, LightGBM, and CatBoost [Chen and Guestrin, 2016, Ke et al., 2017, Prokhorenkova et al., 2018b].”
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification (2021, methodology): “This dataset is part of the Shifts Project [29], which also provides an ensemble of 10 pretrained CatBoost [30] models for making the temperature predictions.”
Attention-Based Prototypical Learning Towards Interpretable, Confident and Robust Deep Neural Networks (2019, background): “For Adult Census Income, CatBoost (Dorogush et al., 2018) achieves around 12% error rate, that is obtained by our framework refraining from 10% of predictions.”
HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism (2020, background): “Due to their cost-effectiveness, less expensive GPUs targeted for desktops and workstations, rather than high-end servers are also commonly used for machine learning training, especially for small and medium size clusters [12,19,44,50,51,53].”
Thou shalt not hate: Countering Online Hate Speech (2018, methodology): “CatBoost (CB) (Dorogush, Ershov, and Gulin 2017), Decision Tree (DT), and neural models such as Multi-layer Per-ceptron (MLP), LSTM.”
An Introduction to Machine Learning (2020, background): “Even without hyperparameter tuning, they usually provide excellent performance with a relatively low computational cost.(11) On the other hand, random forests are usually less prone to overfitting(45) and require less parameter tuning.”
Energy Theft Detection Using Gradient Boosting Theft Detector With Feature Engineering-Based Preprocessing (2019, methodology): “The key idea is that it uses a vectorized representation of the tree (bina-rized splitting criteria within each level), which can be evaluated faster [4].”
A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation (2023, methodology): “Lately, there have been several alterations of boosting algorithms [92], which include likelihood-based boosting [93], model-based boosting [94], GAMBoost [95], CoxBoost [96], and CatBoost [97].”
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (2020, methodology): “, 2017), CatBoost boosted trees (Dorogush et al., 2018), Random Forests, Extremely Randomized Trees, and kNearest Neighbors.”