XGBoost: A Scalable Tree Boosting System

By Tianqi Chen, Carlos Guestrin. 2016

57,634 citations recorded by the source corpus.

Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.

Subfields: Artificial Intelligence, Optimization & Training, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Percy Liang
  3. Yoshua Bengio
  4. Jianfeng Gao
  5. Quoc V. Le
  6. Luke S. Zettlemoyer
  7. Graham Neubig
  8. Yejin Choi

In Optimization & Training

  1. Frank Hutter
  2. Yoshua Bengio
  3. Katharina Eggensperger
  4. Matthias Feurer
  5. Quoc V. Le
  6. Stefan Falkner
  7. Martin Wistuba
  8. Holger H. Hoos

In ML Systems & Hardware

  1. Frank Hutter
  2. Tong Wang
  3. Sebastian Raschka
  4. Su-In Lee
  5. Scott M. Lundberg
  6. Alex Smola
  7. Edouard Oyallon
  8. Satoshi Hara

Citations in the corpus