Stochastic gradient boosted distributed decision trees

By Jerry Ye, Jyh-Herng Chow, Jiang Chen, Zhaohui Zheng. 2009

356 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

XGBoost: A Scalable Tree Boosting System (2016, background): “While there are some existing works on parallel tree boosting [22, 23, 19], the directions such as out-of-core computation, cache-aware and sparsity-aware learning have not been explored.”
A reliable effective terascale linear learning system (2011, background): “However, it is not easy to run MPI on top of existing Hadoop clusters (Ye et al., 2009).”
Yahoo! Learning to Rank Challenge Overview (2010, methodology): “Publicly available packages exist (Ridgeway, 2007), but we used our internal implementation (Ye et al., 2009).”
Parallel boosted regression trees for web search ranking (2011, background): “The data can be partitioned by features [13], by samples [25] or both [31].”
DiFacto: Distributed Factorization Machines (2016, background): “The growing body in deep learning [3], kernels [7], and decision trees [22] bears witness of this development.”
GraphLab: A New Framework For Parallel Machine Learning (2010, methodology): “For example, the MapReduce abstraction [Dean and Ghemawat, 2004] has been successfully applied to a broad range of ML applications [Chu et al., 2006, Wolfe et al., 2008, Panda et al., 2009, Ye et al., 2009].”
An up-to-date comparison of state-of-the-art classification algorithms (2017, background): “Stochastic Gradient Boosting Trees (GBDT) ( Friedman, 2002; Ye, Chow, Chen, & Zheng, 2009 ) combines gradient boosting with bootstrap bagging.”
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design (2016, unknown): “The growing body in deep learning [42], kernel methods [81], and decision trees [148] bears witness of this development.”
VHT: Vertical hoeffding tree (2016, background): “[21] show how to distribute and parallelize Gradient Boosted Decision Trees (GBDT).”
Training Efficient Tree-Based Models for Document Ranking (2013, background): “The effectiveness of tree-based ensembles for learning to rank has been widely demonstrated: an example is the family of gradient-boosted regression trees (GBRTs) [9, 3, 10, 2].”
Scalable hierarchical multitask learning algorithms for conversion optimization in display advertising (2014, background): “This follows the design pattern of [21, 13] and it avoids ﬁle I/O for synchronization.”
Simple and Scalable Response Prediction for Display Advertising (2014, background): “However, it is not easy to run MPI on top of existing Hadoop clusters [Ye et al. 2009].”
PowerLyra: Differentiated Graph Computation and Partitioning on Skewed Graphs (2019, background): “Graph-structured computation has be come increasingly popular, which is evidenced by its adoption in a wide range of areas including social computation, web search, natural language processing, and recommendation systems [5, 25, 54, 67, 88, 98].”
Posthoc Interpretability of Learning to Rank Models using Secondary Training Data (2018, background): “While many algorithms have been suggested, only decision tree based appraoches have shown good performance and reasonable interpretability [9].”
Democrats, republicans and starbucks afficionados: user classification in twitter (2011, methodology): “We use a distributed GBDT architecture running over Hadoop [26] which allows us to scale to the dimensionality of Twitter.”