Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions

By Giovanni Seni, IV JohnF.Elder. Ensemble Methods in Data Mining, 2010

581 citations recorded by the source corpus.

Ensemble methods have been called the most influential development in Data Mining and Machine Learning in the past decade. They combine multiple models into one usually more accurate than the best of its components. Ensembles can provide a critical boost to industrial challenges -- from investment timing to drug discovery, and fraud detection to recommendation systems -- where predictive accuracy is more vital than model interpretability. Ensembles are useful with all modeling algorithms, but this book focuses on decision trees to explain them most clearly. After describing trees and their strengths and weaknesses, the authors provide an overview of regularization -- today understood to be a key reason for the superior performance of modern ensembling algorithms. The book continues with a clear description of two recent developments: Importance Sampling (IS) and Rule Ensembles (RE). IS reveals classic ensemble methods -- bagging, random forests, and boosting -- to be special cases of a single algorithm, thereby showing how to improve their accuracy and speed. REs are linear rule models derived from decision tree ensembles. They are the most interpretable version of ensembles, which is essential to applications such as credit scoring and fault diagnosis. Lastly, the authors explain the paradox of how ensembles achieve greater accuracy on new data despite their (apparently much greater) complexity.This book is aimed at novice and advanced analytic researchers and practitioners -- especially in Engineering, Statistics, and Computer Science. Those with little exposure to ensembles will learn why and how to employ this breakthrough method, and advanced practitioners will gain insight into building even more powerful models. Throughout, snippets of code in R are provided to illustrate the algorithms described and to encourage the reader to try the techniques. (edited by author)

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/30d0600e9be1ce97a196f32be99b7ef57839b76f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Neural Networks and Deep Learning (2018, unknown): “Detailed discussions of ensemble methods for classification may be found in [438, 566].”
A review of evidence about use and performance of species distribution modelling ensembles like BIOMOD (2019, background): “These types of ensembles have been shown in many cases to have superior predictive per‐ formance to individual models (Friedman & Popescu, 2008; Seni & Elder, 2010).”
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models (2017, background): “On the other hand, out-of-sample bootstrap validation yields the best balance between the bias and variance of estimates in the context of our study.”
A Survey of Evolutionary Algorithms for Decision-Tree Induction (2012, background): “Ensembles are created by the induction of different trees from training samples, and the ultimate classification is frequently given through a voting scheme (see [7] and [8]).”
Ensembles for feature selection: A review and future trends (2019, unknown): “…for researchers, that have investigated in many alternatives that have been proposed in classi(cid:12)cation, regression, preprocessing and other (cid:12)elds[89, 39, 90, 91, 92, 93], although there is not a clear and de(cid:12)nite winner method, as in many other areas of Machine Learning.”
A survey of multiple classifier systems as hybrid systems (2014, background): “These reviews include the books by Kuncheva [22], Rokach [23], Seni and Edler [24], and Baruque and Corchado [25].”
Big Data, new epistemologies and paradigm shifts (2014, unknown): “Moreover, since diﬀerent models have solutions (Seni and Elder, 2010).”
Deep Ensembles for Low-Data Transfer Learning (2020, methodology): “Known for improving models, ensembling methods have been studied in depth in and out of deep learning academia (Seni & Elder, 2010).”