DDD: A New Ensemble Approach for Dealing with Concept Drift

By Leandro L. Minku, Xin Yao. IEEE Transactions on Knowledge and Data Engineering, 2012

441 citations recorded by the source corpus.

Online learning algorithms often have to operate in the presence of concept drifts. A recent study revealed that different diversity levels in an ensemble of learning machines are required in order to maintain high generalization on both old and new concepts. Inspired by this study and based on a further study of diversity with different strategies to deal with drifts, we propose a new online ensemble learning approach called Diversity for Dealing with Drifts (DDD). DDD maintains ensembles with different diversity levels and is able to attain better accuracy than other approaches. Furthermore, it is very robust, outperforming other drift handling approaches in terms of accuracy when there are false positive drift detections. In all the experimental comparisons we have carried out, DDD always performed at least as well as other drift handling approaches under various conditions, with very few exceptions.

Subfields: Artificial Intelligence, Biomedical Engineering, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/38f2e8cf9c0e257dbbacb5d20b2101103fb80f47

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Management Science and Operations Research

Citations in the corpus

Tackling Faults in the Industry 4.0 Era—A Survey of Machine-Learning Solutions and Key Aspects (2019, background): “The basis for the proposed solution is diversity for dealing with drift (DDD) [69], enabling data diversity manipulation for constructing ensemble models, with each one comprising several base classifiers to address concept drifts.”
From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors (2022, background): “Hoeffding Drift Detection Method (HDDM) [97] modifies DDM by using the Hoeffding’s inequality [98] to detect substantial changes in the moving average of the 19 Concept drift detection Windowing technique ADWIN SEED ADWIN2 MDDM ADDM MD3 fsNB EDIST ADDS EDIST2 STEPD FPDD WSTD FSDD CSDD Nacre CALMID S”
Ensemble learning: A survey (2018, background): “The diversity for dealing with drifts (DDD) approach (Minku & Yao, 2012) manifests this concept by maintaining two ensembles of decision trees (decision forests) with two different levels of diversity.”
Adaptive random forests for evolving data stream classification (2017, methodology): “Subsequent algorithms like Leveraging Bagging [13] and the Diversity for Dealing with Drifts ensemble (DDD) [34] uses different values of λ to use more instances for training the base models (as in Leveraging Bagging) or to induce more diversity to the ensemble by using varying values of λ (as in DD”
A Survey on Ensemble Learning for Data Stream Classification (2017, methodology): “Only a few studies on ensemble learning for data streams are focused on measuring diversity and on diversity properties [Minku et al. 2010; Minku and Yao 2012] and more recently in [Brzezinski and Stefanowski 2016].”
Distribution-Free One-Pass Learning (2017, background): “Note that although there are plenty of studies about learning with distribution change, most approaches are developed based on the use of sliding windows [18], [19], [20] or ensemble strategies [4], [5], [21], [22].”
Practical Ensemble Classification Error Bounds for Different Operating Points (2013, background): “For example, consider an analyst at an information technology (IT) outsourcing provider who must label service tickets into categories such as network, operating system, or hardware to analyze what problems are frequently occurring at client installations [5].”
Learning in Nonstationary Environments: A Survey (2015, background): “Recent work has shown that using both high and low diversity ensembles can be beneficial for tracking different rates of drift in the data stream [85], [86].”
Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble Algorithm (2014, neutral): “system combining four diversified ensembles [27].”