Scalable real-time classification of data streams with concept drift

By Mark P. Tennant, Frederic T. Stahl, Omer Farooq Rana, João Bártolo Gomes. 2017

78 citations recorded by the source corpus.

Inducing adaptive predictive models in real-time from high throughput data streams is one of the most challenging areas of Big Data Analytics. The fact that data streams may contain concept drifts (changes of the pattern encoded in the stream over time) and are unbounded, imposes unique challenges in comparison with predictive data mining from batch data. Several real-time predictive data stream algorithms exist, however, most approaches are not naturally parallel and thus limited in their scalability. This paper highlights the Micro-Cluster Nearest Neighbour (MC-NN) data stream classifier. MC-NN is based on statistical summaries of the data stream and a nearest neighbour approach, which makes MC-NN naturally parallel. In its serial version MC-NN is able to handle data streams, the data does not need to reside in memory and is processed incrementally. MC-NN is also able to adapt to concept drifts. This paper provides an empirical study on the serial algorithm’s speed, adaptivity and accuracy. Furthermore, this paper discusses the new parallel implementation of MC-NN, its parallel properties and provides an empirical scalability study.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tyler L. Hayes
2. Christopher Kanan
3. Jie Lu
4. Nathan D. Cahill
5. Guangquan Zhang
6. Anjin Liu
7. Feng Gu
8. Fan Dong

In Signal Processing

1. João Gama
2. Jie Lu
3. Anjin Liu
4. Nathan D. Cahill
5. Feng Gu
6. Tyler L. Hayes
7. Christopher Kanan
8. Guangquan Zhang

Citations in the corpus

• Memory Efficient Experience Replay for Streaming Learning (2018, background): “Micro-clustering algorithms first generate local clusters based on the data stream directly, and then these micro-clusters are clustered themselves to generate a global clustering (macro-clusters) [3], [5], [23], [46], [68], [76].”
• Learning under Concept Drift: A Review (2019, background): “A recent study [127] proposed another scalable stream data”