Applying lazy learning algorithms to tackle concept drift in spam filtering

By Florentino Fernández Riverola, Eva Lorenzo Iglesias, Fernando Díaz, José Ramon Méndez, Juan Manuel Corchado. 2007

121 citations recorded by the source corpus.

A great amount of machine learning techniques have been applied to problems where data is collected over an extended period of time. However, the disadvantage with many real-world applications is that the distribution underlying the data is likely to change over time. In these situations, a problem that many global eager learners face is their inability to adapt to local concept drift. Concept drift in spam is particularly difficult as the spammers actively change the nature of their messages to elude spam filters. Algorithms that track concept drift must be able to identify a change in the target concept (spam or legitimate e-mails) without direct knowledge of the underlying shift in distribution. In this paper we show how a previously successful instance-based reasoning e-mail filtering model can be improved in order to better track concept drift in spam domain. Our proposal is based on the definition of two complementary techniques able to select both terms and e-mails representative of the current situation. The enhanced system is evaluated against other well-known successful lazy learning approaches in two scenarios, all within a cost-sensitive framework. The results obtained from the experiments carried out are very promising and back up the idea that instance-based reasoning systems can offer a number of advantages tackling concept drift in dynamic problems, as in the case of the anti-spam filtering domain.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “torheological damper [93], and spam filtering [94]; among others.”
Learning under Concept Drift: an Overview (2010, background): “Finally, concept drift problem is highly relevant for spam filtering [36, 46].”
A review of machine learning approaches to Spam filtering (2009, background): “Fdez-Riverola et al. (2007a) developed SpamHunting, a CBR filter with a frequency representation for terms the message header and body.”