Simultaneous feature selection and clustering using mixture models

By Martin H. C. Law, Mário A. T. Figueiredo, Anil K. Jain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004

684 citations recorded by the source corpus.

Clustering is a common unsupervised learning technique used to discover group structure in a set of data. While there exist many algorithms for clustering, the important issue of feature selection, that is, what attributes of the data should be used by the clustering algorithms, is rarely touched upon. Feature selection for clustering is difficult because, unlike in supervised learning, there are no class labels for the data and, thus, no obvious criteria to guide the search. Another important problem in clustering is the determination of the number of clusters, which clearly impacts and is influenced by the feature selection issue. In this paper, we propose the concept of feature saliency and introduce an expectation-maximization (EM) algorithm to estimate it, in the context of mixture-based clustering. Due to the introduction of a minimum message length model selection criterion, the saliency of irrelevant features is driven toward zero, which corresponds to performing feature selection. The criterion and algorithm are then extended to simultaneously estimate the feature saliencies and the number of clusters.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2a3e107dad356099205fb86a44ede8dfd4d9956e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Semi-supervised Feature Selection via Spectral Analysis (2007, background): “Recently, an increasing number of researchers paid attention to developing unsupervised feature selection [16, 20].”
Unsupervised feature selection for multi-cluster data (2010, methodology): “For unsupervised “wrapper” methods, the clustering is a commonly used mining algorithm [10, 13, 20, 24].”
MILES: Multiple-Instance Learning via Embedded Instance Selection (2006, neutral): “tion [41], [29], [25], [30], [39], [12], [23], [28], [31], [50], [57].”
An ICA Mixture Hidden Markov Model for Video Content Analysis (2008, neutral): “There are some efficient algorithms [18] and [19] that can be used to estimate these initial parameters and thus can effectively help convergence and estimate the number of mixtures.”
Model-based clustering of high-dimensional data: A review (2014, background): “On the one hand, some authors such as [49, 54, 57, 83] tackle the problem of variable selection for model-based clustering within a Bayesian framework.”
Variable Selection for Clustering with Gaussian Mixture Models (2009, methodology): “They thus avoid the unrealistic independence assumption be-tween the relevant and irrelevant variables for the clustering, considered in Tadesse et al. (2005), Kim et al. (2006), and Law et al. (2004).”
Selection of time instants and intervals with Support Vector Regression for multivariate functional data (2020, background): “Variable selection has also been studied in other domains such as clustering (Dash et al., 2002; Dash and Liu, 2000; Dash et al., 1997; Law et al., 2004, 2003; Li et al., 2008) and supervised classiﬁcation (Aytug, 2015; Benítez-Peña et al., 2019; Bertolazzi et al., 2016; Carrizosa et al., 2011;…”
Selection of Variables for Cluster Analysis and Classification Rules (2006, methodology): “Several heuristic proposals were based on the adjusted Rand index (ARI), an adjusted version of Rand’s index (1971) introduced by Hubert and Arabie (1985), to measure pairwise agreement among partitions.”
A survey on feature selection methods (2014, methodology): “Feature selection using unsupervised learning can provide better description and reliability of the data than just unsupervised learning [7].”
Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering (2014, background): “Some consider the inverse task: feature selection for unsupervised clustering [25, 48].”