Normalized Mutual Information Feature Selection

By Pablo A. Estévez, Michel Tesmer, Claudio A. Perez, J. Zurada. IEEE Transactions on Neural Networks, 2009

1,267 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/db52bf6019914a04422cc3c7ccdfe0cf427c7bcd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Machine learning applications in cancer prognosis and prediction (2014, background): “It should be noted that this study estimated also the importance of prognostic factors by means of normalized mutual information index (NMI) [61].”
MIFS-ND: A mutual information-based feature selection method (2014, methodology): “Therefore, feature selection methods are used to determine a minimal feature set which is optimal and does not contain redundant features.”
Review of soft sensor methods for regression applications (2016, methodology): “Another variant of [50; 51] was proposed in [53] and is called normalized mutual information feature selection (NMIFS).”
Building an Intrusion Detection System Using a Filter-Based Feature Selection Algorithm (2016, background): “Several feature selection algorithms, including those in [11], [30], [34], [35], and [36], have been proposed in literature based on the principle of mutual information.”
Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the Art (2017, methodology): “Many extensions of these methods have been proposed in past decades, including modified Fisher’s LDA [41], regularized LDA [42], modified NWFE using spatial and spectral information [43], kernel NWFE [44], extended J–M to multiclass cases [40], J–M distance for spatially invariant features [45], min”
A Minimax Approach to Supervised Learning (2016, methodology): “regression maximizes the worst-case mutual information over ( Q), which seems superior to the heuristic techniques for maximizing an approximation of the mutual information I(X S;Y) in the literature [30,31]. 5 Numerical Experiments We evaluated the performance of the minimax SVM (mmSVM), the minima”
Differential evolution for filter feature selection based on information theory and feature ranking (2018, background): “Estevez et al. [37] normalized the relevance component (between two features) of mRmR by dividing with the minimum entropy of the two features.”
ExploreKit: Automatic Feature Generation and Selection (2016, neutral): “Normalization has also been shown critical to the performance of multiple machine learning algorithms [3].”
Survey on Feature Selection (2015, methodology): “In filters, individual features or subsets are evaluated independently of the learning algorithms, while wrappers use the learning algorithm to evaluate feature subsets (Estévez et al., 2009).”
A Survey on Evolutionary Computation Approaches to Feature Selection (2016, background): “Although there are some existing fast evaluation measures, such as mutual information [12], [32], [34], [240], they evaluate features individually rather than a group of features.”
An Effective Feature Selection Method via Mutual Information Estimation (2012, background): “Using appropriate combinations of joint and marginal density functions, mutual information can provide relationship among random variables that are beyond that of first- and second-order statistics [7], [9], [12].”
Discriminative Least Squares Regression for Multiclass Classification and Feature Selection (2012, background): “The family of supervised feature selection algorithms can be divided into ﬁlters [14], [15], [31], wrappers [32], [33], and embedded methods [21], [23], [34], according to whether a speciﬁc classiﬁer is integrated into the learning framework [7].”
A review of feature selection methods based on mutual information (2013, background): “There is no standard theory about MI normalization [16, 19].”
Feature Importance in Nonlinear Embeddings (FINE): Applications in Digital Pathology (2016, methodology): “Feature selection methods include wrappers and embedded methods [18], which typically involve selection of feature subsets that perform well in a particular classifier training framework, and filters [19], [20], which select features based on a variable ranking scheme.”
Discriminative Semi-Supervised Feature Selection Via Manifold Regularization (2009, background): “A number of performance criteria have been proposed for feature selection, including mutual information [19], [31], maximum margin [22], [52], kernel alignment [15], [37], graph-spectrum based measures [53], [66], construction errors in neural network [35], [42], worst case probability [4], [61], an”
A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture (2018, background): “The second one is Normalized Mutual Information (NMI) [34]: NMI (Y ,C) = I (Y ,C) 1 2 [H (Y )+ H (C)] where Y denotes the ground-truth labels, C denotes the clusters labels, I is the mutual information metric and H is entropy.”