Feature Extraction by Non-Parametric Mutual Information Maximization

By Kari Torkkola. Journal of machine learning research, 2003

724 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/56ed0d2509339b6a107bfc4c60a8ddf3089de3a6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Prediction with Model-Based Neutrality (2013, background): “Thus, the neutrality measures are closely connected to the measures of statistical dependency [16], [17].”
Survey on Feature Selection (2015, methodology): “Similarly, Torkkola (2003) proposed the use of Renyi’s entropy as an alternative to Shannon et al. (1949)’s entropy to solve the multivariate issue, whereas Markov blanket, presented Koller and Sahami (1996), is one other solution.”
Information-preserving hybrid data reduction based on fuzzy-rough techniques (2006, unknown): “It will bring 0167-8655/$ - see front matter 2005 Elsevier B.V.”
Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation (2010, methodology): “A great deal of effort has been made to estimate MI accurately, e.g., based on an adaptive histogram (HIST) [11], kernel density estimation (KDE) [29], the nearest neighbor distance (NN) [19], the Edgeworth expansion (EDGE) [17], and maximum likelihood MI estimation (MLMI) [28].”
Exponential Family Predictive Representations of State (2007, methodology): “Examples include image edge detection (Hamza, 2006), classification (Torkkola, 2003), speech processing (Schraudolph, 2004), image registration (He et al., 2003), and unsupervised learning (Principe et al., 1999).”
A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula (2016, methodology): “It has been used for feature selection in general classification problems (Lefakis and Fleuret, 2014; Peng et al., 2005; Torkkola, 2003) and we hope our estimator would also provide practical advantages in many such applications.”
Opcode sequences as representation of executables for data-mining-based unknown malware detection (2013, background): “…potential advantages such as reducing measurement, storage and transmission; decreasing training and testing times; confronting the curse of dimensionality to improve prediction performance in terms of speed, accuracy and simplicity and facilitating data visualization and understanding [76, 20].”
Learning Deep Representations with Probabilistic Knowledge Transfer (2018, methodology): “The information potentials for the teacher model are derived using these probabilities [45]:”
Estimation of Information Theoretic Measures for Continuous Random Variables (2008, background): “Mutual information is extensively used to assess whether two random variables are independent [2] and has been proposed to solve the all-relevant feature selection problem [8, 24].”
Mathematical Aspects of Divergence Based Vector Quantization Using Fr´ echet-Derivatives (2009, methodology): “In prototype based classification, first approaches utilizing information theoretic approaches were recently proposed [13],[ 56 ],[61].”
Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, methodology): “On the other hand, directly calculating I ( z ; n ) and taking it as a part of the optimization objective are difﬁcult, especially in the optimization of representation learning (Torkkola, 2003).”
Geometric Mean for Subspace Selection (2009, methodology): “Torkkola [40] applied the nonparametric Renyi entropy to model the mutual information between measurements in the selected subspace and corresponding labels.”
On discovery and learning of models with predictive representations of state for agents with continuous actions and observations (2007, methodology): “The solution that has emerged in the literature has been to 1) use a generalized information measure based on generalized entropies, and 2) use a kernel density estimate with a Gaussian kernel [8][13][3]; this is why we chose to model our densities with Gaussian kernel density estimation.”
Learning via Hilbert Space Embedding of Distributions (2007, background): “Quadratic Mutual Information [138] introduces the quadratic mutual information for feature selection.”
Feature Selection via Dependence Maximization (2012, background): “7 Quadratic Mutual Information Torkkola (2003) introduces the quadratic mutual information for feature selection.”
Multiclass Common Spatial Patterns and Information Theoretic Feature Extraction (2008, unknown): “[9], [17], [23]).”
Causality detection based on information-theoretic approaches in time series analysis (2007, methodology): “For example, classiﬁcation and pattern recognition literature uses it them for feature extraction, i.e. Torkkola [233] or for principal or independent component analysis, i.e. Xu et al. [254,184,185].”