Supervised Dimensionality Reduction via Distance Correlation Maximization

By Praneeth Vepakomma, Chetan Tonde, Ahmed M. Elgammal. 2016

42 citations recorded by the source corpus.

In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our proposed formulation is based on learning a low-dimensional feature representation $\mathbf{z}$, which maximizes the squared sum of Distance Correlations between low dimensional features $\mathbf{z}$ and response $y$, and also between features $\mathbf{z}$ and covariates $\mathbf{x}$. We propose a novel algorithm to optimize our proposed objective using the Generalized Minimization Maximizaiton method of \Parizi et. al. (2015). We show superior empirical results on multiple datasets proving the effectiveness of our proposed approach over several relevant state-of-the-art supervised dimensionality reduction methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Optimization & Training

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “A variation of split learning called NoPeek SplitNN [395] reduces the potential leakage via communicated activations, by reducing their distance correlation [394, 378] with the raw data, while maintaining good model performance via categorical cross-entropy.”
NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, methodology): “…function can be expressed as below as distance correlation can be expressed in terms of speciﬁc graph Laplacians whose exact form is detailed in [50]: The iterative update for maximization of the above objective is based on a variant of majorization-minimization [51, 52] and is given by Z t =…”
R EDUCING LEAKAGE IN DISTRIBUTED DEEP LEARNING FOR SENSITIVE HEALTH DATA (2019, methodology): “Distributed deep learning methods: Split learning (Gupta & Raskar, 2018; Vepakomma et al., 2018a) is a recently developed resource efficient method for distributed deep learning by sending intermediate representations (smashed data) of split layer to another entity which completes rest of the…”