Optimal Whitening and Decorrelation

By A. Kessy, A. Lewin, Korbinian Strimmer. 2015

512 citations recorded by the source corpus.

ABSTRACT Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example, based on principal component analysis (PCA), Cholesky matrix decomposition, and zero-phase component analysis (ZCA), among others. Here, we provide an overview of the underlying theory and discuss five natural whitening procedures. Subsequently, we demonstrate that investigating the cross-covariance and the cross-correlation matrix between sphered and original variables allows to break the rotational invariance and to identify optimal whitening transformations. As a result we recommend two particular approaches: ZCA-cor whitening to produce sphered variables that are maximally similar to the original variables, and PCA-cor whitening to obtain sphered variables that maximally compress the original variables.

Subfields: Analytical Chemistry, Environmental Engineering, Signal Processing

View Paper: https://www.semanticscholar.org/paper/df43c5669e6c358572c38e29b7dcfbaaf09d19fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Environmental Engineering

In Signal Processing

Citations in the corpus

Spike sorting with Kilosort4 (2024, unknown): “A separate whitening vector is estimated for each channel based on its nearest 32 channels using the so-called ZCA (zero-phase component analysis) transform 41 .”
Decorrelated Batch Normalization (2018, methodology): “The whitening matrix is not unique because a whitened input stays whitened after an arbitrary rotation [27].”
Normalization Techniques in Training DNNs: Methodology, Analysis and Application (2020, methodology): “13, is called principal components analysis (PCA) whitening, where the whitening matrix There are an inﬁnite number of whitening matrices since a whitened input stays whitened after an arbitrary rotation [33], [35], which will be discussed in the subsequent sections.”
Automatic Classification of Defective Photovoltaic Module Cells in Electroluminescence Images (2018, methodology): “Due to the small size of our test dataset, we avoided an initial dimensionality reduction of the features using PCA in the preprocessing step, but rather used random initialization of embeddings.”
A polynomial chaos expansion in dependent random variables (2018, background): “[12], the selection of Wl depends on the desired cross-covariance”
Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration (2018, methodology): “Actually, we may employ Adaptive Instance Normalization (AdaIN) [12], or Zero-phase Component Analysis (ZCA) [14] used in WCT [19] as the whitening and coloring kernels.”
Whitening and Coloring Batch Transform for GANs (2018, methodology): “Luo (2017) proposes to whiten the activation values using a learnable whitening matrix.”
Autoencoder-based feature learning for cyber security applications (2017, methodology): “To do so, we used Principal Component Analysis (PCA) whitening algorithm [8].”
Randomized Matrix Decompositions using R (2016, background): “This process is also called sphering (Kessy, Lewin, and Strimmer 2018).”
Infrared and Visible Image Fusion with ResNet and zero-phase component analysis (2018, methodology): “In this paper, a novel fusion algorithm based on residual network(ResNet)[19] and zero-phase component analysis(ZCA)[20] is proposed for infrared and visible image fusion task.”
Distractor anticipation during working memory is associated with theta and beta oscillations across spatial scales (2024, unknown): “This method was built upon the decoding procedure described earlier, but instead of evaluating the classiﬁers only at the trained time bin, each classiﬁer was also tested across all other time bins.”