Iterative Gaussianization: From ICA to Random Rotations

By Valero Laparra, Gustau Camps-Valls, Jesus Malo. 2011

144 citations recorded by the source corpus.

Most signal processing problems involve the challenging task of multidimensional probability density function (PDF) estimation. In this paper, we propose a solution to this problem by using a family of rotation-based iterative Gaussianization (RBIG) transforms. The general framework consists of the sequential application of a univariate marginal Gaussianization transform followed by an orthonormal transform. The proposed procedure looks for differentiable transforms to a known PDF so that the unknown PDF can be estimated at any point of the original domain. In particular, we aim at a zero-mean unit-covariance Gaussian for convenience. RBIG is formally similar to classical iterative projection pursuit algorithms. However, we show that, unlike in PP methods, the particular class of rotations used has no special qualitative relevance in this context, since looking for interestingness is not a critical issue for PDF estimation. The key difference is that our approach focuses on the univariate part (marginal Gaussianization) of the problem rather than on the multivariate part (rotation). This difference implies that one may select the most convenient rotation suited to each practical application. The differentiability, invertibility, and convergence of RBIG are theoretically and experimentally analyzed. Relation to other methods, such as radial Gaussianization, one-class support vector domain description, and deep neural networks is also pointed out. The practical performance of RBIG is successfully illustrated in a number of multidimensional problems such as image synthesis, classification, denoising, and multi-information estimation.

Subfields: Artificial Intelligence, Molecular Biology, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Signal Processing

Citations in the corpus

Density Modeling of Images using a Generalized Normalization Transformation (2015, methodology): “More recent examples include marginal and radial transformations (Chen & Gopinath, 2000; Lyu & Simoncelli, 2009b; Laparra et al., 2011), as well as rectiﬁed-linear transformations (Dinh et al., 2014).”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “It was Laparra et al. (2011) that first applied Gaussianization to real data, using the density function to perform one-class classification to detect urban areas in satellite images.”
Learning Speaker-Specific Characteristics With a Deep Neural Architecture (2011, methodology): “In addition, feature selection and data component analysis techniques, e.g., principal component analysis or independent component analysis, have been also applied to explore speaker-specific characteristics [1], [2], [9].”
An introduction to sampling via measure transport (2016, background): “A related scheme is presented in [44]; here the sequence of maps alternates between rotations and diagonal maps that transform the marginals.”
Optimizing the Latent Space of Generative Networks (2017, methodology): “Finally, the idea of mapping between images and noise to learn generative models is a well known technique (Chen & Gopinath, 2000; Laparra et al., 2011; Sohl-Dickstein et al., 2015; Bordes et al., 2017).”
Unsupervised Deep Feature Extraction for Remote Sensing Image Classification (2015, background): “State-of-the-art manifold learning methods [19] include the following: local approaches for the description of remote sensing image manifolds [20]; kernel-based and spectral decompositions that learn mappings optimizing for maximum variance, correlation, entropy, or minimum noise fraction [21]; neur”
From Dependence to Causation (2016, methodology): “When this transformation is a random rotation, the principal component analysis rotation, or the independent component analysis rotation, the sample X(t+1) follows a distribution closer to the Normal distribution than the previous iterate X(t) (Laparra et al., 2011).”