Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization

By Siwei Lyu, Eero P. Simoncelli. 2009

127 citations recorded by the source corpus.

We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics. We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.

Subfields: Cognitive Neuroscience, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Density Modeling of Images using a Generalized Normalization Transformation (2015, methodology): “We then transformed these two-dimensional datasets using ICA, RG, and GDN. Figure 1 (modelled after ﬁgure 4 of Lyu & Simoncelli (2009b)) shows the mutual information in the transformed data (note that, in our case, mutual information is related to the negentropy by an additive constant.)”
Exploiting compositionality to explore a large space of model structures (2012, methodology): “These correspond to plausible structures for the datasets, as discussed in the text. rank-one factorization of the scale matrix, similar to the radial Gaussianization model of Lyu and Simoncelli (2009) for neighboring wavelet coefﬁcients.”
Iterative Gaussianization: From ICA to Random Rotations (2011, methodology): “Radial Gaussianization (RG) was specifically developed to deal with these particular kind of models [21].”
Psychophysically Tuned Divisive Normalization Approximately Factorizes the PDF of Natural Images (2010, unknown): “In particular, if the appropriate value were γ = 1, the transform would give rise to gaussian marginal PDFs, thus becoming similar to radial gaussianization transforms as suggested in Lyu and Simoncelli ( 2009 ).”
From learning models of natural image patches to whole image restoration (2011, methodology): “A different approach was taken recently in the form of radial Gaussianization [8], in which components which are distributed in a radially symmetric manner are made independent by transforming them non-linearly into a radial Gaussian, and thus, independent from one another.”
"Natural Images, Gaussian Mixtures and Dead Leaves" (2012, background): “Recently, there has been a growing interest in comparing different aspects of models for natural images such as log-likelihood and multi-information reduction performance, and much progress has been achieved [1,2, 3,4,5, 6].”
Divisive Normalization: Justification and Effectiveness as Efficient Coding Transform (2010, background): “, [17, 26]) have also hinted that DN may be an approximation to the optimal efficient coding transform.”
Perceptual image quality assessment using a normalized Laplacian pyramid (2016, background): “Despite a large body of work that has been interpreted as evidence that oriented linear filters are an optimal choice for capturing statistical regularities in images [14, 28], several articles have shown that this optimum is shallow [29], that non-oriented linear filters are nearly as effective as ”
How Sensitive Is the Human Visual System to the Local Statistics of Natural Images? (2013, background): “Also, the Lp-spherical distribution can be made factorial by using radial factorization instead of radial Gaussianization [10].”
Computing with a Canonical Neural Circuits Model with Pool Normalization and Modulating Feedback (2014, methodology): “Because of the stabilizing and self-adapting properties of normalization (Sperling, 1970; Grossberg, 1973, 1980, 1988; Lyu & Simoncelli, 2009), we include a stage of divisive normalization in our model that (at steady state) captures equation 2.2.”
Analysis of i-vector Length Normalization in Speaker Recognition Systems (2011, methodology): “In this way, is nothing more than an affine transformation of a multivariate Student’s t distribution which belongs to the family of Elliptically Symmetric Densities (ESD) [9].”
Lp-Nested Symmetric Distributions (2010, methodology): “Interestingly, Lp-spherically symmetric distributions other than the p-generalized Normal give rise to a nonlinear ICA algorithm called Radial Gaussianization for p = 2 [Lyu and Simoncelli, 2009] or Radial Factorization for arbitrary p [Sinz and Bethge, 2009].”
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models (2010, background): “Projection onto a sphere can be considered as a form of divisive normalization (Lyu & Simoncelli, 2009).”
Learning Intermediate-Level Representations of Form and Motion from Natural Movies (2012, background): “We first show how form-selective invariances may be learned from the log amplitudes, which we refer to as amplitude components.”