Density Modeling of Images using a Generalized Normalization Transformation

By Johannes Ballé, Valero Laparra, Eero P. Simoncelli. 2015

446 citations recorded by the source corpus.

We introduce a parametric nonlinear transformation that is well-suited for Gaussianizing data from natural images. The data are linearly transformed, and each component is then normalized by a pooled activity measure, computed by exponentiating a weighted sum of rectified and exponentiated components and a constant. We optimize the parameters of the full transformation (linear transform, exponents, weights, constant) over a database of natural images, directly minimizing the negentropy of the responses. The optimized transformation substantially Gaussianizes the data, achieving a significantly smaller mutual information between transformed components than alternative methods including ICA and radial Gaussianization. The transformation is differentiable and can be efficiently inverted, and thus induces a density model on images. We show that samples of this model are visually similar to samples of natural image patches. We demonstrate the use of the model as a prior probability density that can be used to remove additive noise. Finally, we show that the transformation can be cascaded, with each layer optimized using the same Gaussianization objective, thus offering an unsupervised method of optimizing a deep network architecture.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Generative Models

In Signal Processing

Citations in the corpus

Density estimation using Real NVP (2016, methodology): “This formula has been discussed in several papers including the maximum likelihood formulation of independent components analysis (ICA) [4, 28], gaussianization [14, 11] and deep density models [5, 50, 17, 3].”
Conditional Image Synthesis with Auxiliary Classifier GANs (2016, background): “Improved image models advance the stateof-the-art in image denoising (Ballé et al., 2015), compression (Toderici et al.”
Masked Autoregressive Flow for Density Estimation (2017, background): “Enforcing invertibility with nonsingular weight matrices has been proposed [1, 26], however in such approaches calculating the determinant of the Jacobian scales cubicly with data dimensionality in general.”
End-to-end Optimized Image Compression (2016, methodology): “This scheme is called transform coding due to the central role of the transforma-∗ (Ball´e, Laparra, and Simoncelli, 2016).”
End-to-end optimization of nonlinear transform codes for perceptual quality (2016, background): “The approximate inverse we introduce here is based on the fixed point iteration for inversion of GDN introduced in [5].”
Variational image compression with a scale hyperprior (2018, methodology): “Note that Ballé et al. (2016b) compare models trained for different metrics, but their results are limited by the choice of transforms.”
DeepJSCC-f: Deep Joint Source-Channel Coding of Images With Feedback (2019, methodology): “It consists of CNN layers, followed by normalization obtained by the generalized normalization transformations (GDN/IGDN) [49], followed by a parametric ReLU (PReLU) [50] activation function (or a sigmoid, in the last blocks of decoder and combiner).”
Variable Rate Deep Image Compression With a Conditional Autoencoder (2019, background): “GDN stands for generalized divisive normalization, and IGDN is inverse GDN [2].”
Channel-Wise Autoregressive Entropy Models for Learned Image Compression (2020, methodology): “Finally, we use a simpliﬁed version of generalized divisive normalization (GDN) [31] where α ij and ε i are both set to 1.0.”
Joint Autoregressive and Hierarchical Priors for Learned Image Compression (2018, background): “GDN stands for generalized divisive normalization, and IGDN is inverse GDN [20].”
Wireless Image Transmission Using Deep Source Channel Coding With Attention Modules (2020, background): “Each of the first four modules consists of a convolution layer, a GDN layer [48] and a PReLU layer [49].”
The Devil Is in the Details: Window-based Attention for Image Compression (2022, background): “Generalized divisive normalization (GDN) [4] is a milestone in learned image compression.”
End-to-End Blind Image Quality Assessment Using Deep Neural Networks (2018, methodology): “In addition, GDN is proven to be iteratively invertible under mild assumptions [27], which preserves better information than ReLU.”
Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks (2017, methodology): “use a similar architecture but use a form of local gain control called generalized divisive normalization [3] and replace the non-differentiable quantization step with continuous relaxation by adding uniform noise [4].”
Autoencoder Based Image Compression: Can the Learning be Quantization Independent? (2018, unknown): “To this end, a deep convolutional autoencoder must ﬁrst be built and trained. g e is the composition of a convolutional layer, a generalized divisive normalization (GDN) [15], a convolutional layer, a GDN and a convolutional layer. g d is the reverse composition, replacing each GDN with an inverse g”