Higher Order Statistical Decorrelation without Information Loss

By Gustavo Deco, W. Brauer. 1994

Subfields: Representation & Self-Supervised Learning, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Information Theory

Citations in the corpus

Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design (2019, background): “These exact likelihood models are typically speciﬁed as invertible transformations that are parameterized by neural networks (Deco & Brauer, 1995; Larochelle & Murray, 2011; Uria et al., 2013; Dinh et al., 2014; Germain et al., 2015; van den Oord et al., 2016b; Salimans et al., 2017; Chen et al.,…”
Glow: Generative Flow with Invertible 1x1 Convolutions (2018, background): “While it may look intimidating, its value can be surprisingly simple to compute for certain choices of transformations, as previously explored in (Deco and Brauer, 1995; Dinh et al., 2014; Rezende and Mohamed, 2015; Kingma et al., 2016).”
Improved Variational Inference with Inverse Autoregressive Flow (2016, methodology): “generative models of /larger images, and how planar and radial ows can leverage the topology of latent space, as is possible with IAF. Volume-conserving neural architectures were rst presented in in (Deco and Brauer, 1995), as a form of nonlinear independent component analysis. Another type of norma”
Density estimation using Real NVP (2016, methodology): “This formula has been discussed in several papers including the maximum likelihood formulation of independent components analysis (ICA) [4, 28], gaussianization [14, 11] and deep density models [5, 50, 17, 3].”
The Reversible Residual Network: Backpropagation Without Storing Activations (2017, background): “In Deco and Brauer [8], the residual connections are composed of all ‘prior’ neurons in the layer, while NICE and our own architecture segments a layer into pairs of neurons and additively connect one with a residual function of the other.”
NIPS 2016 Tutorial: Generative Adversarial Networks (2016, background): “g that warps space in complicated ways can yield a complicated distribution over x, and if gis carefully designed, the density is tractable too. Models with nonlinear gfunctions date back at least to Deco and Brauer (1995). The latest member of this family is real NVP (Dinh et al., 2016). See gure 1”
An Introduction to Variational Autoencoders (2019, methodology): “If we use neural networks for such invertible mappings, this is a powerful and flexible approach towards probabilistic modeling of continuous data and nonlinear independent component analysis (Deco and Brauer, 1995).”
A Survey of Unsupervised Deep Domain Adaptation (2018, background): “elihood of the training data given the parameters. For techniques like fully visible belief nets [10, 36, 41, 68, 94] and change of variables or nonlinear independent components analysis (ICA) models [25, 29], the density function is tractable. For 4 Garrett Wilson and Diane J. Cook z G Fake Image D”
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation (2019, methodology): “We choose invertible transformations whose Jacobian determinant in Equation 1 is simple to compute, that is a triangular matrix, diagonal matrix or a permutation matrix as explored in prior work (Rezende & Mohamed, 2015; Deco & Brauer, 1995).”
Adversarial Transfer Learning (2018, methodology): “For techniques like fully visible belief nets [10, 36, 41, 68, 94] and change of variables or nonlinear independent components analysis (ICA) models [25, 29], the density function is tractable.”
Compression with Flows via Local Bits-Back Coding (2019, methodology): “Flow models We wish to construct a computationally efficient code specialized to a flow model f , which is a differentiable bijection between continuous data x ∈ R and latents z = f(x) ∈ R [9–11].”