Nonlinear independent component analysis: Existence and uniqueness results

By Aapo Hyvärinen, Petteri Pajunen. 1999

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, positive): “Similarly, the non-linear ICA community recently shifted to non-iid data types exploiting time dependent or grouped observations (Hyvarinen and Morioka, 2016; Hyvarinen et al., 2019; Gresele et al., 2019) We focus our study on the setting where factors of variations are not observable at all, that i”
Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design (2019, methodology): “A ﬂow model f is constructed as an invertible transformation that maps observed data x to a standard Gaussian latent variable z = f ( x ) , as in nonlinear independent component analysis (Bell & Sejnowski, 1995; Hyv¨arinen et al., 2004; Hyv¨arinen & Pajunen, 1999).”
Density estimation using Real NVP (2016, background): “As the existence proof of nonlinear ICA solutions [29] suggests, auto-regressive models can be seen as tractable instance of maximum likelihood nonlinear ICA, where the residual corresponds to the independent components.”
Learning deep representations by mutual information estimation and maximization (2018, background): “This is the basis of numerous ICA algorithms, some of which are nonlinear (Hyv¨arinen & Pajunen, 1999; Almeida, 2003), but none of which are general enough to apply to deep networks.”
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “Indeed, Hyvärinen and Pajunen (1999) showed that this task is impossible with general nonlinear mixing under the standard assumption of independent latent factors.”
Sharp Minima Can Generalize For Deep Nets (2017, background): “Several practical (Dinh et al., 2014; Rezende & Mohamed, 2015; Kingma et al., 2016; Dinh et al., 2016) and theoretical works (Hyvärinen & Pajunen, 1999) show how powerful bijections can be.”
Toward Causal Representation Learning (2021, background): “case as there are infinitely many equivalent solutions yielding the same observational distribution [117], [158], [188].”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Ideas related to disentangling the factors of variation date back to the non-linear ICA literature (Comon, 1994; Hyv¨arinen and Pajunen, 1999; Bach and Jordan, 2002; Jutten and Karhunen, 2003; Hyvarinen and Morioka, 2016; Hyvarinen et al., 2019; Gresele et al., 2019).”
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, neutral): “One possible construction relies on the use of the inverse cumulative distribution function (cdf) in combination with uniformly-distributed random variables [9] and is also used in non-identiﬁability proofs for non-linear independent component analysis (ICA) [13].”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “Crucially, identifiability is not possible in the nonlinear case from i.i.d. observations (Hyvärinen & Pajunen, 1999).”
On the Fairness of Disentangled Representations (2019, background): “Ideas related to disentangling the factors of variations have a long tradition in machine learning, dating back to the non-linear ICA literature [17, 3, 46, 42, 43, 44, 32].”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Other Related Works Learning disentangled representations is similar in spirit to non-linear ICA, although it relies primarily on (architectural) inductive biases and different degrees of supervision [13, 2, 39, 36, 37, 38, 25, 33, 32].”
The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, background): “The proof of non-identifiability for the general case with unconditionally independent sources was an important negative result [22].”
Causal Discovery from Heterogeneous/Nonstationary Data (2019, background): “…for causal direction identiﬁcation. it is known that with functional causal model-based approaches, there are cases where causal directions are not identiﬁable, e.g., the linear-Gaussian case and the case with a general functional class (Hyv¨arinen and Pajunen, 1999; Zhang et al., 2015b).”
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, positive): “The lack of almost any identifiability result in nonlinear ICA has been a main bottleneck for the utility of the approach (Hyvarinen et al., 2018) and partially motivated alternative machine learning approaches (Desjardins et al., 2012; Schmidhuber, 1992; Cohen & Welling, 2015).”
Isolating Sources of Disentanglement in VAEs (2018, background): “In a similar vein, non-linear independent component analysis [36, 37, 38] studies the problem of inverting a generative process assuming independent latent factors.”