Nonlinear ICA of Temporally Dependent Stationary Sources

By Aapo Hyvärinen, Hiroshi Morioka. 2017

272 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, methodology): “PCL (Hyvarinen and Morioka, 2017) stands for “permutation contrastive learning” and works as follows: Given sequential data {X}t=1, PCL trains a regression function r((x′, x)) to discriminate between pairs of adjacent observations (positive pairs) and randomly matched pairs (negative pairs).”
Toward Causal Representation Learning (2021, background): “These contexts can come from interventions [187], [191], [220], nonstationary time series [101], [116], [192], or multiple views [90], [114].”
Towards Causal Representation Learning (2021, background): “These contexts can come from interventions [218, 189, 192], non-stationary time series [117, 100, 193] or multiple views [89, 115].”
Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series (2020, methodology): “Recent theoretical and practical advances in Nonlinear ICA provide a principled approach to this problem (Hyv¨arinen and Morioka, 2016; Hyv¨arinen and Morioka, 2017; Hyv¨arinen et al., 2019; Khemakhem et al., 2020; Sorrenson et al., 2020).”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “Recently, the ICA community has considered weak forms of supervision such as temporal consistency (Hyvarinen & Morioka, 2016; 2017), auxiliary supervised information (Hyvarinen et al., 2019; Khemakhem et al., 2019), and multiple views (Gresele et al., 2019).”
The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, background): “Furthermore, while [20] considers a generative model of the form x(t) = f(s(t)), thus constraining the mixing function to be the same for any two data points x(t1), x(t2), in our setting we consider two different mixing functions, f1 and f2, for the two different views.”
Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning (2018, methodology): “(2016) is quite similar to the nonlinear ICA by Hyvärinen and Morioka (2017a), while Oord et al. (2018) proposed a method related to Hyvärinen and Morioka (2017b)—see also further self-supervised methods by Noroozi and Favaro (2016); Larsson et al. (2017). These approaches have allowed unsupervised ”
Variational Autoencoders and Nonlinear ICA: A Unifying Framework (2019, background): “Recently, three new nonlinear ICA frameworks [10, 11, 13] exploring the latter direction were proposed, in which it is possible to recover identiﬁable sources, up to some trivial transformations.”
Relative gradient optimization of the Jacobian term in unsupervised deep learning (2020, background): “[27] Aapo Hyvärinen and Hiroshi Morioka.”
Contrastive Learning Inverts the Data Generating Process (2021, methodology): “The authors further provide practical estimation methods for solving the nonlinear ICA problem (Hyvärinen & Morioka, 2016; 2017), similar in spirit to noise contrastive estimation (NCE; Gutmann & Hyvärinen, 2012).”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, methodology): “a time stamp or environment index) renders the sources conditionally independent [45, 54, 55, 58].”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, methodology): “Previous attempts to avoid non-identiﬁability either require some form of supervision in practice [9–11], or are unsuited to model sparse transitions in natural scenes [9, 12, 13].”
Learning Independent Features with Adversarial Nets for Non-linear ICA (2017, methodology): “Various methods for learning ICA models have been proposed and the method has been extended to mixing models that go beyond the original linearity and iid assumptions (Almeida, 2003; Hyvarinen & Morioka, 2017).”