Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series

By Hermanni Hälvä, Aapo Hyvärinen. 2020

94 citations recorded by the source corpus.

Recent advances in nonlinear Independent Component Analysis (ICA) provide a principled framework for unsupervised feature learning and disentanglement. The central idea in such works is that the latent components are assumed to be independent conditional on some observed auxiliary variables, such as the time-segment index. This requires manual segmentation of data into non-stationary segments which is computationally expensive, inaccurate and often impossible. These models are thus not fully unsupervised. We remedy these limitations by combining nonlinear ICA with a Hidden Markov Model, resulting in a model where a latent state acts in place of the observed segment-index. We prove identifiability of the proposed model for a general mixing nonlinearity, such as a neural network. We also show how maximum likelihood estimation of the model can be done using the expectation-maximization algorithm. Thus, we achieve a new nonlinear ICA framework which is unsupervised, more efficient, as well as able to model underlying temporal dynamics.

Subfields: Analytical Chemistry, Artificial Intelligence, Representation & Self-Supervised Learning, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

1. Nan Rosemary Ke
2. Stefan Bauer
3. Bernhard Schölkopf
4. Yoshua Bengio
5. Anirudh Goyal
6. Michel Besserve
7. Luigi Gresele
8. Wieland Brendel

In Artificial Intelligence

1. Bernhard Schölkopf
2. Yoshua Bengio
3. Anirudh Goyal
4. Stefan Bauer
5. Nan Rosemary Ke
6. Francesco Locatello
7. Kyunghyun Cho
8. Quoc V. Le

In Representation & Self-Supervised Learning

1. Bernhard Schölkopf
2. Francesco Locatello
3. Anirudh Goyal
4. Stefan Bauer
5. Yoshua Bengio
6. Nan Rosemary Ke
7. Yash Sharma
8. Nal Kalchbrenner

In Signal Processing

1. Bernhard Schölkopf
2. Stefan Bauer
3. Anirudh Goyal
4. Yoshua Bengio
5. Nal Kalchbrenner
6. Nan Rosemary Ke
7. Francesco Locatello
8. Michel Besserve

In Time-Series & Forecasting

1. Bernhard Schölkopf
2. Stefan Bauer
3. Yoshua Bengio
4. Arthur Gretton
5. Kacper P. Chwialkowski
6. Dino Sejdinovic

Citations in the corpus

• Toward Causal Representation Learning (2021, background): “These contexts can come from interventions [187], [191], [220], nonstationary time series [101], [116], [192], or multiple views [90], [114].”
• Towards Causal Representation Learning (2021, background): “These contexts can come from interventions [218, 189, 192], non-stationary time series [117, 100, 193] or multiple views [89, 115].”
• Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, background): “a time stamp or environment index) renders the sources conditionally independent [45, 54, 55, 58].”