Recurrent Ladder Networks

By Isabeau Prémont-Schwarz, Alexander Ilin, Tele Hotloo Hao, Antti Rasmus, Rinu Boney, Harri Valpola. 2017

41 citations recorded by the source corpus.

We propose a recurrent extension of the Ladder networks whose structure is motivated by the inference required in hierarchical latent variable models. We demonstrate that the recurrent Ladder is able to handle a wide variety of complex learning tasks that benefit from iterative inference and temporal modeling. The architecture shows close-to-optimal results on temporal modeling of video data, competitive results on music modeling, and improved perceptual grouping based on higher order abstractions, such as stochastic textures and motion cues. We present results for fully supervised, semi-supervised, and unsupervised tasks. The results suggest that the proposed architecture and principles are powerful tools for learning a hierarchy of abstractions, learning iterative inference and handling temporal information.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects (2018, neutral): “and Greff, Steenkiste, et al., 2017 learn to decompose images into separate entities by iterative reﬁnement of spatial clusters using either learned updates or the Expectation Maximization algorithm; Ilin et al., 2017 and Steenkiste et al., 2018 extend these approaches to videos, achieving very simi”
Neural Expectation Maximization (2017, methodology): “RTagger [15]: a recurrent extension of Tagger that does support sequential data was developed concurrent to this work.”
Multi-period Time Series Modeling with Sparsity via Bayesian Variational Inference (2017, background): “Although some work such as Recurrent Ladder Network [18] applied multi-layer latent variables to model complex time series, the dynamics of all latent variables is one-step, which cannot capture transitions of time series in multiple periods.”
Generative Modeling of Infinite Occluded Objects for Compositional Scene Representation (2019, methodology): “RTagger (Prémont-Schwarz et al., 2017) extends Tagger to sequential data by substituting the Ladder Network with a Recurrent Ladder Network.”
SimVP: Simpler yet Better Video Prediction (2022, unknown): “2014-2015 [41, 55, 71] [36, 46] [40] 2016-2017 [38, 47, 63, 67] [3, 9, 15, 32, 62] [20, 35, 59] 2018-2019 [22, 43, 56, 65, 69, 74] [6, 10, 61, 66, 73] [70] [16, 29, 72] 2020-2021 [68] [18, 23] [48] [7, 54]”
R-SQAIR: Relational Sequential Attend, Infer, Repeat (2019, methodology): “RTagger [19] replaces the Ladder Network by a Recurrent Ladder Network, thus extending Tagger to sequential settings.”