Slow Feature Analysis: Unsupervised Learning of Invariances

By Laurenz Wiskott, Terrence J. Sejnowski. 2002

1,547 citations recorded by the source corpus.

Subfields: Analytical Chemistry, Cognitive Neuroscience, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Cognitive Neuroscience

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

What makes ImageNet good for transfer learning? (2016, background): “The most well-known such criteria are image reconstruction [7, 38, 31, 29, 34, 23] (see [6] for a comprehensive overview) and feature slowness [48, 18].”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “As a result, and with the recent emergence of research video data sets [1, 59], videos have been investigated in the context of self-supervision (for example, [40, 64, 63, 28, 65, 73, 41, 51, 42, 3, 2]).”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, methodology): “This resulted in some early attempts to structure representations with losses that did not rely on reconstruction: notably, slow feature analysis [19, 63] used the persistence of objects in videos to learn representations that generalized across views.”
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “Learning good image representations is a key challenge in computer vision [1, 2, 3] as it allows for efficient training on downstream tasks [4, 5, 6, 7].”
Representation Learning with Contrastive Predictive Coding (2018, background): “These ’slow features’ [13] that span many time steps are often more interesting (e.g., phonemes and intonation in speech, objects in images, or the story line in books.).”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “This kind of temporal coherence has a long history in visual learning literature [18, 56], and contemporaneous work shows strong improvements on modern detection datasets [54].”
Multi-task Self-Supervised Visual Learning (2017, background): “ temporally order a set of frames sampled from a video [23]. Another is to note that objects generally appear across many frames: thus, we can train features to remain invariant as a video progresses [11, 24, 42, 43, 47]. Finally, motion cues can separate foreground objects from background. Neural n”
Learning Robust and Multilingual Speech Representations (2020, neutral): “On the other hand, if the model predicts further in the future, the representations will need to infer “slow features” (Wiskott and Sejnowski, 2002); more global structures such as phonemes, words and utterances in speech.”
Learning to See by Moving (2015, background): “Another line of work has focused on learning features that are invariant to transformations irrelevant to the identity of objects, either from images [14, 34], or from video [41].”
Learning Image Representations Tied to Ego-Motion (2015, methodology): “Most relevant to our work are feature learning methods based on temporal coherence and “slow feature analysis” [32, 10, 22].”
Time-Contrastive Networks: Self-Supervised Learning from Video (2017, background): “A number of prior works use temporal coherence [28, 29, 30, 31].”
Video (language) modeling: a baseline for generative models of natural videos (2014, methodology): “Wiskott & Sejnowski (2002) have proposed instead a method based on slowness of features through time, an idea later extended in a bilinear model by Gregor & LeCun (2010).”
Unsupervised Learning of Visual Representations Using Videos (2015, background): “Therefore, researchers have started focusing on learning feature representations using videos [11, 53, 27, 43, 56, 16, 48, 34, 45].”
UvA-DARE (Digital Academic Repository) Self-Supervised Video Representation Learning With Odd-One-Out Networks (neutral): “Exploiting the spatio-temporal coherence in videos has been investigated before for unsupervised learning [3, 32, 45].”
Toward Causal Representation Learning (2021, background): “The hypothesis is also related to ideas of looking for features that vary slowly [70], [270].”
Deep learning in neural networks: An overview (2014, methodology): “RL combined with UL based on Slow Feature Analysis (Wiskott and Sejnowski, 2002; Kompella et al., 2012) enabled a real humanoid robot to learn skills from raw high-dimensional video streams (Luciw et al.”