Learning Invariance from Transformation Sequences

By Peter Földiák. 1991

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, methodology): “This resulted in some early attempts to structure representations with losses that did not rely on reconstruction: notably, slow feature analysis [19, 63] used the persistence of objects in videos to learn representations that generalized across views.”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “This kind of temporal coherence has a long history in visual learning literature [18, 56], and contemporaneous work shows strong improvements on modern detection datasets [54].”
Multi-task Self-Supervised Visual Learning (2017, background): “Another is to note that objects generally appear across many frames: thus, we can train features to remain invariant as a video progresses [11, 24, 42, 43, 47].”
Regularizing and Optimizing LSTM Language Models (2017, background): “TAR falls under the broad category of slowness regularizers (Hinton, 1989; Földiák, 1991; Luciw & Schmidhuber, 2012; Jonschkowski & Brock, 2015) which penalize the model from producing large changes in the hidden state.”
Unsupervised Learning of Visual Representations Using Videos (2015, background): “Therefore, researchers have started focusing on learning feature representations using videos [11, 53, 27, 43, 56, 16, 48, 34, 45].”
Toward Causal Representation Learning (2021, background): “The hypothesis is also related to ideas of looking for features that vary slowly [70], [270].”
View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation (2016, neutral): “The computational insight enabling depth-rotation tolerant representations to be learned from experience is that, due to properties of how objects move in the world, temporally adjacent frames (the giw) almost always depict the same object [22, 48, 15, 58, 10, 26].”
Towards Causal Representation Learning (2021, background): “The hypothesis is also related to ideas of looking for features that vary slowly [69, 270].”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “More generally this observation model applies to many natural scenes with moving objects (Földiák, 1991).”
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA (2016, background): “The idea is to find features which change as slowly as possible, originally proposed in [6] for learning invariant features.”
How does the brain solve visual object recognition? (2012, neutral): “More specifically, because each object’s identity is temporally stable, different retinal images of the same object tend to be temporally contiguous (Fazl et al., 2009; Foldiak, 1991; Stryker, 1992;Wallis and Rolls, 1997; Wiskott and Sejnowski, 2002).”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “This idea of continuity is used in a great number of artiﬁcial neural networks and related models (Wiskott and Sejnowski, 2002; Földiák, 2008; Mobahi et al., 2009).”
How Does Our Visual System Achieve Shift and Size Invariance (2004, methodology): “However, for clarity the left type of illustration will be used in the following. objective of extracting slowly changing features from the quickly varying sensory input (F¨oldi´ak, 1991), which then leads to a robust and invariant representation of the environment.”
A quantitative theory of immediate visual recognition. (2007, background): “Stereo (in addition to motion) is likely to play an important role in the learning of invariances such as position and size invariance via a correlation-based rule such as the trace rule (Földiák, 1991).”