Self-organizing neural network that discovers surfaces in random-dot stereograms

By Suzanna Becker, Geoffrey E. Hinton. 1992

Subfields: Computer Vision and Pattern Recognition, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neuroscience

Citations in the corpus

Masked Autoencoders Are Scalable Vision Learners (2021, background): “Recently, contrastive learning [3, 22] has been popular, e.g ., [62, 43, 23, 7], which models image similarity and dissimilarity (or only similarity [21, 8]) between two or more views.”
A Simple Framework for Contrastive Learning of Visual Representations (2020, background): “The idea of making representations of an image agree with each other under small transformations dates back to Becker & Hinton (1992).”
Big Self-Supervised Models are Strong Semi-Supervised Learners (2020, background): “SimCLR learns representations by maximizing agreement [26] between differently augmented views of the same data example via a contrastive loss in the latent space.”
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “Many successful self-supervised learning approaches build upon the cross-view prediction framework introduced in [63].”
On Mutual Information Maximization for Representation Learning (2019, background): “The usual problem setup dates back at least to Becker and Hinton (1992) and can conceptually be described as follows: For a given image X , let X (1) and X (2) be different, possibly overlapping views of X , for instance the top and bottom halves of the image.”
Invariant Information Clustering for Unsupervised Image Classification and Segmentation (2018, background): “One of the earliest works to do so is by Becker and Hinton [3].”
Representation Learning: A Review and New Perspectives (2012, methodology): “The principle of identifying slowly moving/changing factors in temporal/spatial data has been investigated by many (Becker and Hinton, 1992; Wiskott and Sejnowski, 2002; Hurri and Hyvärinen, 2003; Körding et al., 2004; Cadieu and Olshausen, 2009) as a principle for finding useful representations.”
On the Binding Problem in Artificial Neural Networks (2020, background): “In this way learned object representations can also benefit from known inductive biases that focus on feature hierarchies, invariances, and spatio-temporal coherence (Becker and Hinton, 1992), sparsity (Olshausen and Field, 1996), or non-Euclidean feature spaces (Nickel and Kiela, 2017).”
How to Represent Part-Whole Hierarchies in a Neural Network (2021, background): “To prevent collapse, contrastive learning [Becker and Hinton, 1992, Paccanaro and Hinton, 2001, van den Oord et al., 2018] uses negative examples and tries to make representations that should agree be close while maintaining separation between representations which should not agree 21 .”
Learning a Multi-View Stereo Machine (2017, background): “For the binocular case, Becker and Hinton [1] demonstrated that a neural network can learn to predict a depth map from random dot stereograms.”
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels (2020, methodology): “6 Related Work Computer Vision Data augmentation via image transformations has been used to improve generalization since the inception of convolutional networks [5, 40, 30, 9, 8].”
Barlow Twins: Self-Supervised Learning via Redundancy Reduction (2021, background): “This intriguing result is in stark contrast with other popular methods for SSL, such as S IM CLR (Chen et al., 2020a) and BYOL (Grill et al., 2020), for which increasing the dimensionality of the embeddings rapidly saturates performance.”
Deep Learning of Representations: Looking Forward (2013, background): “• Temporal and spatial coherence: this prior introduced in Becker and Hinton (1992) is similar to the natural clustering assumption but concerns sequences of observations: consecutive (from a sequence) or spatially nearby observations tend to be easily predictable from each other.”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “Maximizing mutual information is a natural way of improving learning (Becker and Hinton, 1992; Mohamed and Rezende, 2015), and there are many other ways in which multiple modalities or processing streams could mutually train one an invariant signature of object identity which is independent of…”
Grounded spoken language acquisition: experiments in word learning (2003, background): “A promising path of research is to build machines which similarly integrate evidence across modalities to learn from naturally occurring data without supervision [ 12 ], [13].”
Slow Feature Analysis: Unsupervised Learning of Invariances (2002, background): “Becker and Hinton (1992) trained a network to discover disparity as an invariant variable of random-dot stereograms.”
The Helmholtz Machine (1995, background): “Figure 4 demonstrates the performance of the Helmholtz Machine in a hierarchical learning task (Becker & Hinton, 1992), showing that it is capable of extracting the structure underlying a complicated generative model.”
Conditional Negative Sampling for Contrastive Learning of Visual Representations (2020, background): “The basic idea of learning representations to be invariant under a family of transformations is an old one, having been explored with self-organizing maps (Becker & Hinton, 1992) and dimensionality reduction (Hadsell et al., 2006).”