Tagger: Deep Unsupervised Perceptual Grouping

By Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hotloo Hao, Harri Valpola, J. Urgen Schmidhuber. 2016

169 citations recorded by the source corpus.

We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features. Rather than being trained for any specific segmentation, our framework learns the grouping process in an unsupervised manner or alongside any supervised task. By enriching the representations of a neural network, we enable it to group the representations of different objects in an iterative manner. By allowing the system to amortize the iterative inference of the groupings, we achieve very fast convergence. In contrast to many other recently proposed methods for addressing multi-object scenes, our system does not assume the inputs to be images and can therefore directly handle other modalities. For multi-digit classification of very cluttered images that require texture segmentation, our method offers improved classification performance over convolutional networks despite being fully connected. Furthermore, we observe that our system greatly improves on the semi-supervised result of a baseline Ladder network on our dataset, indicating that segmentation can also improve sample efficiency.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, background): “[24] Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hao, Harri Valpola, and Jürgen Schmidhuber.”
Invariant Information Clustering for Unsupervised Image Classification and Segmentation (2018, methodology): “Optimising for function outputs to be persistent through spatio-temporal or non-material distortion is an idea shared by IIC with several works, including exemplars [17], IMSAT [27], proximity prediction [29], the denoising objective of Tagger [21], temporal slowness constraints [53], and optimising”
Amortised MAP Inference for Image Super-resolution (2016, methodology): “Bayes-optimal denoising approximately learn to take a gradient step along the log-probability of the data distribution (Alain & Bengio, 2014; Rasmus et al., 2015; Greff et al., 2016).”
Reconciling deep learning with symbolic artificial intelligence: representing objects and relations (2019, background): “In a similar vein, some authors have devised architectures that iteratively generate an image by rendering one object at a time [25,26], which also obliges the system to learn representations that comprise separately represented discrete entities.”
Dynamic Routing Between Capsules (2017, background): “Hinton et al. propose the task of segmenting and recognizing highly overlapping digits (Hinton et al. [2000] and others have tested their networks in a similar domain (Goodfellow et al. [2013], Ba et al. [2014], Greff et al. [2016]).”
Towards Deep Symbolic Reinforcement Learning (2016, background): “Deep learning is therefore already a viable solution to the symbol grounding problem in the supervised case, and for the unsupervised case, which is essential for a full solution, rapid progress is being made [20, 21, 22, 23, 24].”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “This has been addressed by simultaneously learning an amortized iterative inference process based on de-noising (Greff et al., 2016), generalized expectation-maximization (Greff et al.”
Towards causal generative scene models via competition of experts (2020, methodology): “Clustering & spatial mixture models One line of work (Greff et al., 2016; 2017; Van Steenkiste et al., 2018) approaches the perceptual grouping task of decomposing scenes into components by viewing separate regions r t as clusters.”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Finally, while this paper focuses on evaluating the benefits of disentangled features, these are complementary to recent work that focuses on the unsupervised “disentangling” of images into compositional primitives given by object-like representations [17, 23, 24, 22, 60, 74, 75].”
Multi-Object Representation Learning with Iterative Variational Inference (2019, methodology): “We also probed the limits of our current setup by applying IODINE to the Textured MNIST dataset (Greff et al., 2016) and to ImageNet, testing how it would deal with texture-segmentation and more complex real-world data (Figure 11).”
MONet: Unsupervised Scene Decomposition and Representation (2019, methodology): “While some unsupervised methods for scene decomposition have been developed, their performance is limited to very simple visual data [Greﬀ et al., 2016, 2017, van Steenkiste et al., 2018, Eslami et al., 2016].”
Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects (2018, methodology): “Greff, Rasmus, et al., 2016 and Greff, Steenkiste, et al., 2017 learn to decompose images into separate entities by iterative reﬁnement of spatial clusters using either learned updates or the Expectation Maximization algorithm; Ilin et al., 2017 and Steenkiste et al., 2018 extend these approaches…”
Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions (2018, methodology): “A closely related approach to N-EM is the TAG framework (Greff et al., 2016), which utilizes a similar mechanism to perform inference over group representations, but in addition performs inference over the group assignments.”
Independently Controllable Features (2017, background): “The binding problem has seen some attention in the representation learning literature (Minin et al., 2012; Greff et al., 2016), but still remains mostly unsolved.”
Neural Expectation Maximization (2017, methodology): “The method most closely related to our approach is Tagger [7], which also features K copies of a neural network that work together by reconstructing different parts of the input.”
GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations (2019, background): “Existing solutions, however, either require expensive inference procedures (Greff et al. [7, 15, 16]) or lack a proper probabilistic formulation (Burgess et al. [6]).”
Learning to Decompose and Disentangle Representations for Video Prediction (2018, methodology): “In particular, some previous works use methods such as Expectation Maximization to perform perceptual grouping and discover individual objects in videos [11, 12, 36].”
Independently Controllable Factors (2017, background): “The binding problem has received some attention in the representation learning literature (Minin et al., 2012; Greff et al., 2016), but still remains mostly unsolved.”
Inductive biases for deep learning of higher-level cognition (2020, background): “High-level variables play a causal role Learning representations of latent entities/attributes (Le Roux et al., 2011; Eslami et al., 2016; Greff et al., 2016) (Raposo et al.”
Conditional Object-Centric Learning from Video (2021, background): “Object-centric representation learning There is a rich literature on learning object representations from static scenes (Greff et al., 2016; Eslami et al., 2016; Greff et al., 2017; 2019; Burgess et al., 2019; Engelcke et al., 2020; Crawford & Pineau, 2019; Lin et al., 2020; Locatello et al., 2020; ”