Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks

By Eric Crawford, Joelle Pineau. 2019

197 citations recorded by the source corpus.

There are many reasons to expect an ability to reason in terms of objects to be a crucial skill for any generally intelligent agent. Indeed, recent machine learning literature is replete with examples of the benefits of object-like representations: generalization, transfer to new tasks, and interpretability, among others. However, in order to reason in terms of objects, agents need a way of discovering and detecting objects in the visual world - a task which we call unsupervised object detection. This task has received significantly less attention in the literature than its supervised counterpart, especially in the case of large images containing many objects. In the current work, we develop a neural network architecture that effectively addresses this large-image, many-object setting. In particular, we combine ideas from Attend, Infer, Repeat (AIR), which performs unsupervised object detection but does not scale well, with recent developments in supervised object detection. We replace AIR’s core recurrent network with a convolutional (and thus spatially invariant) network, and make use of an object-specification scheme that describes the location of objects with respect to local grid cells rather than the image as a whole. Through a series of experiments, we demonstrate a number of features of our architecture: that, unlike AIR, it is able to discover and detect objects in large, many-object scenes; that it has a significant ability to generalize to images that are larger and contain more objects than images encountered during training; and that it is able to discover and detect objects with enough accuracy to facilitate non-trivial downstream processing.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, methodology): “Our attention-based routing procedure could also be employed in conjunction with patch-based decoders, used in architectures such as AIR [26], SQAIR [40], and related approaches [41–44], as an alternative to the typically employed autoregressive encoder [26, 40].”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “3 How to learn part-whole disentanglement for more complicated patterns in large vision datasets? A specific area within unsupervised disentanglement in computer vision is to semantically segment the image into different parts, where each part represents an object or a part of an object (Crawford an”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “Even more recent approaches, such as SPAIR (Crawford and Pineau, 2019), SPACE (Lin et al., 2020), and SCALOR (Jiang et al., 2020), expand on this by incorporating explicit features for the presence of an object and its bounding box into each spatial slot.”
SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition (2020, methodology): “This type of structuring has been used in (Redmon et al., 2016; Santoro et al., 2017; Crawford & Pineau, 2019).”
ROOTS: Object-Centric Representation and Rendering of 3D Scenes (2020, methodology): “, 2019) in a 3D-aware fashion through partial observations of scenes without supervision, in machine learning this problem has only been tackled for simple 2D fully observable images (Eslami et al., 2016; Lin et al., 2020; Crawford and Pineau, 2019; Greff et al., 2017, 2019; Locatello et al., 2020; ”
Hierarchical Relational Inference (2020, methodology): “…representations from raw visual observations typically make use of mixture models (Greff, van Steenkiste, and Schmidhuber 2017; Greff et al. 2019) or attention (Eslami et al. 2016; Kosiorek et al. 2018; Stanić and Schmidhuber 2019; Burgess et al. 2019; Crawford and Pineau 2019; Jiang et al. 2020).”
R-SQAIR: Relational Sequential Attend, Infer, Repeat (2019, background): “SPAIR [6] scales AIR to scenarios with many objects and SuPAIR[28] improves speed and robustness of learning in AIR.”
Scalable Object-Oriented Sequential Generative Models (2019, methodology): “The implementation of the rendering process is the same as in SPAIR (Crawford & Pineau, 2019), except that we process the objects in parallel.”
Generative Neurosymbolic Machines (2020, background): “For example, in object-centric latent variable models [13, 30, 10, 24], a univariate Gaussian distribution p(zwhere x ) = N (μx, σx) can be introduced to define a symbolic prior on the x-coordinate of an object in an image.”
SCALOR: Generative World Models with Scalable Object Representations (2019, methodology): “In Crawford & Pineau (2019b), the authors demonstrated that this sequential discovery can easily fail beyond the scale of a few objects.”
Conditional Object-Centric Learning from Video (2021, background): “…representations from static scenes (Greff et al., 2016; Eslami et al., 2016; Greff et al., 2017; 2019; Burgess et al., 2019; Engelcke et al., 2020; Crawford & Pineau, 2019; Lin et al., 2020; Locatello et al., 2020; Du et al., 2021a) or videos (van Steenkiste et al., 2018; Kosiorek et al., 2018;…”