Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking

By Eric Crawford, Joelle Pineau. 2019

69 citations recorded by the source corpus.

The ability to detect and track objects in the visual world is a crucial skill for any intelligent agent, as it is a necessary precursor to any object-level reasoning process. Moreover, it is important that agents learn to track objects without supervision (i.e. without access to annotated training videos) since this will allow agents to begin operating in new environments with minimal human assistance. The task of learning to discover and track objects in videos, which we call unsupervised object tracking, has grown in prominence in recent years; however, most architectures that address it still struggle to deal with large scenes containing many objects. In the current work, we propose an architecture that scales well to the large-scene, many-object setting by employing spatially invariant computations (convolutions and spatial attention) and representations (a spatially local object specification scheme). In a series of experiments, we demonstrate a number of attractive features of our architecture; most notably, that it outperforms competing methods at tracking objects in cluttered scenes with many objects, and that it can generalize well to videos that are larger and/or contain more objects than videos encountered during training.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SCALOR: Generative World Models with Scalable Object Representations (2019, methodology): “In Crawford & Pineau (2019b), the authors demonstrated that this sequential discovery can easily fail beyond the scale of a few objects.”
ROOTS: Object-Centric Representation and Rendering of 3D Scenes (2020, methodology): “For 2D fully observable images, extracting object regions can be solved by recent 2D scene decomposition methods (Eslami et al., 2016; Crawford and Pineau, 2019; Lin et al., 2020).”
Conditional Object-Centric Learning from Video (2021, background): “…2019; Engelcke et al., 2020; Crawford & Pineau, 2019; Lin et al., 2020; Locatello et al., 2020; Du et al., 2021a) or videos (van Steenkiste et al., 2018; Kosiorek et al., 2018; Stelzner et al., 2019; Kipf et al., 2020; Crawford & Pineau, 2020; Creswell et al., 2021) without explicit supervision.”
Opening up Open World Tracking (2021, methodology): “Methods by [45, 19, 39] learn to decompose scenes into a set of object instances based on the input video sequence in an unsupervised manner.”