SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

By Zhixuan Lin, Yi-Fu Wu, Skand Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, Sungjin Ahn. 2020

280 citations recorded by the source corpus.

The ability to decompose complex multi-object scenes into meaningful abstractions like objects is fundamental to achieve higher-level cognition. Previous approaches for unsupervised object-oriented scene representation learning are either based on spatial-attention or scene-mixture approaches and limited in scalability which is a main obstacle towards modeling real-world scenes. In this paper, we propose a generative latent variable model, called SPACE, that provides a unified probabilistic modeling framework that combines the best of spatial-attention and scene-mixture approaches. SPACE can explicitly provide factorized object representations for foreground objects while also decomposing background segments of complex morphology. Previous models are good at either of these, but not both. SPACE also resolves the scalability problems of previous methods by incorporating parallel spatial-attention and thus is applicable to scenes with a large number of objects without performance degradations. We show through experiments on Atari and 3D-Rooms that SPACE achieves the above properties consistently in comparison to SPAIR, IODINE, and GENESIS. Results of our experiments can be found on our project website: this https URL

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, methodology): “[44] Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, and Sungjin Ahn.”
Toward Causal Representation Learning (2021, background): “Note that the problem of object-centric representation learning [11], [40], [84], [87], [88], [138], [155], [160], [255], [262] can also be considered a special case of disentangled factorization as discussed here.”
Towards Causal Representation Learning (2021, background): “learning [10, 39, 83, 86, 87, 138, 155, 160, 262, 255] can also be considered a special case of disentangled factorization as discussed here.”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “Even more recent approaches, such as SPAIR (Crawford and Pineau, 2019), SPACE (Lin et al., 2020), and SCALOR (Jiang et al., 2020), expand on this by incorporating explicit features for the presence of an object and its bounding box into each spatial slot.”
SCALOR: Generative World Models with Scalable Object Representations (2019, methodology): “A similar approach is used in SPAIR (Crawford & Pineau, 2019b), but it infers the object latent representations sequentially and thus is difﬁcult to scale to a large number of objects (Lin et al., 2020).”
Object Scene Representation Transformer (2022, background): “, generating objects ordered by their distance to the camera [1, 21].”
Investigating Simple Object Representations in Model-Free Deep Reinforcement Learning (2020, unknown): “More recently, this inductive bias served as a source of inspiration for several recent advancements in deep learning, such as Kulkarni et al. (2016), van Steenkiste et al. (2018), Kulkarni et al. (2019), Veerapaneni et al. (2019), and Lin et al. (2020).”
Better Set Representations For Relational Reasoning (2020, background): “Unsupervised object detection also focuses on object representations from unstructured data [4, 6, 11].”
Unsupervised Object Keypoint Learning using Local Spatial Predictability (2020, background): “, 2018) or hybrid models that are combinations of both (Lin et al., 2020).”
Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views (2021, background): “More recently, several works [4, 9, 18, 17, 1] have succeeded in approximating the factorized posterior p ( z 1 , z 2 , ..., z K | x ) within the VAE framework, achieving impressive unsupervised object-level scene factorization.”
Generative Neurosymbolic Machines (2020, background): “Unlike SPACE, the proposed model generates the latent structure map from the global representation, which is distributed and groundable (binding learnable).”
ROOTS: Object-Centric Representation and Rendering of 3D Scenes (2020, methodology): “For 2D fully observable images, extracting object regions can be solved by recent 2D scene decomposition methods (Eslami et al., 2016; Crawford and Pineau, 2019; Lin et al., 2020).”