ROOTS: Object-Centric Representation and Rendering of 3D Scenes
By Chang Chen, Fei Deng, Sungjin Ahn. 2020
57 citations recorded by the source corpus.
A crucial ability of human intelligence is to build up models of individual 3D objects from partial scene observations. Recent works achieve object-centric generation but without the ability to infer the representation, or achieve 3D scene representation learning but without object-centric compositionality. Therefore, learning to represent and render 3D scenes with object-centric compositionality remains elusive. In this paper, we propose a probabilistic generative model for learning to build modular and compositional 3D object models from partial observations of a multi-object scene. The proposed model can (i) infer the 3D object representations by learning to search and group object areas and also (ii) render from an arbitrary viewpoint not only individual objects but also the full scene by compositing the objects. The entire learning process is unsupervised and end-to-end. In experiments, in addition to generation quality, we also demonstrate that the learned representation permits object-wise manipulation and novel scene generation, and generalizes to various settings. Results can be found on our project website: this https URL
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- J. Urgen Schmidhuber
- Mehdi S. M. Sajjadi
- Klaus Greff
- Aravindh Mahendran
- Sjoerd van Steenkiste
- Leonidas J. Guibas
- Charles Ruizhongtai Qi
- Andreas Geiger
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrea Vedaldi
- Dacheng Tao
- Ming-Hsuan Yang
- Jitendra Malik
- Trevor Darrell
- Xiaogang Wang
- Andrew Zisserman
Citations in the corpus
- On the Binding Problem in Artificial Neural Networks (2020, background): “Further improvements can be made by assuming access to multiple different views to explicitly model 3D structure at a representational level (Chen et al., 2020; Nanbo et al., 2020).”
- Object Scene Representation Transformer (2022, methodology): “We also tried further alternatives, such as allowing the model only 8 slots, but instead of larger dimensionality, without success.”