SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation
By Yang Zhou, Zachary While, Evangelos Kalogerakis. 2019
113 citations recorded by the source corpus.
In this paper we propose a neural message passing approach to augment an input 3D indoor scene with new objects matching their surroundings. Given an input, potentially incomplete, 3D scene and a query location, our method predicts a probability distribution over object types that fit well in that location. Our distribution is predicted though passing learned messages in a dense graph whose nodes represent objects in the input scene and edges represent spatial and structural relationships. By weighting messages through an attention mechanism, our method learns to focus on the most relevant surrounding scene context to predict new scene objects. We found that our method significantly outperforms state-of-the-art approaches in terms of correctly predicting objects missing in a scene based on our experiments in the SUNCG dataset. We also demonstrate other applications of our method, including context-based 3D object recognition and iterative scene generation.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Matthias Nießner
- Shuran Song
- Ali Farhadi
- Roozbeh Mottaghi
- Sanja Fidler
- Aniruddha Kembhavi
- Amlan Kar
- Kiana Ehsani
In Computer Vision and Pattern Recognition
- Kristen Grauman
- Roozbeh Mottaghi
- Luc Van Gool
- Aniruddha Kembhavi
- Ali Farhadi
- Jitendra Malik
- Abhinav Kumar Gupta
- Trevor Darrell
Citations in the corpus
- Continuous Scene Representations for Embodied AI (2022, background): “However, others propose to create 3D scene graphs that encode 3D relationships between the components of a scene [2, 12, 20, 51, 66].”
- Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation (2020, background): “[76] use a scene graph representation and learn adding objects into existing scenes.”
- SceneFormer: Indoor Scene Generation with Transformers (2020, methodology): “Graph-based methods A natural representation of a scene is a graph [30, 15, 20], where each object is a node, and an edge is a relation between objects such as ‘is next to‘ or ‘is on top of‘.”
- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, background): “40 [131] Yang Zhou, Zachary While, and Evangelos Kalogerakis.”