SceneFormer: Indoor Scene Generation with Transformers
By Xinpeng Wang, Chandan Yeshwanth, Matthias Nießner. 2020
216 citations recorded by the source corpus.
We address the task of indoor scene generation by generating a sequence of objects, along with their locations and orientations conditioned on a room layout. Large-scale indoor scene datasets allow us to extract patterns from user-designed indoor scenes, and generate new scenes based on these patterns. Existing methods rely on the 2D or 3D appearance of these scenes in addition to object positions, and make assumptions about the possible relations between objects. In contrast, we do not use any appearance information, and implicitly learn object relations using the self-attention mechanism of transformers. We show that our model design leads to faster scene generation with similar or improved levels of realism compared to previous methods. Our method is also flexible, as it can be conditioned not only on the room layout but also on text descriptions of the room, using only the cross-attention mechanism of transformers. Our user study shows that our generated scenes are preferred to the state-of-the-art FastSynth scenes 53.9% and 56.7% of the time for bedroom and living room scenes, respectively. At the same time, we generate a scene in 1.48 seconds on average, 20% faster than FastSynth.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Luc Van Gool
- Roozbeh Mottaghi
- Ali Farhadi
- Aniruddha Kembhavi
- Kristen Grauman
- Abhinav Kumar Gupta
- Jitendra Malik
Citations in the corpus
- LayoutGPT: Compositional Visual Planning and Generation with Large Language Models (2023, background): “Visual layout is an essential symbolic representation that has been widely studied as it reflects the global compositions of a visual space [32, 53, 47, 45, 24, 33].”
- A Survey on Vision Transformer (2020, methodology): “[234] proposed SceneFormer to utilize transformer in 3D indoor scene generation.”
- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, background): “10 [113] Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner.”
- DeepCAD: A Deep Generative Network for Computer-Aided Design Models (2021, background): “Not simply is the parameter quantization a follow-up of the common practice for training Transformer-based networks [36, 31, 44].”