IM2CAD

By Hamid Izadinia, Qi Shan, S. M. Seitz. 2016

215 citations recorded by the source corpus.

Given a single photo of a room and a large database of furniture CAD models, our goal is to reconstruct a scene that is as similar as possible to the scene depicted in the photograph, and composed of objects drawn from the database. We present a completely automatic system to address this IM2CAD problem that produces high quality results on challenging imagery from interior home design and remodeling websites. Our approach iteratively optimizes the placement and scale of objects in the room to best match scene renderings to the input photo, using image comparison metrics trained via deep convolutional neural nets. By operating jointly on the full scene at once, we account for inter-object occlusions. We also show the applicability of our method in standard scene understanding benchmarks where we obtain significant improvement.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image (2016, methodology): “In another recent approach, called IM2CAD, CAD model of a room is generated from a single RGB image [14].”
Taskonomy: Disentangling Task Transfer Learning (2018, background): “Room Layout Estimation Estimating and aligning a 3D bounding box is a mid-level task that includes vanishing point estimation as a sub-problem, and has applications for robotic navigation [38], scene reconstruction [15], and augmented reality [8].”
Holistic++ Scene Understanding: Single-View 3D Holistic Scene Parsing and Human Pose Estimation With Human-Object Interaction and Physical Commonsense (2019, methodology): “Single-view 3D Scene Reconstruction: Single-view 3D scene reconstruction has three main approaches: (i) Pre-dict room layouts by extracting geometric features to rank 3D cuboids proposals [62, 40, 17, 61].”
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene (2017, methodology): “Similarly, in CAD retrieval scenarios, one assumes the object [3, 4, 13, 19, 20] or scene [17] can be represented in terms of a pre-determined dictionary of shapes; a great deal of earlier work [12, 18, 27] tackled this with a dictionary of box models.”
Cross-View Semantic Segmentation for Sensing Surroundings (2019, background): “room layout estimation [50, 22, 43, 19, 16], free space estimation [17]).”
OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets (2020, methodology): “Several methods build 3D models for indoor scenes from a single image [26] or scans [5, 6, 12, 14].”
Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve (2020, background): “Izadinia and Seitz [20] and Huang et al. [17] propose to optimize for both scene layouts and CAD models of objects from image input, leveraging analysis-by-synthesis approaches; these methods involve costly optimization for each input image (minutes to hours).”
Scan2CAD: Learning CAD Model Alignment in RGB-D Scans (2018, methodology): “We present Scan2CAD 1 , a novel data-driven method that learns to align clean 3D CAD models from a shape database to the noisy and incomplete geometry of an RGB-D scan.”
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild (2020, methodology): “Earlier methods optimize a deformable shape model to image silhouettes [39,32,5,30,25].”
Improved Adversarial Systems for 3D Object Generation and Reconstruction (2017, background): “Several of these shape models have been shown in practical applications such as interactive editing [20] and reconstructing room geometry from a single image [4, 24, 13].”
What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction (2021, unknown): “Most work on invisible surfaces focuses on single objects with voxels [10, 20, 23], point-clouds [16, 38], CAD models [29] and meshes [21, 22].”
OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets (2020, unknown): “Several recent methods can build CAD models for indoor scenes from a single image [22] or a scanned point cloud [2, 9, 12].”
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, background): “For instance, Izadinia et al. [136] proposed an approach that is based on recognizing objects in indoor scenes, inferring room geometry, and optimizing 3D object poses and sizes in the room to best match synthetic renderings to the input photo.”