Amodal Completion and Size Constancy in Natural Scenes

By Abhishek Kar, Shubham Tulsiani, João Carreira, Jitendra Malik. 2015

66 citations recorded by the source corpus.

We consider the problem of enriching current object detection systems with veridical object sizes and relative depth estimates from a single image. There are several technical challenges to this, such as occlusions, lack of calibration data and the scale ambiguity between object size and distance. These have not been addressed in full generality in previous work. Here we propose to tackle these issues by building upon advances in object recognition and using recently created large-scale datasets. We first introduce the task of amodal bounding box completion, which aims to infer the the full extent of the object instances in the image. We then propose a probabilistic framework for learning category-specific object size distributions from available annotations and leverage these in conjunction with amodal completions to infer veridical sizes of objects in novel images. Finally, we introduce a focal length prediction approach that exploits scene recognition to overcome inherent scale ambiguities and demonstrate qualitative results on challenging real-world scenes.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images (2015, background): “Beyond predicting only the visible part of an object, [14] further extended RCNN to estimate the amodal box for the whole object.”
Semantic Amodal Segmentation (2015, background): “Finally there has been some algorithmic work on amodal completion [14, 15, 38, 19].”
Efficient inference in occlusion-aware generative models of images (2015, methodology): “In this case, the foreground object may obscure almost all of the background object, yet people are still able to detect that there are two objects present, to correctly segment out both of them, and even to amodally complete the hidden parts of the occluded object (cf., Kar et al. (2015)).”
Visual Concepts and Compositional Voting (2017, methodology): “It introduces difficulties for tasks such as object detection [16] or segmentation [20].”
Zoom Better to See Clearer: Human Part Segmentation with Auto Zoom Net (2015, background): “In practice, directly estimating object scale in DCNN has also been discussed in works for object-detections or related fields (Hariharan et al., 2015; Dai et al., 2015; Ren et al., 2015; Redmon et al., 2015; Huang et al., 2015; Kar et al., 2015), while this work provides a more general view to unde”
Im2Calories: Towards an Automated Mobile Vision Food Diary (2015, background): “In particular, it requires solving various problems, such as: fine-grained recognition (to distinguish subtly different forms of food); hierarchical label spaces (to handle related labels); open-world recognition (to handle an unbounded number of class names); visual attribute recognition (to distin”
Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net (2015, unknown): “ting object scale in DCNN has also been discussed in works for object-detections or related ﬁelds (Hariharan et al., 2015; Dai et al., 2015; Ren et al., 2015; Redmon et al., 2015; Huang et al., 2015; Kar et al., 2015), while this work provides a more general view to understand it based on a perspect”
Self-Supervised Scene De-Occlusion (2020, methodology): “[32] use keypoint annotations to align 3D object templates to 2D image objects, so as to generate the ground truth of amodal bounding boxes.”
SAIL-VOS: Semantic Amodal Instance Level Video Object Segmentation – A Synthetic Dataset and Baselines (2019, methodology): “…been used for optical ﬂow [42, 41, 9, 7, 69, 81, 58], robustness estimation of features [51], visual odometry [35, 81], hand tracking [87], shape from shading [82], semantic segmentation [83, 50, 81, 58], amodal segmentation [52, 24], multi-view stereopsis [47] and human pose estimation [89, 26].”
Amodal Instance Segmentation (2016, methodology): “For instance, we can estimate the physical dimensions of an object in the real world using its amodal bounding box, as demonstrated by Kar et al. [21] The fact that all these occlusion reasoning problems can be reduced to amodal instance segmentation implies that amodal instance segmentation is…”