Recovering Occlusion Boundaries from a Single Image

By Derek Hoiem, Andrew N. Stein, Alexei A. Efros, Martial Hebert. 2007

246 citations recorded by the source corpus.

Occlusion reasoning, necessary for tasks such as navigation and object search, is an important aspect of everyday life and a fundamental problem in computer vision. We believe that the amazing ability of humans to reason about occlusions from one image is based on an intrinsically 3D interpretation. In this paper, our goal is to recover the occlusion boundaries and depth ordering of free-standing structures in the scene. Our approach is to learn to identify and label occlusion boundaries using the traditional edge and region cues together with 3D surface and depth cues. Since some of these cues require good spatial support (i.e., a segmentation), we gradually create larger regions and use them to improve inference over the boundaries. Our experiments demonstrate the power of a scene-based approach to occlusion reasoning.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Holistically-Nested Edge Detection (2015, background): “…response fusion [38, 30, 28], (3) engagement of different levels of visual perception [16, 25, 37, 15] such as mid-level Gestalt law information [7], (4) structural information [6] and context [36], (5) holistic image prediction [23], (6) 3D geometry [13], and (7) occlusion boundaries [14].”
Designing deep networks for surface normal estimation (2014, background): “Another theme that has emerged both in the past [33, 17, 2, 19] and recently [16, 20, 10] is the reasoning between surface normals and the edges in the images.”
Monocular Object Instance Segmentation and Depth Ordering with CNNs (2015, background): “Our work is related to a line of work that aims at predicting depth ordering using weak object information as well as mid-level cues such as junctions and boundaries [15, 17, 19].”
An empirical study of context in object detection (2009, methodology): “Geometric context aims to capture the coarse 3D geometric structure of a scene, or the “surface layout” [16], which can be used to reason about supporting surfaces [17], occlusions [15], contact points, etc.”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, background): “10) Depth Estimation/Localization/SLAM: Estimating depth using cameras is a long-standing research topic [109], [129]–[131].”
Affinity CNN: Learning Pixel-Centric Pairwise Relations for Figure/Ground Embedding (2015, background): “For example, image segmentation follows from edge detection [12, 1, 2, 7, 4]; figure/ground, occlusion, or depth layering follows from reasoning over discrete contours or regions [27, 16, 21, 36, 18] with some systems also reliant on motion cues [30, 15, 32, 31].”
Ieee Transactions on Pattern Analysis and Machine Intelligence 1 3d Reasoning from Blocks to Stability (background): “The first attempts for geometric inference from a single color image were proposed in [1], [3] and [4] for estimating the depth of each segment using only color features.”
Second-order constrained parametric proposals and sequential search-based structured prediction for semantic segmentation in RGB-D images (2015, background): “Some of the more recent work has focused on the analysis of cluttered indoor scenes [22, 29, 16, 17, 20].”
Unsupervised Monocular Depth Estimation with Left-Right Consistency (2016, background): “Explicitly reasoning about occlusion during training [23, 25] could improve these issues.”
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling (2014, methodology): “We determine this weighting by applying a sigmoidal function to the gradients, which we found to produce more pleasing inferred depth maps than using other boundary detecting schemes such as [32], [33].”
LabelMe: Online Image Annotation and Applications (2010, background): “For example, recent work has looked at learning directly the dependency o f image brightness on depth from photographs registered with range data [53] or the orientation of major scene components , such as walls or ground surfaces, from a variety of image features [22], [23], [24].”
Multi-scale Improves Boundary Detection in Natural Images (2008, background): “This has led many researchers to pursue other paths, such as exploring mid-level grouping [22,32,23,33,24,25], complex example-based models [11], or scene knowledge [ 34 ].”
Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics (2010, methodology): “We use the occlusion boundary segmenter of [10],”
Layered object detection for multi-class segmentation (2010, background): “One approach of capturing rough depth ordering is to estimate occlusion boundaries, typically done in a Markov Random Field framework [26, 8, 9].”
Semantic Edge Detection with Diverse Deep Supervision (2018, methodology): “e detection (SED) [4, 14, 30, 41] is an active research topic in computer vision due to its wide-ranging applications in problems such as object proposal generation [4], occlusion and depth reasoning [1, 17], 3D reconstruction [33], object detection [11, 12], image-based localization [32] and so on.”
Self-Supervised Scene De-Occlusion (2020, background): “Other works [24, 25] rely on additional depth cues.”
Robust real-time visual odometry for dense RGB-D mapping (2013, methodology): “  Object-object relations !”