Towards Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models

By Congcong Li, Adarsh Kowdle, Ashutosh Saxena, Tsuhan Chen. 2010

35 citations recorded by the source corpus.

In many machine learning domains (such as scene understanding), several related sub-tasks (such as scene categorization, depth estimation, object detection) operate on the same raw data and provide correlated outputs. Each of these tasks is often notoriously hard, and state-of-the-art classifiers already exist for many sub-tasks. It is desirable to have an algorithm that can capture such correlation without requiring to make any changes to the inner workings of any classifier. We propose Feedback Enabled Cascaded Classification Models (FE-CCM), that maximizes the joint likelihood of the sub-tasks, while requiring only a 'black-box' interface to the original classifier for each sub-task. We use a two-layer cascade of classifiers, which are repeated instantiations of the original ones, with the output of the first layer fed into the second layer as input. Our training method involves a feedback step that allows later classifiers to provide earlier classifiers information about what error modes to focus on. We show that our method significantly improves performance in all the sub-tasks in two different domains: (i) scene understanding, where we consider depth estimation, scene categorization, event categorization, object detection, geometric labeling and saliency detection, and (ii) robotic grasping, where we consider grasp point detection and object classification.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning human activities and object affordances from RGB-D videos (2012, methodology): “At any given time, the affordance of the object depends on the sub-activity it is involved in.”
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling (2014, methodology): “Better depth estimates have been achieved by incorporating semantic labels [10], or more sophisticated models [11].”
Hallucinated Humans as the Hidden Context for Labeling 3D Scenes (2013, background): “, [26, 27, 11, 21, 12, 22]), where an approximate 3D geometry is inferred from 2D images.”
Semantic Labeling of 3D Point Clouds for Indoor Scenes (2011, unknown): “In some recent works, 3D layout or depths have been used for improving object detection (e.g., [11, 12, 14, 20, 21, 22, 27, 28]).”
Learning Object Arrangements in 3D Scenes using Human Context (2012, background): “Some works extract 3D scene geometry from a single image for object detection (e.g., Saxena et al., 2005; Hoiem et al., 2006; Heitz et al., 2008; Lee et al., 2010; Li et al., 2010).”
Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes (2016, background): “Recent works have looked at both joint [10, 11, 16, 21, 29, 36, 42, 43, 53, 59, 63, 65] and sequential/cascaded [12, 25, 26, 28, 38, 41] reasoning over different perception modules (e.”
Contextually guided semantic labeling and search for three-dimensional point clouds (2013, methodology): “In the past, a significant amount of work has been done in semantic labeling of 2D images (Murphy et al., 2003; Felzenszwalb et al., 2008; Heitz and Koller, 2008; Collet et al., 2011; Li et al., 2012).”
Contextually Guided Semantic Labeling and Search for 3D Point Clouds (2011, methodology): “In the past, a signiﬁcant amount of work has been done in semantic labeling of 2D images (Murphy et al., 2003; Heitz and Koller, 2008; Felzenszwalb et al., 2008; Collet et al., 2011; Li et al., 2012).”
Overview of Environment Perception for Intelligent Vehicles (2017, methodology): “In [175], a feedback enabled cascaded classification model was developed to jointly optimize several subtasks in scene understanding.”