Context as Supervisory Signal: Discovering Objects with Predictable Context

By Carl Doersch, Abhinav Kumar Gupta, Alexei A. Efros. 2014

44 citations recorded by the source corpus.

This paper addresses the well-established problem of unsupervised object discovery with a novel method inspired by weakly-supervised approaches. In particular, the ability of an object patch to predict the rest of the object (its context) is used as supervisory signal to help discover visually consistent object clusters. The main contributions of this work are: 1) framing unsupervised clustering as a leave-one-out context prediction task; 2) evaluating the quality of context prediction by statistical hypothesis testing between thing and stuff appearance models; and 3) an iterative region prediction and context alignment approach that gradually discovers a visual object cluster together with a segmentation mask and fine-grained correspondences. The proposed method outperforms previous unsupervised as well as weakly-supervised object discovery approaches, and is shown to provide correspondences detailed enough to transfer keypoint annotations.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Context Encoders: Feature Learning by Inpainting (2016, methodology): “Visual Memex [29] used context to nonparametrically model object relations and to predict masked objects in scenes, while [6] used context to establish correspondences for unsupervised object discovery.”
Unsupervised Visual Representation Learning by Context Prediction (2015, methodology): “In principle, similar reasoning could be applied in the image domain, a kind of visual “fill in the blank” task, but, again, one runs into the problem of determining whether the predictions themselves are correct [12], unless one cares about predicting only very low-level features [14, 30, 50].”
Learning to See by Moving (2015, background): “Although motor theories of perception have a long history [7], there has been little work in formulating computational models of perception that make use of motor information.”
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation (2015, background): “Exploiting contextual information has been widely studied in the literature (e.g., [39, 21, 7]).”
Unsupervised Learning of Visual Representations Using Videos (2015, neutral): “org/patients-and-public/good-vision-throughoutlife/childrens-vision/infant-vision-birth-to-24-months-of-age selves [41, 6, 7].”
Generative Image Modeling Using Style and Structure Adversarial Networks (2016, background): “There are two common approaches for unsupervised learning: (a) using a discriminative framework with auxiliary tasks where supervision comes for free, such as context prediction [1,2] or temporal embedding [3,4,5,6,7,8]; (b) using a generative framework where the underlying model is compositional an”
Transitive Invariance for Self-Supervised Visual Representation Learning (2017, background): “Our work is also closely related to mid-level patch clustering [46, 7, 8] and unsupervised discovery of semantic classes [43, 47] as we attempt to find reliable clusters in our affinity graph.”
Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals (2015, background): “Recent work on discriminative patch discovery [15, 44, 50] learns mid-level visual representations in a weakly-supervised mode, and use them for object recognition [15, 44] and discovery [13, 50].”
Dataset fingerprints: Exploring image collections through data mining (2015, methodology): “Discussion In all of our experiments, the input of our system is a set of randomly sampled patches without removing highly overlapping patches or the other duplicate-removal heuristics that are applied in [4, 19].”
DOCK: Detecting Objects by Transferring Common-Sense Knowledge (2018, methodology): “Context has been used for various vision tasks including object detection [31,9,8,33,4,15], semantic segmentation [28], and object discovery [20,10].”
ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization (2016, unknown): “[30] align contextual regions of an object patch to gradually discovers a visual object cluster in their method of iterative region prediction and context alignment.”
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences (2015, background): “But going from coarse to fine-grained pixel-wise correspondence is still very much an open problem, the solution to which could benefit many vision and graphics tasks, including image edit propagation [14, 46], co-segmentation [34, 7], structurefrom-motion [36], 3D object reconstruction [4, 39] unsu”
Joint Object and Part Segmentation Using Deep Learned Potentials (2015, methodology): “In our case, we solve such problem through modelling over object scale context, which is also suggested in prior arts [5, 9, 37, 10].”