Using Multiple Segmentations to Discover Objects and their Extent in Image Collections

By Bryan C. Russell, William T. Freeman, Alexei A. Efros, Josef Sivic, Andrew Zisserman. 2006

745 citations recorded by the source corpus.

Given a large dataset of images, we seek to automatically determine the visually similar object and scene classes together with their image segmentation. To achieve this we combine two ideas: (i) that a set of segmented objects can be partitioned into visual object classes using topic discovery models from statistical text analysis; and (ii) that visual object classes can be used to assess the accuracy of a segmentation. To tie these ideas together we compute multiple segmentations of each image and then: (i) learn the object classes; and (ii) choose the correct segmentations. We demonstrate that such an algorithm succeeds in automatically discovering many familiar objects in a variety of image datasets, including those from Caltech, MSRC and LabelMe.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Visual Representation Learning by Context Prediction (2015, background): “Visual data mining [41, 13, 47, 42], or unsupervised object discovery [48, 44, 20], aims to use a large image collection to discover image fragments which happen to depict the same semantic objects.”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, background): “One form of discriminative learning combines clustering with hand-crafted features to learn visual representations such as image-patches [13, 62], object discovery [57, 63].”
Learning to See by Moving (2015, background): “The existence of close links between perceptual and motor systems in biological agents is supported by the discovery of neurons in mammalian brain that show increased neural activity not only when the animal itself performs a certain action, but also when the animal perceives another animal performi”
LabelMe: A Database and Web-Based Tool for Image Annotation (2008, background): “Even algorithms requiring no supervision (Sivic et al. 2005; Russell et al. 2006; Fei-Fei et al. 2003; Sudderth et al. 2005b, 2005a; Quelhas et al. 2005) need this quantitative framework.”
Unsupervised Learning of Visual Representations Using Videos (2015, methodology): “The second class of algorithms use manually defined features such as SIFT or HOG and perform clustering over training data to discover semantic classes [42, 38].”
Recognizing indoor scenes (2009, methodology): “Each ROI is decomposed into a 2x2 grid and histograms of visual words are computed for each window [7, 1, 12].”
Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks (2019, methodology): “Related work from computer vision includes (Cao and Fei-Fei 2007; Kim and Torralba 2009; Lee and Grauman 2009; Russell et al. 2006; Karpathy, Miller, and Fei-Fei 2013), as well as work on co-segmentation (Zhu et al. 2016) and unsupervised object recognition (Tuytelaars et al. 2010).”
Unsupervised Discovery of Mid-Level Discriminative Patches (2012, methodology): “al [24] baseline, plus a number of intermediate results of our method: 1) HOG K-Means (visual word analog for HOG features), 2) Initial Clustering (SVMs trained on the K-Means clusters without discriminative re-clustering), and 3) No Cross-Validation (iterative, discriminatively-trained clusters but”
Unsupervised Representation Learning by Sorting Sequences (2017, methodology): “Before the resurgence of CNNs, hand-craft features such as SIFT and HOG have been used to discover semantic classes using clustering [33, 36], or mining discriminative mid-level features [35, 6, 39].”
Transitive Invariance for Self-Supervised Visual Representation Learning (2017, background): “Our work is also closely related to mid-level patch clustering [46, 7, 8] and unsupervised discovery of semantic classes [43, 47] as we attempt to find reliable clusters in our affinity graph.”
Visual Relationship Detection with Language Priors (2016, neutral): “Relationships have also improved object localization [32,33,6,34].”
Context as Supervisory Signal: Discovering Objects with Predictable Context (2014, background): “Proponents of unsupervised representation learning [1,2,3,4] and unsupervised object discovery [5,6,7,8,9,10,11,12] have long argued that these approaches have the potential to solve two fundamental problems with supervised methods.”
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input (2018, neutral): “e of the ﬁrst works within this realm is [26], which utilized an iterative clustering and classiﬁcation algorithm to discover object categories. Further works borrowed ideas from textual topic models [27], assuming that Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input 3 cer”
Unsupervised Object Discovery and Tracking in Video Collections (2015, background): “…process a discovery and tracking procedure since ﬁnding the tubes maximizing foreground conﬁdence across videos is akin to unsupervised object discovery [4, 11, 12, 28, 31], whereas ﬁnding the tubes maximizing temporal consistency within a video is similar to object tracking [1, 2, 13, 26, 40, 41].”