Image Co-segmentation via Consistent Functional Maps

By Fan Wang, Qi-Xing Huang, Leonidas J. Guibas. 2013

144 citations recorded by the source corpus.

Joint segmentation of image sets has great importance for object recognition, image classification, and image retrieval. In this paper, we aim to jointly segment a set of images starting from a small number of labeled images or none at all. To allow the images to share segmentation information with each other, we build a network that contains segmented as well as unsegmented images, and extract functional maps between connected image pairs based on image appearance features. These functional maps act as general property transporters between the images and, in particular, are used to transfer segmentations. We define and operate in a reduced functional space optimized so that the functional maps approximately satisfy cycle-consistency under composition in the network. A joint optimization framework is proposed to simultaneously generate all segmentation functions over the images so that they both align with local segmentation cues in each particular image, and agree with each other under network transportation. This formulation allows us to extract segmentations even with no training data, but can also exploit such data when available. The collective effect of the joint processing using functional maps leads to accurate information sharing among images and yields superior segmentation results, as shown on the iCoseg, MSRC, and PASCAL data sets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks (2017, methodology): “More recently, higher-order cycle consistency has been used in structure from motion [56], 3D shape matching [19], co-segmentation [51], dense semantic alignment [59, 60], and depth estimation [12].”
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation (2016, methodology): “Inspired by the recent work on synchronization [23, 25, 26, 9], we propose to align these different spectral domains using functional map [21].”
Unsupervised Object Discovery and Tracking in Video Collections (2015, background): “Related approaches have been proposed for salient region detection [18], image cosegmentation [36, 37], and image colo-calization [4].”
Temporal Cycle-Consistency Learning (2019, methodology): “It has been applied successfully for tasks like co-segmentation [43, 44], structure from motion [49, 51], and image matching [54, 55, 56].”
Unsupervised Multi-class Joint Image Segmentation (2014, methodology): “The mapped results are shown in (c) when the maps are optimized independently without any consistency enforced; (d) when the maps are optimized with global consistency as in [20]; (e)when the consistency term Eq.”
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences (2015, background): “Graph consistency: The idea of utilizing consistency constraints within a global graph structure has been applied to various vision and graphics problems, including cosegmentation [40, 41], structure from motion [47, 42], and shape matching [17].”
Real-Time Monocular Depth Estimation Using Synthetic Data with Domain Adaptation via Image Style Transfer (2018, methodology): “This approach uses adversarial training [27] and cycle-consistency [26, 79, 89, 86] to translate between two sets of unaligned images from different domains.”
Unsupervised Visual Representation Learning by Graph-Based Consistent Constraints (2016, methodology): “Cycle consistency in a graph has been applied to various computer vision and graphics problems, including co-segmentation [28,29], structure from motion [30,31] and image matching [32,33].”