Maximin affinity learning of image segmentation

By Srinivas C. Turaga, Kevin L. Briggman, Moritz Helmstaedter, Winfried Denk, H. Sebastian Seung. 2009

150 citations recorded by the source corpus.

Images can be segmented by first using a classifier to predict an affinity graph that reflects the degree to which image pixels must be grouped together and then partitioning the graph to yield a segmentation. Machine learning has been applied to the affinity classifier to produce affinity graphs that are good in the sense of minimizing edge misclassification rates. However, this error measure is only indirectly related to the quality of segmentations produced by ultimately partitioning the affinity graph. We present the first machine learning algorithm for training a classifier to produce affinity graphs that are good in the sense of producing segmentations that directly minimize the Rand index, a well known segmentation performance measure. The Rand index measures segmentation performance by quantifying the classification of the connectivity of image pixel pairs after segmentation. By using the simple graph partitioning algorithm of finding the connected components of the thresholded affinity graph, we are able to train an affinity classifier to directly minimize the Rand index of segmentations resulting from the graph partitioning. Our learning algorithm corresponds to the learning of maximin affinities between image pixel pairs, which are predictive of the pixel-pair connectivity.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, background): “The pixel-level pairwise affinity can be either hand-constructed [7, 23, 24] or learned [16, 17, 28, 34, 35,40,50].”
Boundary Learning by Optimization with Topological Constraints (2010, methodology): “The Rand error is becoming more popular as a metric of segmentation performance [7], and can be used as a cost function for gradient learning of image segmentation [9].”
Scene parsing with Multiscale Feature Learning, Purity Trees, and Optimal Covers (2012, background): “Other directions for improvements include the use of structured learning criteria such as Turaga et al .’s Maximin Learning [25] to learn low-level feature vectors from which better segmentation trees can be produced.”
Crowdsourcing the creation of image segmentation algorithms for connectomics (2015, methodology): “Rand error has been proposed as a metric of segmentation performance (Unnikrishnan et al., 2007; Arbelaez et al., 2011), and has also been used as an objective function for directly optimizing the performance of machine learning algorithms (Turaga et al., 2009).”
A fast learning algorithm for image segmentation with max-pooling convolutional networks (2013, methodology): “MPCNN scale well to large images and excel in many object recognition [9, 8, 10, 7] and segmentation [3, 6, 5] benchmarks.”
Randomized Optimum Models for Structured Prediction (2012, background): “One possible model of these outputs is as connected components, as in (Turaga et al., 2010).”
Topology-Preserving Deep Image Segmentation (2019, methodology): “In neuron image segmentation, some methods [17, 39] directly find an optimal partition of the image into neurons, and thus avoid segmenting membranes.”
Semantic Image Segmentation with Contextual Hierarchical Models (2015, methodology): “They have also been used for biological image segmentation [42], [43], [44] and semantic segmentation [2], [36], [45], [46], [47].”
Matrix Backpropagation for Deep Networks with Structured Layers (2015, methodology): “Turaga et al [39] learn a model end-to-end based on a CNN by optimizing a standard segmentation criterion.”