Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

By Srinivas C. Turaga, Joseph F. Murray, Viren Jain, Fabian Roth, Moritz Helmstaedter, Kevin L. Briggman, Winfried Denk, H. Sebastian Seung. 2010

408 citations recorded by the source corpus.

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

Subfields: Biophysics, Structural Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Structural Biology

Citations in the corpus

C3D: Generic Features for Video Analysis (2014, background): “3D ConvNets were proposed for human action recognition [20, 46] and for medical image segmentation [19, 48].”
Learning Human Pose Estimation Features with Convolutional Networks (2013, background): “Convnets that produce segmentation maps, for example, avoid pooling completely [47, 13].”
Machines that learn to segment images: a crucial technology for connectomics. (2010, methodology): “Such a NN graph has been used for EM images (Turaga et al., 2010) as well as natural images (Fowlkes et al.”
Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks (2016, background): “The main obstacles lie in the expensive computational cost, memory requirement and time consumption [30], [31].”
Network anatomy and in vivo physiology of visual cortical neurons (2011, background): “Until it is possible to fully reconstruct large EM volumes (Helmstaedter et al., 2008; Mishchenko et al., 2010; Turaga et al., 2010), analysis of network connectivity will be limited to a partial sampling of the underlying anatomy.”
Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation (2015, background): “Extensions of ConvNets to 3D have been proposed, but computational cost and memory consumption can be still too high to efficiently implement them on current computer graphics hardware units [23].”
Cellular-resolution connectomics: challenges of dense neural circuit reconstruction (2013, background): “Improvements in algorithms have saturated at the level of voxel classification (‘voxel classifiers’ 27,28,32,37–39 ), and the current focus is on segmentation-based classifiers (‘super-voxel classifiers’ 31,34,40 ).”
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives (2012, background): “A recent success of a supervised convolutional network, applied to image segmentation, is Turaga et al. (2010).”
Boundary Learning by Optimization with Topological Constraints (2010, background): “Finally, it should also be possible to extend BLOTC to boundary detectors that classify the edges of an affinity graph rather than pixels [28, 29, 9].”
A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations (2014, methodology): “Extensions of CNNs to 3D have been proposed, but computational cost and memory consumption are still too high to be efficiently implemented on today’s computer graphics hardware units [11,13].”
Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring (2016, neutral): “, object recognition, scene parsing [39], cell segmentation [40], neural circuit segmentation [41], [42], analysis of images the breast [43]–[46].”
Deep Convolutional Neural Networks for Multi-Modality Isointense Infant Brain Image Segmentation (2015, methodology): “In Jain et al. (2007), Jain and Seung (2009), Turaga et al. (2010), and Helmstaedter et al. (2013), CNNs were applied to restore and segment the volumetric electron microscopy images.”
3D Convolutional Neural Networks for Human Action Recognition (2010, neutral): “…competitive performance in visual object recognition [17], [19], [22]–[24], human action recognition [25]–[27], natural language processing [28], audio classification [29], brain-computer interaction [30], human tracking [31], image restoration [32], denoising [33], and segmentation tasks [34].”
High-precision automated reconstruction of neurons with flood-filling networks (2017, methodology): “The first (‘baseline’) approach combined a 3D convolutional neural network trained to produce long-range affinity graphs 16 , affinity-graph watershed segmentation 33 , and random forest object agglomeration by GALA 21,28 .”
Machine Learning for Medical Ultrasound: Status, Methods, and Future Opportunities (2018, background): “such as object detection [52–54], face detection [55–57], and segmentation [58, 59].”