Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation

By Lin F. Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, Danny Ziyi Chen. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2017

560 citations recorded by the source corpus.

Image segmentation is a fundamental problem in biomedical image analysis. Recent advances in deep learning have achieved promising results on many biomedical image segmentation benchmarks. However, due to large variations in biomedical images (different modalities, image settings, objects, noise, etc.), to utilize deep learning on a new application, it usually needs a new set of training data. This can incur a great deal of annotation effort and cost, because only biomedical experts can annotate effectively, and often there are too many instances in images (e.g., cells) to annotate. In this paper, we aim to address the following question: With limited effort (e.g., time) for annotation, what instances should be annotated in order to attain the best performance? We present a deep active learning framework that combines fully convolutional network (FCN) and active learning to significantly reduce annotation effort by making judicious suggestions on the most effective annotation areas. We utilize uncertainty and similarity information provided by FCN and formulate a generalized version of the maximum set cover problem to determine the most representative and uncertain areas for annotation. Extensive experiments using the 2015 MICCAI Gland Challenge dataset and a lymph node ultrasound image segmentation dataset show that, using annotation suggestions by our method, state-of-the-art segmentation performance can be achieved by using only 50% of training data.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/bda5728aae68fa9c05535bc6cc23167f2f8947cb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis (2019, background): “To this end, Yang et al. (2017) present Suggestive Annotation, a deep active learning framework for medical image segmentation, which uses an alternative formulation of uncertainty sampling combined with a form of representativeness density weighting.”
A Continual Learning Framework for Uncertainty-Aware Interactive Image Segmentation (2021, background): “One notable work leverages uncertainty to actively query labels on an entire image (Yang et al. 2017), but strictly speaking, it is not an interactive segmentation model as it does not refine segmentation results.”
MILD‐Net: Minimal information loss dilated network for gland instance segmentation in colon histology images (2018, methodology): “For example, a deep active learning framework was presented in Yang et al. (2017) for gland segmentation using suggestive annotation.”
The Power of Ensembles for Active Learning in Image Classification (2018, methodology): “For comparison, this study implements the approaches presented in [52] (in the following called Core-Set) and [61] (in the following called REPR).”
A Survey of Deep Active Learning (2020, methodology): “tion. For example, [48] proposes an AL-based transfer learning mechanism for medical image segmentation, which can effectively improve the image segmentation performance on a limited labeled dataset. [176] combines fully convolutional networks (FCN) and AL to create a DAL framework for biological im”
Extending pretrained segmentation networks with additional anatomical structures (2018, background): “In order to account for both of these conditions (i) and (ii), we propose to first pick a large batch of samples for which the trained model is “confident” [9] and then to prune them with the aim of maximizing how well they represent the entire dataset [4].”
Deep learning in medical imaging and radiation therapy. (2018, background): “322 Yang et al. 240 created a synthetic CT dataset through the use of the Radon transform for a known object and modeled different exposure conditions through adding noise to the data, for training a CNN to estimate high-dose projections from low-dose ones.”
New machine learning method for image-based diagnosis of COVID-19 (2020, background): “ML has demonstrated high performance for several image processing applications such as image analysis [5, 6], image classification [7], and image segmentation [8].”
Learn the new, keep the old: Extending pretrained models with new anatomy and images (2018, methodology): “Similarly to [6], we use spatially averaged activations at the abstraction-layer (c.f. Fig.1) of our network to represent a given image as a vector I R abs .”
Learning Loss for Active Learning (2019, background): “Thus, a variety of hybrid strategies have been proposed [29, 59, 41, 56] for their specific tasks.”
Variational Adversarial Active Learning (2019, methodology): “While we show that our model outperforms both [30] and [55], we argue that VAAL achieves this by learning the representation and uncertainty together such that they act in favor of each other resulting in better active learning performance.”