DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks

By Martin Rajchl, M. J. Lee, Ozan Oktay, Konstantinos Kamnitsas, Jonathan Passerat-Palmbach, Wenjia Bai, Bernhard Kainz, Daniel Rueckert. IEEE Transactions on Medical Imaging, 2016

410 citations recorded by the source corpus.

In this paper, we propose <italic>DeepCut</italic>, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known <italic>GrabCut</italic> <xref ref-type="bibr" rid="ref1">[1]</xref> method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the <italic>DeepCut</italic> method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6e84d83faca994438e5628525123f3466ee9c55d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++ (2018, background): “[25] combined GrabCut with CNNs to annotate structures in medical imagery.”
A survey on deep learning in medical image analysis (2017, methodology): “Rajchl et al. (2017) Segmentation given bounding box MRI CNN CNN and CRF for segmentation of structures”
Learning to Segment via Cut-and-Paste (2018, methodology): “Deepcut [11] propose a modification to Grabcut using a CNN+CRF model.”
A Continual Learning Framework for Uncertainty-Aware Interactive Image Segmentation (2021, neutral): “Interactive Segmentation Interactive segmentation allow user to make edits by clicks (Xu et al. 2016), scrib-bles (Grady et al. 2005), bounding boxes (Rajchl et al. 2016; Castrejon et al. 2017), or extreme points (Maninis et al. 2018; Khan et al. 2019).”
A review on automatic fetal and neonatal brain MRI segmentation (2017, neutral): “, & Rueckert, D. (2017). A review on automatic fetal and neonatal brain MRI segmentation.”
BoxInst: High-Performance Instance Segmentation with Box Annotations (2020, background): “A few works [2, 7, 13, 17, 19, 22, 24, 26] attempted to obtain (semantic or instance-level) mask prediction with boxlevel annotations.”
Adversarial Deep Structural Networks for Mammographic Mass Segmentation (2016, methodology): “The DeepCut network iteratively updated weak annotations and learned the convolutional neural network followed by CRF to do brain and lung segmentation in MR images [26].”
Guide Me: Interacting with Deep Networks (2018, methodology): “Nonetheless, some methods towards interactive deep learning have been proposed, such as weakly-supervised semantic segmentation from scribbles [38], user-provided clicks and Euclidean distance maps [61] or bounding boxes used as region initialization [17, 53].”
Pose2Seg: Detection Free Human Instance Segmentation (2018, methodology): “[24] propose a complex framework of partitioning and labeling body-parts generated using a CNN.”
On Regularized Losses for Weakly-supervised CNN Segmentation (2018, methodology): “These schemes typically iterate two steps: CNN training and proposal generation via regularization-based shallow interactive segmentation, e.g. graph cut [17] or dense CRF mean-ﬁeld inference [15,14].”
Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning (2017, methodology): “(1) can be decomposed into steps that alternatively update the segmentation label Ŷ and network parameters θ [3], [20].”