Attention-Based Transformers for Instance Segmentation of Cells in Microstructures

By Tim Prangemeier, Christoph Reich, Heinz Koeppl. 2020

110 citations recorded by the source corpus.

Detecting and segmenting object instances is a common task in biomedical applications. Examples range from detecting lesions on functional magnetic resonance images, to the detection of tumours in histopathological images and extracting quantitative single-cell information from microscopy imagery, where cell segmentation is a major bottleneck. Attention-based transformers are state-of-the-art in a range of deep learning fields. They have recently been proposed for segmentation tasks where they are beginning to outperform other methods. We present a novel attention-based cell detection transformer (CellDETR) for direct end-to-end instance segmentation. While the segmentation performance is on par with a state-of-the-art instance segmentation method, Cell-DETR is simpler and faster. We showcase the method’s contribution in a the typical use case of segmenting yeast in microstructured environments, commonly employed in systems or synthetic biology. For the specific use case, the proposed method surpasses the state-of-the-art tools for semantic segmentation and additionally predicts the individual object instances. The fast and accurate instance segmentation performance increases the experimental information yield for a posteriori data processing and makes online monitoring of experiments and closed-loop optimal experimental design feasible. Code and data sample is available at https://git.rwth-aachen.de/ bcs/projects/cell-detr.git.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

1. Dacheng Tao
2. Dinggang Shen
3. Yunhong Wang
4. Qian Wang
5. Qingjie Liu

In Computer Vision and Pattern Recognition

1. Yunhe Wang
2. Kai Han
3. Jianyuan Guo
4. Chunjing Xu
5. Dacheng Tao
6. Yehui Tang
7. Zhenhua Liu
8. Xinghao Chen

In Media Technology

1. Yehui Tang
2. Dacheng Tao
3. Dinggang Shen
4. Chunjing Xu
5. Yunhong Wang
6. Yunhe Wang
7. Hanting Chen
8. Jianyuan Guo

Citations in the corpus

• A Survey on Vision Transformer (2020, methodology): “There is also a attempt to use transformer for cell instance segmentation [95], which is based on the DETR panoptic segmentation model.”
• Visual Grounding with Transformers (2021, methodology): “[27] propose an attention-based transformer to solve instance segmentation of cells in microstructures.”
• Transformers in Medical Image Analysis: A Review (2022, background): “gland segmentation [24], [84], [101], [91], nucleus segmentation [24], [84], [91], [101], cell segmentation [87], [105], [106], spleen segmentation [90], lung field/COVID-19 pneumonia”
• Unknown Object Segmentation from Stereo Images (2021, methodology): “Based on the initial DETR model [1] several works have been published for instance segmentation [31], [32].”
• A Survey of Visual Transformers (2021, methodology): “Instead of using a multistage serial training process, Cell-DETR and VisTR develop a parallel model for end-to-end instance segmentation [Fig.”