Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer

By Xiangde Luo, Minhao Hu, Tao Song, Guotai Wang, Shaoting Zhang. International Conference on Medical Imaging with Deep Learning, 2021

332 citations recorded by the source corpus.

Recently, deep learning with Convolutional Neural Networks (CNNs) and Transformers has shown encouraging results in fully supervised medical image segmentation. However, it is still challenging for them to achieve good performance with limited annotations for training. In this work, we present a very simple yet efficient framework for semi-supervised medical image segmentation by introducing the cross teaching between CNN and Transformer. Specifically, we simplify the classical deep co-training from consistency regularization to cross teaching, where the prediction of a network is used as the pseudo label to supervise the other network directly end-to-end. Considering the difference in learning paradigm between CNN and Transformer, we introduce the Cross Teaching between CNN and Transformer rather than just using CNNs. Experiments on a public benchmark show that our method outperforms eight existing semi-supervised learning methods just with a simpler framework. Notably, this work may be the first attempt to combine CNN and transformer for semi-supervised medical image segmentation and achieve promising results on a public benchmark. The code will be released at: https://github.com/HiLab-git/SSL4MIS.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/4cc5ac4a27a4c33722173c8b42aab1009b7bb793

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Qian Wang
2. Dinggang Shen
3. Yichi Zhang
4. Zihao Yin
5. Wen Ji
6. Junfeng Zhang
7. Zhuoyuan Li
8. Rushi Jiao

In Radiology, Nuclear Medicine and Imaging

1. Dinggang Shen
2. Qian Wang
3. Islem Rekik

Citations in the corpus

• Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “Liu et al. [114] combine CNN blocks and Swin Transformer blocks as the backbone.”
• Transformers in Medical Image Analysis: A Review (2022, background): “Luo et al.[201] first combined a CNN and Transformer for semi-supervised medical image segmentation.”