U-Net Transformer: Self and Cross Attention for Medical Image Segmentation

By Olivier Petit, Nicolas Thomé, Clément Rambour, Luc Soler. MLMI@MICCAI, 2021

333 citations recorded by the source corpus.

Medical image segmentation remains particularly challenging for complex and low-contrast anatomical structures. In this paper, we introduce the U-Transformer network, which combines a U-shaped architecture for image segmentation with self- and cross-attention from Transformers. U-Transformer overcomes the inability of U-Nets to model long-range contextual interactions and spatial dependencies, which are arguably crucial for accurate segmentation in challenging contexts. To this end, attention mechanisms are incorporated at two main levels: a self-attention module leverages global interactions between encoder features, while cross-attention in the skip connections allows a fine spatial recovery in the U-Net decoder by filtering out non-semantic features. Experiments on two abdominal CT-image datasets show the large performance gain brought out by U-Transformer compared to U-Net and local Attention U-Nets. We also highlight the importance of using both self- and cross-attention, and the nice interpretability features brought out by U-Transformer.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/baa071671b6b8f570b8fe0031165df29a5f9cc28

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xing Wu
2. Feng Yang
3. Luc Van Gool
4. Peyman Milanfar
5. A. Bovik
6. Hossein Talebi
7. Yinxiao Li
8. Han Zhang

In Biomedical Engineering

1. Luc Van Gool
2. Peyman Milanfar
3. Han Zhang
4. Yulun Zhang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Lei Zhang
3. Ming-Hsuan Yang
4. Radu Timofte
5. Dacheng Tao
6. Chen Change Loy
7. Yulun Zhang
8. Peyman Milanfar

Citations in the corpus

• DiffIR: Efficient Diffusion Model for Image Restoration (2023, methodology): “Presently, the transformer has been adopted in numerous vision tasks, such as image recognition [17, 60], segmentation [62, 69, 87, 49], object detection [5, 89], and image restoration [7, 38, 74, 36, 8].”
• MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “Inspired by the gated filtering of skip connections [65, 69], we extend the gated MLP (gMLP) to build a cross gating block (CGB, Fig.”