U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

By Xuebin Qin, Chenyang Huang, Masood Dehghan, Osmar R Zaiane, Martin Jägersand. Pattern Recognition, 2020

2,199 citations recorded by the source corpus.

Abstract In this paper, we design a simple yet powerful deep network architecture, U2-Net, for salient object detection (SOD). The architecture of our U2-Net is a two-level nested U-structure. The design has the following advantages: (1) it is able to capture more contextual information from different scales thanks to the mixture of receptive fields of different sizes in our proposed ReSidual U-blocks (RSU), (2) it increases the depth of the whole architecture without significantly increasing the computational cost because of the pooling operations used in these RSU blocks. This architecture enables us to train a deep network from scratch without using backbones from image classification tasks. We instantiate two models of the proposed architecture, U2-Net (176.3 MB, 30 FPS on GTX 1080Ti GPU) and U2-Net† (4.7 MB, 40 FPS), to facilitate the usage in different environments. Both models achieve competitive performance on six SOD datasets. The code is available: https://github.com/NathanUA/U-2-Net .

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3f5539ec77ae04623ddadf95eec32ad3f17d1487

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Kubric: A scalable dataset generator (2022, methodology): “Classical methods involve active-contour [47, 69] or graphcut [20, 104] techniques, but there also exist techniques with human-in-the-loop [72, 81, 100], and more recently deep learning variants [30, 73, 74, 105, 117].”
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything (2023, methodology): “Specifically, we take a state-of-the-art saliency object detection model, U 2 -net[45], to predict saliency map and uniformly sample 3 random points ( saliency map to perform instance segmentation with our EfficientSAM.”
Counterfactual Generative Networks (2021, methodology): “Therefore, we add a pre-trained U2-Net (Qin et al., 2020) as a head on top of the BigGAN backbone.”
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation (2023, methodology): “We also develop a two-stage matting pipeline based on the U(2)Net [47] and FBA [17] matting model to generate the foreground masks for all the frames.”
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (2023, methodology): “We preprocess the input image by background removal (Qin et al., 2020) and recentering of the foreground object.”
Salient Object Detection via Integrity Learning (2021, background): “As SOD is widely used in the field of computer vision, it plays a vital role in many downstream tasks, such as object detection [4], image retrieval [5], co-salient object detection [6], multi-modal matching [7], VR/AR applications [8] and semantic segmentation [9], [10], [11].”
MA-Net: A Multi-Scale Attention Network for Liver and Tumor Segmentation (2020, methodology): “Except for the comparison with U-Net, we run experiments with methods of SegNet [39], Res-Unet [40], U-Net ++ [7] and U2Net [41], respectively.”
TEXTure: Text-Guided Texturing of 3D Shapes (2023, methodology): “For this task, we segment the prominent object from the image using a pretrained saliency network [34], apply standard scale and crop augmentations, and paste the result onto a randomly-colored background.”
FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network (2022, methodology): “Inspired by U 2 -Net [15], this paper proposes a similar topology named U 2 -LSTM as shown in Fig.3”
TransAttUnet: Multi-Level Attention-Guided U-Net With Transformer for Medical Image Segmentation (2021, methodology): “Notably, many advanced works [11], [14], [25] have demonstrated the effectiveness of multi-scale feature fusion in encoding global and local contexts.”
HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation (2020, methodology): “Network h uses the nested U-Net structure introduced by Xuebin et al. [34].”
AnyDoor: Zero-shot Object-level Image Customization (2023, neutral): “The segmentor model could be either automatic [26, 38] or interactive [11, 12, 32].”
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation (2024, methodology): “Since the images generated by MVDream may contain various backgrounds, we apply background removal [37] and use white background.”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, methodology): “Meanwhile, the structural development based on residual connections is also a promising direction [25, 66-69].”
UIU-Net: U-Net in U-Net for Infrared Small Object Detection (2022, methodology): “We define U = [u1, u2, · · · , uC ] as the encoder high-level features after ReSidual U-blocks [44] of deep network layer, where uc = Fh c ∈ RW×H is the c-th channel feature, and C is the total number of channels.”