End-to-End Semi-Supervised Object Detection with Soft Teacher

By Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, Zicheng Liu. 2021

662 citations recorded by the source corpus.

This paper presents an end-to-end semi-supervised object detection approach, in contrast to previous more complex multi-stage methods. The end-to-end training gradually improves pseudo label qualities during the curriculum, and the more and more accurate pseudo labels in turn benefit object detection training. We also propose two simple yet effective techniques within this framework: a soft teacher mechanism where the classification loss of each unlabeled bounding box is weighed by the classification score produced by the teacher network; a box jittering approach to select reliable pseudo boxes for the learning of box regression. On the COCO benchmark, the proposed approach outperforms previous methods by a large margin under various labeling ratios, i.e. 1%, 5% and 10%. Moreover, our approach proves to perform also well when the amount of labeled data is relatively large. For example, it can improve a 40.9 mAP baseline detector trained using the full COCO training set by +3.6 mAP, reaching 44.5 mAP, by leveraging the 123K unlabeled images of COCO. On the state-of-the-art Swin Transformer based object detector (58.9 mAP on test-dev), it can still significantly improve the detection accuracy by +1.5 mAP, reaching 60.4 mAP, and improve the instance segmentation accuracy by +1.2 mAP, reaching 52.4 mAP. Further incorporating with the Object365 pre-trained model, the detection accuracy reaches 61.3 mAP and the instance segmentation accuracy reaches 53.0 mAP, pushing the new state-of-the-art. The code and models will be made publicly available at https://github.com/microsoft/SoftTeacher.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

UFO: A UniFied TransfOrmer for Vision-Language Representation Learning (2021, background): “Motivated by [16, 28, 50], we add a momentum teacher to guide the pre-training.”
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, neutral): “6 SoftTeacher [52] Swin-L O365,SS-COCO 60.”
RegionCLIP: Region-based Language-Image Pretraining (2021, background): “Recently, semi-supervised learning was explored [48, 56, 66], where pretrained detectors are used to create pseudo labels of image regions.”
Grounded Language-Image Pre-training (2021, methodology): “We list the performance of MDETR, the SoTA grounding model.”
ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, methodology): “3 [59] on MS-COCO [35] in 2021, and knowing that the task can be solved by humans with a little training, we believe that the computer vision community will eventually come up with efﬁcient methods for this challenging task.”
Swin Transformer V2: Scaling Up Capacity and Resolution (2021, positive): “…segmentation, and 86.8% top-1 accuracy on Kinetics-400 video action classiﬁcation, which are +NA%, +4.4/+3.3, +6.3 and +1.9 higher than the best numbers in the original Swin Transformers [46,47], and surpass previous best records by +0.8% ( [80]), +1.8/+1.4 ( [74]), +1.5 ( [4]) and +1.4% ( [57]).”
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model (2022, background): “To alleviate the reliability of DNNs on big data and elaborate annotations, different paradigms such as semisupervised learning [34], few-shot learning [25, 27, 36], zero-shot learning [21,30,33], self-supervised learning [31], open-set learning [6, 22, 29] and advanced training strategies [17, 35] ”
Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation (2022, methodology): “Apart from semi-supervised classification, the methodology in FixMatch has swept across a wide range of research topics and achieved booming success, such as semantic segmentation [19, 28, 81], object detection [42, 57, 66], unsupervised domain adaptation [45], and action recognition [62,67].”
Focal Modulation Networks (2022, neutral): “9 Soft-Teacher (Swin-L) [93] 284M IN-22K (14M) COCO-unlabeled + O365 3 60.”
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment (2023, background): “Semi-supervised Object Detection (SSOD) methods [45, 46, 51, 63] aim to improve object detection systems by exploiting unlabeled data on the basis of some available labeled data.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “Consistency regularization (Section II-B1) is introduced for semi-supervised object detection to propagate the soft label and bounding boxes assignment on unlabeled images based on dual consistency constraints on clas-siﬁcation and regression [29], [169], [196], [197], [198], [199].”
Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling (2021, methodology): “We observe that conventional pseudo-labeling methods Soft-Teacher, Unbiased Teacher have no signiﬁcant improvements over caption-pretraining baselines since they cannot utilize textual modality in captioned images to spot novel classes correctly.”
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022, neutral): “Soft Teacher+SwinL [38] 284M IN-22K-14M O365 ✓ 60.”
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, background): “This belongs to the research area of semi-supervised learning [31, 55, 89], which is popular with various applications [70, 74].”
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale (2022, neutral): “Soft-Teacher [115] HTC++ [17] 284M IN-21K (14M) COCO(unlabeled)+O365 60.”