CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise

By Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang. 2017

515 citations recorded by the source corpus.

In this paper, we study the problem of learning image classification models with label noise. Existing approaches depending on human supervision are generally not scalable as manually identifying correct or incorrect labels is time-consuming, whereas approaches not relying on human supervision are scalable but less effective. To reduce the amount of human supervision for label noise cleaning, we introduce CleanNet, a joint neural embedding network, which only requires a fraction of the classes being manually verified to provide the knowledge of label noise that can be transferred to other classes. We further integrate CleanNet and conventional convolutional neural network classifier into one framework for image classification learning. We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets. Experimental results show that CleanNet can reduce label noise detection error rate on held-out classes where no human supervision available by 41.5% compared to current weakly supervised methods. It also achieves 47% of the performance gain of verifying all images with only 3.2% images verified on an image classification task. Source code and dataset will be available at kuanghuei.github.io/CleanNetProject.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, methodology): “Comparing PuriDivER with other methods, in particular, accuracy increases up to 16% on CIFARs, 11% and 3% on WebVision and Food-101N, respectively.”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “There have been multiple lines of work to learn amidst noisy labels such as loss regularization [84, 96, 31], data re-weighting [68, 72], label cleaning [64, 42, 61], and”
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels (2017, methodology): “Their implementations are based on the TensorFlow slim implementation 1 , where the inception is detailed in (Zhang et al., 2017) and the wider resnet is in (Zagoruyko & Komodakis, 2016).”
Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations (2021, unknown): “44M / 100K 1000 N/A 7 7 3 Food-101N [Lee et al., 2018] 310K in all 101 Estimated 20% 7 7 3 Animal-10N [Song et al.”
Inverse Cooking: Recipe Generation From Food Images (2018, background): “As a result, there is currently a vast literature in computer vision dealing with a variety of food related tasks, with special focus in image classification [26, 39, 38, 33, 6, 24, 30, 60, 16, 17].”
NLNL: Negative Learning for Noisy Labels (2019, methodology): “In other studies, each training sample is re-weighted differently depending on the reliability of the given label [13, 27, 20].”
Stacked Cross Attention for Image-Text Matching (2018, background): “In computer vision, visual attention aims to focus on specific images or subregions [1, 40, 41, 26].”
L_DMI: An Information-theoretic Noise-robust Loss Function (2019, unknown): “Multi-task frameworks that jointly estimates true labels and learns to classify images are also introduced [19,38,41,45].”
Omni-sourced Webly-supervised Learning for Video Recognition (2020, background): “Webly-supervised learning Leveraging information from the Internet, termed webly-supervised learning , has been extensively explored [24,49,13].”
Deep Self-Learning From Noisy Labels (2019, methodology): “Among all the above work, CleanNet [15] achieved the existing state-of-the-art performance on real-world dataset such as Clothing1M [36].”
Robust Multi-View Clustering With Incomplete Information (2022, methodology): “…based methods [33], [34], which modify the network architecture to simulate the label transition matrix; iii) sample re-weighting methods [35], [36], which iteratively compute the conﬁdences of samples as clean and reweigh their importances to guide the network optimization; and iv)…”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, methodology): “If labels are obtained from a single expert, as in our experiments in Section V-A, annotations may be systematically biased due to annotation habits or subjective opinion of a single annotator, risking generalizability when compared with the “true label".”
Robust Person Re-Identification by Modelling Feature Uncertainty (2019, background): “The recently proposed CleanNet [14] learns the similarity between class- and queryembedding vectors, which is then used to detect noisy samples.”
O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks (2019, background): “In [10], Lee et al. propose a joint neural embedding network named Clean-Net.”
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, background): “A typical manner [31, 38] under this category is to train two or multiple learners simultaneously and exploit their agreement of predictions to measure the label reliability.”
Robust Learning by Self-Transition for Handling Noisy Labels (2020, methodology): “For WebVision 1.0, we trained an Inception ResNet-V2 [36] from scratch for the first 50 classes of the Google image subset [5]; for FOOD-101N, we fine-tuned a ResNet-50 with the ImageNet pretrained weights for the entire training set [17].”
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach (2019, methodology): “Motivated by learning with noisy labels [Lee et al. , 2018; Wang et al. , 2018b], we deﬁne a conﬁdence score for each of the assigned pseudo-labels.”
Symmetric Cross Entropy for Robust Learning With Noisy Labels (2019, background): “One common approach is to correct noisy labels to their true labels via a clean label inference step using complex noise models characterized by directed graphical models [26], conditional random fields [23], neural networks [11, 24] or knowledge graphs [12].”