Iterative Learning with Open-set Noisy Labels

By Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, Shutao Xia. 2018

357 citations recorded by the source corpus.

Large-scale datasets possessing clean label annotations are crucial for training Convolutional Neural Networks (CNNs). However, labeling large-scale data can be very costly and error-prone, and even high-quality datasets are likely to contain noisy (incorrect) labels. Existing works usually employ a closed-set assumption, whereby the samples associated with noisy labels possess a true class contained within the set of known classes in the training data. However, such an assumption is too restrictive for many applications, since samples associated with noisy labels might in fact possess a true class that is not present in the training data. We refer to this more complex scenario as the open-set noisy label problem and show that it is nontrivial in order to make accurate predictions. To address this problem, we propose a novel iterative learning framework for training CNNs on datasets with open-set noisy labels. Our approach detects noisy labels and learns deep discriminative features in an iterative fashion. To benefit from the noisy label detection, we design a Siamese network to encourage clean labels and noisy labels to be dissimilar. A reweighting module is also applied to simultaneously emphasize the learning from clean labels and reduce the effect caused by noisy labels. Experiments on CIFAR-10, ImageNet and real-world noisy (web-search) datasets demonstrate that our proposed model can robustly train CNNs in the presence of a high proportion of open-set as well as closed-set noisy labels.

Subfields: Artificial Intelligence, Civil and Structural Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Civil and Structural Engineering

Citations in the corpus

Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “There have been approaches that aim to suppress the contribution of noisy samples by re-weighting the loss [83, 68].”
Co-teaching: Robust training of deep neural networks with extremely noisy labels (2018, background): “In addition, there are some other deep learning solutions to deal with noisy labels [24, 41].”
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels (2018, methodology): “[39], we reported the last-epoch test accuracy after training for 100 epochs.”
Unsupervised label noise modeling and loss correction (2019, background): “Recent efforts to deal with label noise address two scenarios (Wang et al., 2018b): closed-set and open-set label noise.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “Training networks with erroneous labels is further studied in [187]–[190].”
MoPro: Webly Supervised Learning with Momentum Prototypes (2020, methodology): “, 2018; 2020), iterative training (Tanaka et al., 2018; Yi & Wu, 2019; Wang et al., 2018; Chen et al., 2019), or meta-learning (Li et al.”
Early-Learning Regularization Prevents Memorization of Noisy Labels (2020, background): “We also assume that the correct classes are known (as opposed to [42]).”
Robust Inference via Generative Classifiers for Handling Noisy Labels (2019, methodology): “Our method is also evaluated on semantic noisy labels from a machine classifier and open-set noisy labels (Wang et al., 2018).”
Normalized Loss Functions for Deep Learning with Noisy Labels (2020, background): “…Han et al., 2018a); 3) reﬁned training strategies that modify the training procedure to be more adaptive to incorrect labels (Jiang et al., 2018; Wang et al., 2018; Tanaka et al., 2018; Ma et al., Proceedings of the 37 th International Conference on Machine Learning , Vienna, Austria, PMLR…”
Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise (2020, neutral): “Note that both IDN and CCN consider close-set noise, as contrast to a speciﬁc label noise termed open-set noise (Wang et al. 2018), where the noisy instances does not belong to any considered classes.”
Learning to Learn From Noisy Labeled Data (2018, methodology): “An iterative training method is proposed to identify and downweight noisy samples [30].”
Probabilistic End-To-End Noise Correction for Learning With Noisy Labels (2019, background): “, in [14, 15, 25]), and different techniques such as local inherent dimensionality have been brought into the noisy label learning domain.”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “the OD problem, the data cleaning procedure, which is the prerequisite procedure of learning from open-set noisy data [84], [85] and open set semi-supervised learning [86],”
Understanding and Improving Early Stopping for Learning with Noisy Labels (2021, neutral): “Recently, based on this phenomenon, many methods [9, 26, 27, 16, 17] have been proposed and achieved promising performance.”
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics (2022, background): “…(Arazo et al., 2019), using them only in a self-supervised training objective (Li et al., 2020a), or even completely ignoring them during training (Wang et al., 2018; Talukdar et al., 2021), upweighting or balancing training on atypical, or avoid memorization of noisy labels (Brown et al., 2020),…”
Dimensionality-Driven Learning with Noisy Labels (2018, methodology): “Another approach is to use a refined training strategy that utilizes correction methods to adjust the loss function to eliminate the influence of noisy samples (Wang et al., 2018).”
Learning Sound Event Classifiers from Web Audio with Noisy Labels (2019, background): “[19, 20] use the terms closed-set for IV, and open-set for OOV.”