Joint Optimization Framework for Learning with Noisy Labels

By Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, Kiyoharu Aizawa. 2018

806 citations recorded by the source corpus.

Deep neural networks (DNNs) trained on large-scale datasets have exhibited significant performance in image classification. Many large-scale datasets are collected from websites, however they tend to contain inaccurate labels that are termed as noisy labels. Training on such noisy labeled datasets causes performance degradation because DNNs easily overfit to noisy labels. To overcome this problem, we propose a joint optimization framework of learning DNN parameters and estimating true labels. Our framework can correct labels during training by alternating update of network parameters and labels. We conduct experiments on the noisy CIFAR-10 datasets and the Clothing1M dataset. The results indicate that our approach significantly outperforms other state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Co-teaching: Robust training of deep neural networks with extremely noisy labels (2018, background): “[38] presented a joint optimization framework to learn parameters and estimate true labels simultaneously.”
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels (2018, background): “Alternatively, one could gradually replace noisy labels by neural network predictions [32, 35].”
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning (2020, methodology): “E.g., relabeling the data during training (Reed et al. 2015), jointly optimizing the model’s parameters and estimating true labels (Tanaka et al. 2018), using noise-tolerant loss function (Ghosh, Kumar, and Sastry 2017; Van Rooyen, Menon, and Williamson 2015) or noise tolerant training algorithms…”
Unsupervised label noise modeling and loss correction (2019, methodology): “We follow (Zhang et al., 2017; 2018; Tanaka et al., 2018) criterion for label noise addition, which consists of randomly selecting labels for a percentage of the training data using all possible labels (i.”
How does Disagreement Help Generalization against Label Corruption? (2019, background): “[33] presented a joint optimization framework to learn parameters and estimate true labels simultaneously.”
Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey (2019, methodology): “and propagating noisy labels to cleaner labels is presented in [58].”
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks (2019, methodology): “d ramp loss [56]. Several other approaches include bootstrap that leverages predicted labels to generate training targets [57] and alternatively updating network parameters and labels during training [58]. Our EmotionDL is most related to [59], which applies distribution learning to learn labels wit”
Understanding and Improving Early Stopping for Learning with Noisy Labels (2021, methodology): “Current methods [27, 25] usually adopt an early stopping strategy, which decides the stopping point by considering the network as a whole.”
Generating Accurate Pseudo-Labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations (2019, background): “We also investigate how Hermite activations behave in presence of a standard robust learning method with noisy labels, specifically [26].”
DualGraph: A graph-based method for reasoning about label noise (2021, methodology): “These datasets are popularly used for the evaluation of learning with noisy labels in previous literatures [7, 11, 29, 36, 35].”
Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (2019, methodology): “Following the previous works [65, 66], we used ResNet-50 pre-trained on ImageNet.”
Are Anchor Points Really Indispensable in Label-Noise Learning? (2019, background): “For example, many state-of-the-art approaches in this category are speciﬁcally designed to, e.g., select reliable examples [45, 14, 24], reweight examples [33, 15], correct labels [23, 17, 37, 32], employ side information [39, 21], and (implicitly) add regularization [13, 12, 43, 39, 21].”
Learning to Learn From Noisy Labeled Data (2018, methodology): “(4) Joint Optimization [23]: alternatively updates network parameters and corrects labels during training.”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, background): “One study proposed to learn the network parameters by optimizing the joint likelihood of the network parameters and true labels [156].”
SELF: Learning to Filter Noisy Labels with Self-Ensembling (2019, methodology): “, 2017), JointOpt (Tanaka et al., 2018), the source codes are available and hence used for evaluation.”
NLNL: Negative Learning for Noisy Labels (2019, methodology): “Here, we used the soft label as the updated label, similar to [30].”
Unsupervised Person Re-Identification via Multi-Label Classification (2020, methodology): “Inspired by (Arazo et al., 2019; Tanaka et al., 2018),we aim to push the probability on each positive label to 1/|P∗ i |, where |P∗ i | is the number of positive classes of xi .”