Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization

By Hongxin Wei, Lei Feng, Xiangyu Chen, Bo An. 2020

687 citations recorded by the source corpus.

Deep Learning with noisy labels is a practically challenging problem in weakly-supervised learning. The state-of-the-art approaches "Decoupling" and "Co-teaching+" claim that the "disagreement" strategy is crucial for alleviating the problem of learning with noisy labels. In this paper, we start from a different perspective and propose a robust learning paradigm called JoCoR, which aims to reduce the diversity of two networks during training. Specifically, we first use two networks to make predictions on the same mini-batch data and calculate a joint loss with Co-Regularization for each training example. Then we select small-loss examples to update the parameters of both two networks simultaneously. Trained by the joint loss, these two networks would be more and more similar due to the effect of Co-Regularization. Extensive experimental results on corrupted data from benchmark datasets including MNIST, CIFAR-10, CIFAR-100 and Clothing1M demonstrate that JoCoR is superior to many state-of-the-art approaches for learning with noisy labels.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, methodology): “∗Equal Contribution training procedures [85, 36].”
Learning From Noisy Labels With Deep Neural Networks: A Survey (2020, background): “In contrast, JoCoR [143] reduces the diversity of two networks via co-regularization, making predictions of the two networks closer.”
Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks (2022, background): “JoCoR [63] trains two similar neural networks on the same data set and tries to correlate two different labels.”