Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise

By Pengfei Chen, Guangyong Chen, Jingwei Zhao, P. Heng. 2020

159 citations recorded by the source corpus.

Supervised learning under label noise has seen numerous advances recently, while existing theoretical findings and empirical results broadly build up on the class-conditional noise (CCN) assumption that the noise is independent of input features given the true label. In this work, we present a theoretical hypothesis testing and prove that noise in real-world dataset is unlikely to be CCN, which confirms that label noise should depend on the instance and justifies the urgent need to go beyond the CCN assumption.The theoretical results motivate us to study the more general and practical-relevant instance-dependent noise (IDN). To stimulate the development of theory and methodology on IDN, we formalize an algorithm to generate controllable IDN and present both theoretical and empirical evidence to show that IDN is semantically meaningful and challenging. As a primary attempt to combat IDN, we present a tiny algorithm termed self-evolution average label (SEAL), which not only stands out under IDN with various noise fractions, but also improves the generalization on real-world noise benchmark Clothing1M. Our code is released. Notably, our theoretical analysis in Section 2 provides rigorous motivations for studying IDN, which is an important topic that deserves more research attention in future.

Subfields: Artificial Intelligence, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jingwei Zhao
2. P. Heng
3. Pengfei Chen
4. Guangyong Chen
5. Qi-Yan Dou

Citations in the corpus

• Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, neutral): “SEAL [9] averages the model’s softmax output of each example over the whole training set.”
• Noise against noise: stochastic label noise helps combat inherent label noise (2021, methodology): “• Instance-dependent noise is challenging since the mislabeling probability should be dependent on each instance’s input features (Xia et al., 2020; Chen et al., 2020a).”