Robust Inference via Generative Classifiers for Handling Noisy Labels

By Kimin Lee, Sukmin Yun, Kibok Lee, Honglak Lee, Bo Li, Jinwoo Shin. 2019

163 citations recorded by the source corpus.

Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets. To mitigate the issue, we propose a novel inference method, termed Robust Generative classifier (RoG), applicable to any discriminative (e.g., softmax) neural classifier pre-trained on noisy datasets. In particular, we induce a generative classifier on top of hidden feature spaces of the pre-trained DNNs, for obtaining a more robust decision boundary. By estimating the parameters of generative classifier using the minimum covariance determinant estimator, we significantly improve the classification accuracy with neither re-training of the deep model nor changing its architectures. With the assumption of Gaussian distribution for features, we prove that RoG generalizes better than baselines under noisy labels. Finally, we propose the ensemble version of RoG to improve its performance by investigating the layer-wise characteristics of DNNs. Our extensive experimental results demonstrate the superiority of RoG given different learning models optimized by several training techniques to handle diverse scenarios of noisy labels.

Subfields: Artificial Intelligence, Civil and Structural Engineering, Statistics and Probability, Generative Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Weilong Yang
2. Davood Karimi
3. Ivor Wai-Hung Tsang
4. Yueming Lyu
5. Di Huang
6. Lu Jiang
7. Haoran Dou
8. A. Gholipour

In Civil and Structural Engineering

1. Haoran Dou
2. A. Gholipour
3. Di Huang
4. Simon Keith Warfield
5. Davood Karimi
6. Lu Jiang
7. Weilong Yang

In Statistics and Probability

1. A. Gholipour
2. Simon Keith Warfield
3. Ivor Wai-Hung Tsang
4. Lu Jiang
5. Weilong Yang

In Generative Models

1. A. Gholipour
2. Shaoting Zhang
3. Ivor Wai-Hung Tsang
4. Lu Jiang
5. Weilong Yang

In Safety, Robustness & Interpretability

1. Weilong Yang
2. Ivor Wai-Hung Tsang
3. Di Huang
4. Haoran Dou
5. A. Gholipour
6. Simon Keith Warfield
7. Davood Karimi
8. Lu Jiang

Citations in the corpus

• VOXLINGUA107: A Dataset for Spoken Language Recognition (2020, methodology): “On top of the extracted features, we trained a generative classifier which utilizes the minimum covariance determinant (MCD) estimator to estimate its parameters, as proposed in [11].”
• A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images (2020, methodology): “Other techniques including consistency-based regularization [32], model ensemble [33] and iterative training [31] have also been used for learning from noisy labels for classification.”
• Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels (2019, background): “…label is also commonly used as a benchmark to evaluate robust learning methods in a noise-control setting e.g. in (Vahdat, 2017; Shu et al., 2019; Jiang et al., 2018; Ma et al., 2018; Han et al., 2018b; Van Rooyen et al., 2015; Li et al., 2019; Arazo et al., 2019; Charoenphakdee et al., 2019).”
• Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, background): “A typical example is the work of [96], where the authors consider the correlation of the features learned by a deep learning model.”
• Curriculum Loss: Robust Learning and Generalization against Label Corruption (2019, methodology): “We use the code of (Lee et al., 2019) online, and only change the loss for comparison.”