Robust Loss Functions for Learning Multi-class Classifiers

By Himanshu Kumar, P. Shanti Sastry. 2018

17 citations recorded by the source corpus.

Robust learning in presence of label noise is an important problem of current interest. Training data often has label noise due to subjective biases of experts, crowd-sourced labelling or other automatic labelling processes. Recently, some sufficient conditions on a loss function are proposed so that risk minimization under such loss functions is provably tolerant to label noise. The standard loss functions such as cross-entropy or mean-squared error, used for learning neural network classifiers, do not satisfy these conditions. It was shown that a loss function based on mean absolute value of error satisfies the conditions and is also empirically seen to be robust to label noise. However, minimizing absolute value of error is a difficult optimization problem. In this paper we propose a new loss function, called robust log loss and show that it satisfies the sufficient conditions for robustness. The resulting optimization problem of minimizing empirical risk is well behaved. Through extensive empirical results we show that, in terms of accuracy and learning rate, the proposed loss function is as good as cross-entropy loss for learning neural network classifiers when there is no label noise and that it is better when the training data has label noise.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qingquan Song
2. Xia Hu
3. Yi-Wei Chen
4. Xi Liu
5. Chao Lan
6. P. Shanti Sastry
7. Ninghao Liu
8. Fan Yang

Citations in the corpus

• On Robustness of Neural Architecture Search Under Label Noise (2020, methodology): “The performance of neural networks decreases by label noise, but the symmetric loss can alleviate the adverse influence, as shown in Kumar and Sastry (2018).”