On Robustness of Neural Architecture Search Under Label Noise
By Chao Lan, Yi-Wei Chen, Qingquan Song, Xi Liu, P. Shanti Sastry, Xia Hu. 2020
11 citations recorded by the source corpus.
Neural architecture search (NAS), which aims at automatically seeking proper neural architectures given a specific task, has attracted extensive attention recently in supervised learning applications. In most real-world situations, the class labels provided in the training data would be noisy due to many reasons, such as subjective judgments, inadequate information, and random human errors. Existing work has demonstrated the adverse effects of label noise on the learning of weights of neural networks. These effects could become more critical in NAS since the architectures are not only trained with noisy labels but are also compared based on their performances on noisy validation sets. In this paper, we systematically explore the robustness of NAS under label noise. We show that label noise in the training and/or validation data can lead to various degrees of performance variations. Through empirical experiments, using robust loss functions can mitigate the performance degradation under symmetric label noise as well as under a simple model of class conditional label noise. We also provide a theoretical justification for this. Both empirical and theoretical results provide a strong argument in favor of employing the robust loss function in NAS under high-level noise.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Neural Architecture Search & AutoML
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Safety, Robustness & Interpretability
In Neural Architecture Search & AutoML
Citations in the corpus
- Techniques for Automated Machine Learning (2019, background): “One could make AutoDL robust to adversarial attack [18] and label noise [15].”