WCP: Worst-Case Perturbations for Semi-Supervised Deep Learning
By Liheng Zhang, Guo-Jun Qi. 2020
50 citations recorded by the source corpus.
In this paper, we present a novel regularization mechanism for training deep networks by minimizing the {\em Worse-Case Perturbation} (WCP). It is based on the idea that a robust model is least likely to be affected by small perturbations, such that its output decisions should be as stable as possible on both labeled and unlabeled examples. We will consider two forms of WCP regularizations -- additive and DropConnect perturbations, which impose additive noises on network weights, and make structural changes by dropping the network connections, respectively. We will show that the worse cases of both perturbations can be derived by solving respective optimization problems with spectral methods. The WCP can be minimized on both labeled and unlabeled data so that networks can be trained in a semi-supervised fashion. This leads to a novel paradigm of semi-supervised classifiers by stabilizing the predicted outputs in presence of the worse-case perturbations imposed on the network weights and structures.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Massimiliano Mancini
- Elisa Ricci
- Enrico Fini
- Zeynep Akata
- Moin Nabi
- Xiatian Zhu
- Irwin King
- Zenglin Xu
Citations in the corpus
- A soft nearest-neighbor framework for continual semi-supervised learning (2022, background): “Another class of approaches was derived with the idea of imposing similar predictions from the network for two samples obtained with different input perturbations [3,4,27,33, 35, 41, 47, 50, 60].”
- A Survey on Deep Semi-Supervised Learning (2021, methodology): “Image classification MNIST [60], SVHN [61], STL-10 [62], Cifar-10,Cifar-100 [63], ImageNet [64] [65], [66], [67], [68], [69], [70], [71], [72], [73] Object detection PASCAL VOC [74], MSCOCO [75], ILSVRC [76] [58], [77], [78], [79] 3D object detection SUN RGB-D [80], ScanNet [81], KITTI [82] [83],…”
- Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Worst-Case Perturbations (WCP) introduces both addictive perturbations and drop connections on model parameters, where drop connections set certain model weights to zero to further change the network structure [63].”