Guided Dropout

By Rohit Keshari, Richa Singh, Mayank Vatsa. 2018

37 citations recorded by the source corpus.

Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes random drop of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the traditional dropout. In this research, we propose “guided dropout” for training deep neural network which drop nodes by measuring the strength of each node. We also demonstrate that conventional dropout is a specific case of the proposed guided dropout. Experimental evaluation on multiple datasets including MNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet demonstrate the efficacy of the proposed guided dropout.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wei Hu
2. Sanjeev Arora
3. Yuanzhi Li
4. Zhiyuan Li
5. Zeyuan Allen-Zhu
6. Tengyu Ma
7. Rong Ge
8. Yi Zhang

In Computer Vision and Pattern Recognition

1. Sanjeev Arora
2. Wei Hu
3. Zhiyuan Li
4. Yi Zhang
5. Yuanzhi Li
6. Rong Ge
7. Dong-Wan Choi
8. Xiang Wang

Citations in the corpus

• Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network (2021, methodology): “As a base network, we use ResNet-18 for both datasets as it is widely used for various benchmark datasets including CIFAR-100 and TinyImageNet in the literature (Devries and Taylor 2017; Keshari, Singh, and Vatsa 2019).”
• Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets (2019, background): “We also demonstrate that the VGG-11 (Simonyan and Zisserman, 2014) architecture trained with channel-wise dropout (Tompson et al., 2015; Keshari et al., 2018) with p = 0 .”