Dropout distillation

48 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “It was widely applied in learning a compact network by fitting the soft labels generated by a pretrained large model (LopezPaz et al., 2015; Bulò et al., 2016; Wang et al., 2018c; Chen et al., 2019a; 2020b), therefore compressing the latter.”
Learning from Noisy Labels with Distillation (2017, methodology): “used distillation to extract an optimal predictor from a model trained with dropout, which outperforms the standard scaling based dropout [2].”
Noise as a Resource for Learning in Knowledge Distillation (2020, unknown): “[7] and Gurau et al .”
Demystifying Dropout (2019, methodology): “† denotes the result is extracted from (Bulò et al., 2016).”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, background): “Some other regularization approaches are proposed as well, details in [124].”
Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization (2017, background): “Bulo et al. [6] improve dropout by reducing gap between training and inference procedure, where the output of dropout layers in inference stage is given by learning expected average of multiple dropouts.”
Uncertainty in Deep Learning (2016, background): “In this section three main applications will be surveyed: new tools [Bulò et al., 2016], image processing for autonomous driving [Kendall and Cipolla, 2016; Kendall et al.”