Smart Regularization of Deep Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Variational Dropout Sparsifies Deep Neural Networks (2017, background): “Variational Spike-and-Slab Neural Networks (Louizos, 2015) is yet another Bayesian interpretation of Binary Dropout that allows for tuning of individual dropout rates and also leads to a sparse solution.”
Bayesian Compression for Deep Learning (2017, background): “Dropout [29, 67], one of the most popular regularization techniques for neural networks, can be interpreted as positing a spike and slab distribution over the weights where the variance of the “slab” is zero [17, 45].”
Structured Bayesian Pruning via Log-Normal Multiplicative Noise (2017, neutral): “Variational Dropout [8] and Variational Spike-and-Slab Neural Networks [13].”
Robustly representing inferential uncertainty in deep neural networks through sampling (2016, methodology): “Interestingly, Bernoulli dropout and dropconnect can be seen as approximations to spike-and-slab distributions for units and weights, respectively (Louizos, 2015; Gal, 2016).”