Surprising properties of dropout in deep networks

37 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Understanding the Role of Training Regimes in Continual Learning (2020, background): “Dropout [27] is a well-established technique in deep learning, which is also well-studied theoretically [3, 70, 26, 18, 74].”
Dropout as an Implicit Gating Mechanism For Continual Learning (2020, background): “Dropout [10] is a well-established technique in deep learning, which is also well-studied theoretically [2, 24, 9, 25].”
Demystifying Dropout (2019, methodology): “In (Helmbold & Long, 2017), dropout is also interpreted as a regularization method to avoid overfitting.”
The Implicit and Explicit Regularization Effects of Dropout (2020, background): “…several theoretical works studying its properties for both optimization and generalization (Wager et al., 2013, 2014; Baldi & Sadowski, 2013; Helmbold & Long, 2015; Gal & Ghahramani, 2016a; Helmbold & Long, 2017; Cavazza et al., 2017; Mianjy et al., 2018; Mianjy & Arora, 2019; Arora et al., 2020).”