Optimal Architectures in a Solvable Model of Deep Networks
By Jonathan Kadmon, Haim Sompolinsky. 2016
28 citations recorded by the source corpus.
Subfields: Scaling & Learning Theory, Statistical Learning Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Scaling & Learning Theory
- Andrew M. Saxe
- Madhu S. Advani
- David D. Cox
- Yamini Bansal
- Brendan D. Tracey
- Artemy Kolchinsky
- Joel Dapello
- Ravid Shwartz-Ziv
In Statistical Learning Theory
- Ravid Shwartz-Ziv
- Shotaro Akaho
- Shun-ichi Amari
- Naftali Tishby
- David D. Cox
- Michal Moshkovitz
- Ryo Karakida
- Andrew M. Saxe
Citations in the corpus
- Opening the Black Box of Deep Neural Networks via Information (2017, positive): “This is consistent with other recent suggestions on the role of noise in Deep Learning [Achille & Soatto (2016),Kadmon & Sompolinsky (2016)].”
- Mixing Complexity and its Applications to Neural Networks (2017, background): “Understanding neural network learning is an active research area in machine learning and neuroscience (Shamir, 2016; Safran & Shamir, 2016; Eldan & Shamir, 2016; Daniely et al., 2016; Raghu et al., 2016; Arora et al., 2014; Livni et al., 2014; Tishby & Zaslavsky, 2015; Kadmon & Sompolinsky, 2016).”
- On the information bottleneck theory of deep learning (2018, background): “These practical successes have spawned many attempts to explain the performance of deep learning systems (Kadmon and Sompolinsky 2016), mostly in terms of the properties and dynamics of the optimization problem in the space of weights (Saxe et al 2014, Choromanska et al 2015, Advani and Saxe 2017),…”
- Universal statistics of Fisher information in deep neural networks: mean field approach (2018, methodology): “One widely used approach for this purpose is to consider deep networks with random connectivity and a large width limit [2–14].”
- Shared cortex-cerebellum dynamics in the execution and learning of a motor task (2019, unknown): “Finally, while the classical theory of granule cell function (Albus, 1971; Marr, 1969) focused on dimensionality expansion, a modern reanalysis of this theory indicates that maximal dimensionality expansion is not always preferable, as such expansion alone can also amplify input noise (Babadi and So”