Mixing Complexity and its Applications to Neural Networks
By Michal Moshkovitz, Naftali Tishby. 2017
11 citations recorded by the source corpus.
We suggest analyzing neural networks through the prism of space constraints. We observe that most training algorithms applied in practice use bounded memory, which enables us to use a new notion introduced in the study of space-time tradeoffs that we call mixing complexity. This notion was devised in order to measure the (in)ability to learn using a bounded-memory algorithm. In this paper we describe how we use mixing complexity to obtain new results on what can and cannot be learned using neural networks.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Ravid Shwartz-Ziv
- Naftali Tishby
- Alessandro Achille
- Luis A. Lastras
- Stefano Soatto
- Jacob Andreas
- Adar Elad
- Charles H. Martin
Citations in the corpus
- Opening the Black Box of Deep Neural Networks via Information (2017, methodology): “Good overviews on methods for mutual information estimation can be found in Paninski (2003) and Kraskov et al.”