The effectiveness of layer-by-layer training using the information bottleneck principle

By Adar Elad, Doron Haviv, Yochai Blau, Tomer Michaeli. 2018

10 citations recorded by the source corpus.

Subfields: Optimization & Training, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Peter O'Connor
2. Michele Scarpiniti
3. Aurelio Uncini
4. Enzo Baccarelli
5. Simone Scardapane
6. Bastiaan S. Veeling
7. Sindy Löwe

In Information Theory

1. Simone Scardapane

Citations in the corpus

• Why Should We Add Early Exits to Neural Networks? (2020, background): “Interestingly, one possibility for the actual implementation and test of the IB principle results in a different training scenario for multi-exit neural networks [ 19 ] .”
• Greedy InfoMax for Biologically Plausible Self-Supervised Representation Learning (2019, background): “In an attempt to validate information bottleneck theory, Elad et al. [2018] develop a supervised, layer-wise training method that maximizes the mutual information between the outputs of a layer and the target whilst minimizing the mutual information between the inputs and outputs.”
• Greedy Layerwise Learning Can Scale to ImageNet (2018, unknown): “Concurrent work (Elad et al., 2019) follows the same argument to use a layerwise training procedure to evaluate mutual information more directly.”