On the information bottleneck theory of deep learning

By Andrew M. Saxe, Yamini Bansal, Joel Dapello, Madhu S. Advani, Artemy Kolchinsky, Brendan D. Tracey, David D. Cox. 2018

692 citations recorded by the source corpus.

The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct phases consisting of an initial fitting phase and a subsequent compression phase; second, that the compression phase is causally related to the excellent generalization performance of deep networks; and third, that the compression phase occurs due to the diffusion-like behavior of stochastic gradient descent. Here we show that none of these claims hold true in the general case, and instead reflect assumptions made to compute a finite mutual information metric in deterministic networks. When computed using simple binning, we demonstrate through a combination of analytical results and simulation that the information plane trajectory observed in prior work is predominantly a function of the neural nonlinearity employed: double-sided saturating nonlinearities like yield a compression phase as neural activations enter the saturation regime, but linear activation functions and single-sided saturating nonlinearities like the widely used ReLU in fact do not. Moreover, we find that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa. Next, we show that the compression phase, when it exists, does not arise from stochasticity in training by demonstrating that we can replicate the IB findings using full batch gradient descent rather than stochastic gradient descent. Finally, we show that when an input domain consists of a subset of task-relevant and task-irrelevant information, hidden representations do compress the task-irrelevant information, although the overall information about the input may monotonically increase with training time, and that this compression happens concurrently with the fitting process rather than during a subsequent compression period.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Scaling & Learning Theory

Citations in the corpus

To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks (2019, methodology): “For the same settings, we obtain non-zero values with earlier estimators (Saxe et al., 2018), which seem to be less reliable for higher numbers of dimensions.”
Why Should We Add Early Exits to Neural Networks? (2020, background): “However, it can be replaced with a noisy approximation [55].”
Wireless Network Intelligence at the Edge (2018, background): “7(b)] [92] and the training objective function [93], which can be partly rectified by modifying the objective function and/or inserting noise [94].”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “…machine learning community (Zimmer et al., 2014; Zhong et al., 2017; Tian, 2017; Du et al., 2018; Soltanolkotabi et al., 2018; Aubin et al., 2018; Saxe et al., 2018; Baity-Jesi et al., 2018; Goldt et al., 2019; Ghorbani et al., 2019; Yoshida & Okada, 2019; Ndirango & Lee, 2019; Gabri ´ e, 2020;…”
Where is the Information in a Deep Neural Network? (2019, methodology): “This presents a challenge for theories of Deep Learning based on Shannon Information [15].”
On Variational Bounds of Mutual Information (2019, background): “Existing sample-based estimators are brittle, with the hyperparameters of the estimator impacting the scientiﬁc conclusions (Saxe et al., 2018).”
Three Factors Influencing Minima in SGD (2017, background): “In (Saxe et al., 2018; Shwartz-Ziv & Tishby, 2017) (see also (Zhu et al., 2018) who conﬁrm this), they show the squared norm of the mean gradient is much smaller than the expected squared norm of the gradient From this we have that We then have that our expression for the sample covariance matrix…”
On Identifiability in Transformers (2019, background): “., 2017; Radford et al., 2018; 2019; Devlin et al., 2019) to playing complex games (Mnih et al., 2015; Silver et al., 2018), is confronted with lack of explainability (Shwartz-Ziv & Tishby, 2017; Saxe et al., 2018). Deep learning models are known to be “black boxes” (Shwartz-Ziv & Tishby, 20”
Continual Learning with Foundation Models: An Empirical Study of Latent Replay (2022, background): “…privacy since storing latent representations of a privacy-preserving model (Xu et al., 2021) for replay may be more secure than storing raw data, as well as (c) better data efﬁciency since latent representations contain less task-irrelevant information such as pixel noise (Saxe et al., 2019).”
Policy Gradients Incorporating the Future (2021, background): “, data collection) when the future trajectory is unavailable, we place an information bottleneck [33, 43] on the additional inputs, encouraging the learned functions to minimize their reliance on this privileged information.”
Caveats for information bottleneck in deterministic scenarios (2018, neutral): “Finally, Saxe et al. (2018) and Amjad & Geiger (2018) observed that when T is continuous-valued and a deterministic function of a continuous-valued X , I ( X ; T ) can be unbounded, making the application of the IB framework problematic.”
Measuring Compositionality in Representation Learning (2019, background): “(For discussion of limitations of this approach to computing mutual information between inputs and representations, see Saxe et al., 2018.)”
Heterogeneous Knowledge Distillation Using Information Flow Modeling (2020, background): “After forming these critical connections, the ﬁtting and compression (when applicable) phases follow [21, 20].”
Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle (2018, methodology): “4 shed new light in this debate and are in line with the observations made in [3].”
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges (2021, background): “This has led to attempts at explaining the behavior of deep neural networks with the information bottleneck formalism [123].”
Why Do Better Loss Functions Lead to Less Transferable Features? (2020, background): “More generally, studies have investigated how class information evolves through the hidden layers of neural networks using linear classifiers [2], binning estimators of mutual information [61, 58, 26], Euclidean distances [59], the soft nearest neighbor loss [24], and manifold geometry [3, 16].”