Predictive information in a sensory population

By S. Palmer, Olivier Marre, Michael J. Berry, William Bialek. 2013

269 citations recorded by the source corpus.

Significance Prediction is an essential part of life. However, are we really “good” at making predictions? More specifically, are pieces of our brain close to being optimal predictors? To assess the efficiency of prediction, we need to measure the information that neurons carry about the future of our sensory experiences. We show how to do this, at least in simplified contexts, and find that groups of neurons in the retina indeed are close to maximally efficient at separating predictive information from the nonpredictive background. Efficient coding of predictive information is a principle that can be applied at every stage of neural computation. Guiding behavior requires the brain to make predictions about the future values of sensory inputs. Here, we show that efficient predictive computation starts at the earliest stages of the visual system. We compute how much information groups of retinal ganglion cells carry about the future state of their visual inputs and show that nearly every cell in the retina participates in a group of cells for which this predictive information is close to the physical limit set by the statistical structure of the inputs themselves. Groups of cells in the retina carry information about the future state of their own activity, and we show that this information can be compressed further and encoded by downstream predictor neurons that exhibit feature selectivity that would support predictive computations. Efficient representation of predictive information is a candidate principle that can be applied at each stage of neural computation.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Evaluation & Benchmarks

Citations in the corpus

Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “, 2019), and theories of both spatial and temporal predictability have succeeded in describing properties of early visual areas (Rao & Ballard, 1999; Palmer et al., 2015).”
Deep Variational Information Bottleneck (2017, background): “This form of the information bottleneck is known as predictive information (Bialek et al., 2001; Palmer et al., 2015).”
On Variational Bounds of Mutual Information (2019, methodology): “Mutual information estimators are used in computational neuroscience (Palmer et al., 2015), Bayesian optimal experimental design (Ryan et al., 2016; Foster et al., 2018), understanding neural networks (Tishby et al., 2000; Tishby & Zaslavsky, 2015; Gabri´e et al., 2018), and more.”
Theory of cortical function (2017, background): “Because of space limitations, a number of influential papers were not cited in the main text (1-27).”
Toward a unified theory of efficient, predictive, and sparse coding (2017, background): “However, Bialek and coworkers (14, 15) recently proposed an alternative theory, in which neurons are hypothesized to preferentially encode sensory information that can be used to predict the future, while discarding other nonpredictive information (14–17).”
Greedy InfoMax for Biologically Plausible Self-Supervised Representation Learning (2019, background): “Empirical evidence indicates, for example, that retinal cells carry signiﬁcant mutual information between the current and the future state of their own activity [Palmer et al., 2015].”
Putting An End to End-to-End: Gradient-Isolated Learning of Representations (2019, methodology): “his prediction error [Friston, 2010]. Empirical evidence indicates, for example, that retinal cells carry signiﬁcant mutual information between the current and the future state of their own activity [Palmer et al., 2015]. Rao and Ballard [1999] indicate that this process may happen at each layer wit”
Wasserstein Dependency Measure for Representation Learning (2019, background): “[45] which has connections to predictive coding in speech [4, 19], predictive processing in the brain [10, 41, 46] and the free energy principle [20].”
Relevant sparse codes with variational information bottleneck (2016, methodology): “Finally, the IB approach has important implications for models of early sensory processing [17, 18].”