Predictability , Complexity , and Learning

By A. U.S.. 2002

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Variational Information Bottleneck (2017, background): “This form of the information bottleneck is known as predictive information (Bialek et al., 2001; Palmer et al., 2015).”
Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo (2015, methodology): “Exponential families (in Hilbert space) are an example, see e.g. Bialek et al. (2001); Hofmann et al. (2008); Wainwright & Jordan (2008).”
Predictable Feature Analysis (2013, background): “It has been frequently proposed that predictable features are crucial to obtain such a model, see [1] for a review.”
A Free Energy Principle for Biological Systems (2012, unknown): “This perspective on self-organisation is interesting because it connects probabilistic descriptions of the states occupied by biological systems to probabilistic modelling or inference as described by Bayesian probability and information theory [1,2].”
The free-energy principle: a unified brain theory? (2010, background): “It has been extended to cover dynamics and motion trajectorie”
Perceptions as Hypotheses: Saccades as Experiments (2012, background): “Given these assumptions, one can simulate a whole variety of neuronalprocessesbyspecifyingtheparticularequationsthatcon-stitutethebrain’sgenerativemodel.”
Automated adaptive inference of phenomenological dynamical models (2015, unknown): “This example also demonstrates that adaptive modeling can hint at the complexity of the hidden dynamics beyond those measured: the best performing sigmoidal model requires three hidden variables, for a total of six chemical species, which is exactly what one would expect for a seven-dimensional syst”
Neural processing of natural sounds (2014, background): “It has also been argued that neither white-noise (a random signal with equal power at all frequencies) nor a pure sine wave can qualify as complex and, thus, information rich or perceptually sophisticated[13,18].”
Generative Temporal Models with Memory (2017, background): “Bialek et al. (2001) quantitatively show that Markov assumptions fail to describe physical systems with long-range correlations, and fail to approximate the long-distance dependencies in written literature.”
Predictive information in a sensory population (2013, background): “The statement that predictive power is limited is, quantitatively, the statement that the predictive information, Ipred(T ) ≡ I(Xpast;Xfuture) is finite [4].”
Information Theory of Decisions and Actions (2011, background): “One implication of this assumption is that the future information of the organism is bounded by the predictive information (defined e.g. in Shalizi and Crutchfield 2002; Bialek et al. 2001) of the environment (Bialek et al. 2007).”
Predicting evolution (2017, background): “The ‘information gain’, defined as the log ratio of predicted and prior probability, measures how much the prediction reduces our uncertainty about the future process 13,128 .”
Life as we know it (2013, unknown): “Furthermore, we have shown elsewhere [11,41] that free energy minimization is consistent with information-theoretic formulations of sensory processing and behaviour [23,42,43].”
A free energy principle for a particular physics (2019, background): “…(Barlow, 1961; Barlow, 1974), the Infomax principle (Linsker, 1990) and formulations of epistemic foraging in terms of information theory and artificial curiosity (Bialek et al., 2001; Itti and Baldi, 2009; Schmidhuber, 2010; Still and Precup, 2012; Sun et al., 2011; Tishby and Polani, 2010).”
Relevant sparse codes with variational information bottleneck (2016, methodology): “The IB approach has been used to tackle a wide variety of problems, including filtering, prediction and learning [2-5].”