Free Energy, Value, and Attractors

By Karl J. Friston, Ping Ao. 2011

149 citations recorded by the source corpus.

It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.

Subfields: Cognitive Neuroscience, Global and Planetary Change

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Global and Planetary Change

Citations in the corpus

Information seeking, curiosity and attention: computational and neural mechanisms (2013, positive): “Converging evidence suggests that, when deployed in the service of a task, eye movements may be explained by the simple imperative to sample information in order to reduce uncertainty regarding future states (Tatler, Hayhoe et al. 2011; Friston and Ao 2012).”
The math is not the territory: navigating the free energy principle (2021, methodology): “The static solution to the Fokker-Planck equation is a probability density termed the Nonequilibrium Steady State density, or NESS density (Friston 2019; Friston and Ao 2012).”
A free energy principle for a particular physics (2019, methodology): “random fluctuations to reveal classical mechanics. However, it is interesting to consider what would happen if they were reintroduced. Random fluctuations on (classical) momentum emerges as friction (Friston and Ao, 2012), where, following (7.7): 0 00 , 0 0 ( | ) I Q I K ª º * |«» «»® ® ¬ ¯ ¿¬”
A Free Energy Principle for Biological Systems (2012, background): “Domain Process or paradigm Perception • Perceptual categorisation (bird songs) [18] • Novelty and omission-related responses [18] • Perceptual inference (speech) [19] Sensory learning • Perceptual learning (mismatch negativity) [33] Attention • Attention and the Posner paradigm [20] • Attention and ”
Life as we know it (2013, unknown): “…potential G ( x ) we will call Gibbs energy [32], Using this standard form [33], it is straightforward to show that p ( x j m ) ¼ exp( 2 G ( x )) is the equilibrium solution to the Fokker–Planck equation [12]: This means that we can express the flow in terms of the ergodic density f ¼ð G þ R Þ (cid”
Answering Schrödinger's question: A free-energy formulation (2017, unknown): “…also been applied to biological systems across spatial and temporal scales, ranging from phenomena at the micro-scale (e.g., dendritic self-organisation and morphogenesis; [27,28]), across intermediate scales (e.g., cultural ensembles; [29]), and at the macro-scale (e.g., natural selection; [30]).”