Thermodynamics as a theory of decision-making with information-processing costs

By Pedro A. Ortega, Daniel A. Braun. 2012

306 citations recorded by the source corpus.

Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here, we propose a thermodynamically inspired formalization of bounded rational decision-making where information processing is modelled as state changes in thermodynamic systems that can be quantified by differences in free energy. By optimizing a free energy, bounded rational decision-makers trade off expected utility gains and information-processing costs measured by the relative entropy. As a result, the bounded rational decision-making problem can be rephrased in terms of well-known variational principles from statistical physics. In the limit when computational costs are ignored, the maximum expected utility principle is recovered. We discuss links to existing decision-making frameworks and applications to human decision-making experiments that are at odds with expected utility theory. Since most of the mathematical machinery can be borrowed from statistical physics, the main contribution is to re-interpret the formalism of thermodynamic free-energy differences in terms of bounded rational decision-making and to discuss its relationship to human decision-making experiments.

Subfields: Atomic and Molecular Physics, and Optics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Statistical and Nonlinear Physics

Citations in the corpus

The anatomy of choice: active inference and agency (2013, background): “Important examples here include artiﬁcial curiosity (Schmidhuber, 1991), empowerment (Klyubin et al., 2005), information to go (Tishby and Polani, 2011) and computational complexity (Ortega and Braun, 2011, 2013).”
Active inference and epistemic value (2015, background): “Minimizing this divergence is a cornerstone of risk-sensitive control (Van Den Broek, Wiegerinck, & Kappen, 2010) and utilitybased free energy treatments of bounded rationality (Ortega & Braun, 2011, 2013).”
Active Inference: A Process Theory (2017, background): “…same quantity minimized in risk-sensitive or KL control (Klyubin, Polani, & Nehaniv, 2005; van den Broek, Wiegerinck, & Kappen, 2010), and underpins related variational formulations of bounded rationality based on complexity costs (Braun, Ortega, Theodorou, & Schaal, 2011; Ortega & Braun, 2013).”
Information-Theoretic Bounded Rationality (2015, methodology): “Perfect Rationality Bounded Rationality Transformations Variational Principle Sequential Decisions Conclusions Axioms of Transformation Costs Let ◮ (Ω,Σ) measurable space; ◮ P(·|·) : (Ω× Ω) → [0, 1] conditional probability measure.”
Exploiting Hierarchy for Learning and Transfer in KL-regularized RL (2019, background): “Another closely related regularization for RL is using information bottleneck (Tishby & Polani, 2011; Still & Precup, 2012; Rubin et al., 2012; Ortega & Braun, 2013; Tiomkin & Tishby, 2017).”
Information asymmetry in KL-regularized RL (2019, methodology): “…using tools from information and probability theory (e.g. Tishby & Polani, 2011; Ortega & Braun, 2011; Still & Precup, 2012; Rubin et al., 2012; Ortega & Braun, 2013; Tiomkin & Tishby, 2017), for instance, via constraints on the channel capacity between past states and future actions in a…”
Reward-rational (implicit) choice: A unifying formalism for reward learning (2020, methodology): “The finding provides validation for the approach pursued by Palan et al. [2019] and Ibarz et al.”
A Free Energy Principle for Biological Systems (2012, background): “Free energy formulations also provide a powerful account of optimal control beyond biological settings and can be derived axiomatically by considering the information costs and complexity of bounded rational decision-making [24].”