Collective intelligence for control of distributed dynamical systems

By David Hilton Wolpert, K. Wheeler, Kagan Tumer. 1999

We consider the El Farol bar problem, also known as the minority game (W. B. Arthur, The American Economic Review, 84 (1994) 406; D. Challet and Y. C. Zhang, Physica A, 256 (1998) 514). We view it as an instance of the general problem of how to configure the nodal elements of a distributed dynamical system so that they do not “work at cross purposes”, in that their collective dynamics avoids frustration and thereby achieves a provided global goal. We summarize a mathematical theory for such configuration applicable when (as in the bar problem) the global goal can be expressed as minimizing a global energy function and the nodes can be expressed as minimizers of local free energy functions. We show that a system designed with that theory performs nearly optimally for the bar problem.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Difference rewards ( D ) [48] and potential-based reward shaping ( PBRS ) [49] are two commonly used shaping approaches.”
A Theoretical and Empirical Analysis of Reward Transformations in Multi-Objective Stochastic Games (2017, background): “A difference reward (Di) is a shaped reward signal that aims to quantify each agent’s individual contribution to the system performance [20]:”
Unifying temporal and structural credit assignment problems (2004, methodology): “The fully observable difference utility has been successfully applied to various domains, including packet routing over a data network [15], congestion games [16], data downloads from a constellation of satellites [14], and multi-agent gridworlds [11].”
A Survey of Collectives (2004, methodology): “In addition we have explored COIN-based techniques on variants of congestion games [238, 242, 243], in particular of a more challenging variant of Arthur’s El Farol bar attendance problem [5] (also known as the “minority game” [48]).”
Learning sequences of actions in collectives of autonomous agents (2002, methodology): “multiple domains including packet routing over a data network [20] and the congestion game known as Arthur’s El Farol Bar problem [23].”