Malthusian Reinforcement Learning

By Joel Z. Leibo, Julien Pérolat, Edward Hughes, Steven Wheelwright, Adam Henry Marblestone, Edgar A. Duéñez-Guzmán, Peter Sunehag, Iain Dunning, Thore Graepel. 2018

43 citations recorded by the source corpus.

Here we explore a new algorithmic framework for multi-agent reinforcement learning, called Malthusian reinforcement learning, which extends self-play to include fitness-linked population size dynamics that drive ongoing innovation. In Malthusian RL, increases in a subpopulation's average return drive subsequent increases in its size, just as Thomas Malthus argued in 1798 was the relationship between preindustrial income levels and population growth. Malthusian reinforcement learning harnesses the competitive pressures arising from growing and shrinking population size to drive agents to explore regions of state and policy spaces that they could not otherwise reach. Furthermore, in environments where there are potential gains from specialization and division of labor, we show that Malthusian reinforcement learning is better positioned to take advantage of such synergies than algorithms based on self-play.

Subfields: Economics and Econometrics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

Citations in the corpus

Emergent Tool Use From Multi-Agent Autocurricula (2019, background): “[51] Joel Z Leibo, Julien Perolat, Edward Hughes, Steven Wheelwright, Adam H Marblestone, Edgar Duéñez-Guzmán, Peter Sunehag, Iain Dunning, and Thore Graepel.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “works have also shown that plain self-play does not yield the best results. However, adding diversity, i.e.,evolutionary methods [232, 233, 227, 228]or sampling-based methods, have shown good results [155, 176, 156]. A drawback of these solutions is the additional computational requirements since th”
Options as responses: Grounding behavioural hierarchies in multi-agent RL (2019, methodology): “Second , we demonstrate that OPRE successfully resolves the social dilemma game Allelopathy [17] in self-play.”
Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research (2019, background): “A recent paper describing an algorithm called Malthusian reinforcement learning considered them in more general settings (Leibo et al., 2019).”
Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning (2020, background): “Leibo et al. [17] note the overall beneﬁts on exploration when sets of agents are dynamically evolved and mixed.”