Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning

By Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro A. Ortega, DJ Strouse, Joel Z. Leibo, Nando de Freitas. 2018

553 citations recorded by the source corpus.

Subfields: Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “Agents may incorporate ideas such as intrinsic rewards and auxiliary tasks (Jaderberg et al., 2017; Jaques et al., 2019), ensembling (Osband et al., 2018), prioritized experience replay (Schaul et al., 2015), distributional backups (Bellemare et al., 2017), specialised neural network architectures…”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “…the emergence of tool usage (Baker et al., 2020), communication (Foerster et al., 2016; Sukhbaatar et al., 2016; Lazaridou et al., 2017; Das et al., 2019), social inﬂuence (Jaques et al., 2019), and inequity aversion (Hughes et al., 2018) can also be better explained from a MARL perspective.”
Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks (2020, methodology): “These algorithm classes are frequently used in the literature either as baselines or building blocks for more complex algorithms [He et al., 2016, Sukhbaatar et al., 2016, Foerster et al., 2016, Raileanu et al., 2018, Jaques et al., 2019, Iqbal and Sha, 2019, Du et al., 2019, Ryu et al., 2020].”
Emergent Tool Use From Multi-Agent Autocurricula (2019, background): “[50] consider multi-agent scenarios and adopt causal influence as a motivation for coordination.”
Learning Nearly Decomposable Value Functions Via Communication Minimization (2019, methodology): “Social inﬂuence (Jaques et al., 2019) and InfoBot (Goyal et al., 2019) penalize message that has no effect on policies of other agents.”
Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning (2019, background): “[16] approximate the direct causal effect between agent’s actions and use it as an intrinsic reward to encourage social empowerment.”
An Introduction to Deep Reinforcement Learning (2018, unknown): “Other works have shown how it is possible to take into account a term that either considers the learning of the other agent (Foerster et al., 2018) or an internal model that predicts the actions of other agents (Jaques et al., 2018).”
Multi-agent deep reinforcement learning: a survey (2021, methodology): “2017), in social dilemmas (Jaques et al. 2019; Letcher et al. 2019), and by predicting behaviors from observations (Hong et al.”
Influence-Based Multi-Agent Exploration (2019, background): “One notably relevant work is Jaques et al. (2018), which models the social influence of one agent on other agents’ policies.”
Building Cooperative Embodied Agents Modularly with Large Language Models (2023, background): “…Samvelyan et al., 2019; Carroll et al., 2019; Puig et al., 2021; 2023), or use uninterpretable continuous vectors (Jiang & Lu, 2018; Das et al., 2019) or limited discrete symbols (Lowe et al., 2017; Jaques et al., 2019; Jain et al., 2020; Patel et al., 2021; Resnick et al., 2018) for communication.”
Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium (2020, background): “, 2016), and learning communication and emergent behaviors (Foerster et al., 2016; Lowe et al., 2017; Bansal et al., 2017; Jaques et al., 2018; Baker et al., 2019); see the surveys in Busoniu et al.”
Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning (2020, methodology): “In recent years, a growing body of work has explored the effects of furnishing agents in mixedmotive games with various motivations such as inequity aversion [24, 56], social imitation [15], and social influence [28].”
FOP: Factorizing Optimal Joint Policy of Maximum-Entropy Multi-Agent Reinforcement Learning (2021, background): “…made great progress in many aspects by solving social dilemma (Hughes et al., 2018; Eccles et al., 2019), characterizing inﬂuence between agents (Jaques et al., 2019), optimizing both efﬁciency and fairness (Jiang & Lu, 2019), incorporating agent communication (Sukhbaatar et al., 2016; Das et…”
Influencing Long-Term Behavior in Multiagent Reinforcement Learning (2022, unknown): “Our work is also related to frameworks that leverage the inferred policy dynamics of other agents to impact their future policies by maximizing the discounted return objective (Jaques et al., 2019; Xie et al., 2020; Wang et al., 2021).”
Off-Policy Multi-Agent Decomposed Policy Gradients (2020, background): “Cooperative multi-agent reinforcement learning (MARL) has achieved great progress in recent years (Hughes et al., 2018; Jaques et al., 2019; Vinyals et al., 2019; Zhang et al., 2019; Baker et al., 2020; Wang et al., 2020c).”
Dealing with Non-Stationarity in MARL via Trust-Region Decomposition (2021, methodology): “Therefore, we model the cooperative relationship between two agents based on counterfactual (Foerster et al., 2018b; Jaques et al., 2019).”
Local Information Agent Modelling in Partially-Observable Environments (2021, methodology): “The information of all agents can be utilised during training for various reasons, such as computing a joint value function [Lowe et al., 2017, Foerster et al., 2018, Sunehag et al., 2018, Rashid et al., 2018], or generating intrinsic rewards [Jaques et al., 2019].”