Cooperative Multi-agent Control Using Deep Reinforcement Learning

By Jayesh K. Gupta, Maxim Egorov, Mykel J. Kochenderfer. 2017

1,047 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Computer Networks and Communications, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Computer Networks and Communications

In Reinforcement Learning

Citations in the corpus

Counterfactual Multi-Agent Policy Gradients (2017, methodology): “Gupta et al. (2017) investigate actor-critic methods for decentralised execution with centralised training.”
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, neutral): “Gupta et al. (2017) present a centralised actor-critic algorithm with per-agent critics, which scales easily with the number of agents but mitigates the advantages of centralisation.”
Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication (2020, neutral): “It is possible to hand-engineer agent-level local reward functions in some scenarios [8], however, this technique has diﬃ-culties generalizing to other complex problems and requires domain expertise.”
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017, background): “Another approach is to indirectly arrive at cooperation via sharing of policy parameters [12], but this requires a homogeneous agent population.”
Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing (2021, methodology): “One common implementation technique to facilitate training with a larger number of agents is parameter sharing (e.g. (Gupta et al., 2017)) whereby agents share some or all parameters in their policy networks.”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, methodology): “The framework of centralized training with decentralized execution (CTDE) (Foerster et al., 2016; Gupta et al., 2017; Rashid et al., 2018) combines the advantages of these two paradigms.”
QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, methodology): “Gupta et al. (2017) implements per-agent critics to opt for better scalability at the cost of diluted benefits of centralization.”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, methodology): “…can learn a fully centralised state-action value function Q tot and then use it to guide the optimisation of decentralised policies in an actor-critic framework, an approach taken by counterfactual multi-agent (COMA) policy gradients (Foerster et al., 2018), as well as work by Gupta et al. (2017).”
Action Semantics Network: Considering the Effects of Actions in Multiagent Systems (2019, background): “If agents are homogeneous, their policy networks can be trained more efficiently using PS which greatly reduces the training complexity (Gupta et al., 2017).”
Multi-Agent Common Knowledge Reinforcement Learning (2018, background): “exploration and their changing policies. Rashid et al.(2018a) andSunehag et al.(2017) propose learning a centralised value function that factors into peragent components to allow for decentralisation.Gupta et al. (2017) learn separate policies for each agent in an actorcritic framework, where the cr”
Actor-Attention-Critic for Multi-Agent Reinforcement Learning (2018, methodology): “As deep learning based approaches to reinforcement learning have grown more popular, they have, naturally, been applied to the MARL setting (Tampuu et al., 2017; Gupta et al., 2017), allowing multi-agent learning in highdimensional/continuous state spaces; however, naive applications of Deep RL meth”
Negative Update Intervals in Deep Multi-Agent Reinforcement Learning (2018, methodology): “We therefore follow the example of recent publications by conducting evaluations in gridworlds with sufficient complexity to warrant a MA-DRL approach [7, 11, 19].”
Deep Reinforcement Learning for Swarm Systems (2018, methodology): “Gupta et al. [2017] show a comparison of centralized, concurrent and parameter sharing approaches to cooperative deep MARL, using TRPO [Schulman et al., 2015], DDPG [Lillicrap et al., 2015] and DQN.”
Mean Field Multi-Agent Reinforcement Learning (2018, background): “In addressing the non-stationary issue in MARL, various solutions have been proposed including neural-based opponent modeling (He & Boyd-Graber, 2016), policy parameters sharing (Gupta et al., 2017), etc .”
Lenient Multi-Agent Deep Reinforcement Learning (2017, methodology): “[11] evaluated policy gradient, temporal difference error, and actor critic methods on cooperative control tasks that included discrete and continuous state and action spaces, using a decentralized parameter sharing approach with centralized learning.”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “This method was proposed concurrently in different works [96,124] and later it has been successfully applied in many others [99, 253,266,312,313].”
Learning to Schedule Communication in Multi-agent Reinforcement Learning (2019, methodology): “To coordinate this exchange of messages, we adopt the centralized training and distributed execution paradigm popularized in recent works, e.g. , Foerster et al. (2017a); Lowe et al. (2017); Sunehag et al. (2018); Rashid et al. (2018); Gupta et al. (2017).”