Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments

By Yan Zheng, Jianye Hao, Zongzhang Zhang. 2018

38 citations recorded by the source corpus.

Despite single agent deep reinforcement learning has achieved significant success due to the experience replay mechanism, Concerns should be reconsidered in multiagent environments. This work focus on the stochastic cooperative environment. We apply a specific adaptation to one recently proposed weighted double estimator and propose a multiagent deep reinforcement learning framework, named Weighted Double Deep Q-Network (WDDQN). To achieve efficient cooperation, \textit{Lenient Reward Network} and \textit{Mixture Replay Strategy} are introduced. By utilizing the deep neural network and the weighted double estimator, WDDQN can not only reduce the bias effectively but also be extended to many deep RL scenarios with only raw pixel images as input. Empirically, the WDDQN outperforms the existing DRL algorithm (double DQN) and multiagent RL algorithm (lenient Q-learning) in terms of performance and convergence within stochastic cooperative environments.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computational Theory and Mathematics

Citations in the corpus

Negative Update Intervals in Deep Multi-Agent Reinforcement Learning (2018, methodology): “[31] introduce a Weighted Double-DQN that makes use of a lenient reward network alongwith a scheduled replay strategy to improve the convergence rate within stochastic cooperative environments.”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “er (i.e., keeping hs;a;r;s0i) will probably fail. Adding information in the experience tuple that can help disambiguate the sample, is the solution adopted in many works, whether a value based method [62, 63, 64, 65] or a policy gradient method [71]. In this regard, it is an open question to conside”
Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning (2020, background): “2017), multiagent system (Zheng et al. 2018b; 2018a), video game playing (Mnih et al.”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, background): “Very recently, single-objective multi-agent RL has received considerable attention as well [30,32,39,55,97,81,109, 130].”
Interleaved Q-Learning with Partially Coupled Training Process (2019, unknown): “The original double Q-learning also inspires several improvements, such as deep double DQN [23], double delayed Q-learning [1] and weighted multi-agent deep double Q-learning [27].”