Reinforcement Learning With Task Decomposition for Cooperative Multiagent Systems

By Changyin Sun, Wenzhang Liu, Lu Dong. 2020

89 citations recorded by the source corpus.

In this article, we study cooperative multiagent systems (MASs) with multiple tasks by using reinforcement learning (RL)-based algorithms. The target for a single-agent RL system is represented by its scalar reward signals. However, for an MAS with multiple cooperative tasks, the holistic reward signal consists of multiple parts to represent the tasks, which makes the problem complicated. Existing multiagent RL algorithms search distributed policies with holistic reward signals directly, making it difficult to obtain an optimal policy for each task. This article provides efficient learning-based algorithms such that each agent can learn a joint optimal policy to accomplish these multiple tasks cooperatively with other agents. The main idea of the algorithms is to decompose the holistic reward signal for each agent into multiple parts according to the subtasks, and then the proposed algorithms learn multiple value functions with the decomposed reward signals and update the policy with the sum of distributed value functions. In addition, this article presents a theoretical analysis of the proposed approach. Finally, the simulation results for both discrete decision-making and continuous control problems have demonstrated the effectiveness of the proposed algorithms.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chongjie Zhang
2. Tonghan Wang
3. Bei Peng
4. Shimon Whiteson
5. Anuj Mahajan
6. Tarun Gupta
7. Christian Schröder de Witt
8. J. Urgen Schmidhuber

In Computational Theory and Mathematics

1. Chongjie Zhang
2. Tonghan Wang
3. Anuj Mahajan
4. Bei Peng
5. Shimon Whiteson
6. Zongqing Lu
7. Jiechuan Jiang
8. Fernando Fernández

In Computer Networks and Communications

1. Tonghan Wang
2. Shimon Whiteson
3. Bei Peng
4. Chongjie Zhang
5. Aadirupa Saha
6. Masrour Zoghi

Citations in the corpus

• RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “However, roles and the associated responsibilities (or subtask-specific rewards (Sun et al., 2020)) are prede-fined using prior knowledge in these systems (Lhaksmana et al., 2018).”