Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach

By Roel Dobbe, David Fridovich-Keil, Claire Jennifer Tomlin. 2017

45 citations recorded by the source corpus.

Learning cooperative policies for multi-agent systems is often challenged by partial observability and a lack of coordination. In some settings, the structure of a problem allows a distributed solution with limited communication. Here, we consider a scenario where no communication is available, and instead we learn local policies for all agents that collectively mimic the solution to a centralized multi-agent static optimization problem. Our main contribution is an information theoretic framework based on rate distortion theory which facilitates analysis of how well the resulting fully decentralized policies are able to reconstruct the optimal solution. Moreover, this framework provides a natural extension that addresses which nodes an agent should communicate with to improve the performance of its individual policy.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

D2: Decentralized Training over Decentralized Data (2018, methodology): “An information theoretic approach was used to analyze decentralization in Dobbe et al. [2017].”
Communication Compression for Decentralized Training (2018, background): “Dobbe et al. [2017] applied an information theoretic framework for decentralize analysis.”
Tackling Climate Change with Machine Learning (2019, methodology): “One strategy is to explicitly design local control algorithms; for instance, recent work has controlled energy storage and solar inverters using supervised learning techniques trained on historical optimization data [70–73].”
CESMA: Centralized Expert Supervises Multi-Agents (2019, background): “In (Dobbe et al., 2017), they examine decentralization of policies from an information-theoretic perspective.”
Who2com: Collaborative Perception via Learnable Handshake Communication (2020, background): “Similar advantages have also been shown using rate distortion theory [8].”
Multi-agent deep reinforcement learning: a survey (2021, background): “In further works, DTDE has been applied to cooperative navigation tasks (Chen et al. 2016; Strouse et al. 2018), to partially observable domains (Dobbe et al. 2017; Nguyen et al. 2017b; Srinivasan et al. 2018), and to social dilemmas (Leibo et al. 2017).”
A Broader View on Bias in Automated Decision-Making: Reflecting on Epistemology and Dynamics (2018, background): “From an information theoretic perspective, modeling bias inﬂuences the extent to which distortion can be minimized when reconstructing a phenomenon from a compressed or sampled version of the original (Cover & Thomas, 2012; Dobbe et al., 2017).”