Planning, Learning and Coordination in Multiagent Decision Processes

By Craig Boutilier. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Multi-agent reinforcement learning as a rehearsal for decentralized planning (2016, neutral): “Much of the prior work in MARL addressed fully observable settings where states and joint actions are publicly observed, and the key problem is coordination among the agents [7].”
QPLEX: Duplex Dueling Multi-Agent Q-Learning (2020, methodology): “In the two-state MMDP, Optimal line of Figure 3b is approximately ∑99 i=0 γ i = 63.4 in one episode of 100 timesteps.”
Optimal and Approximate Q-value Functions for Decentralized POMDPs (2008, background): “In such a case, a Dec-POMDP effectively reduces to a multiagent MDP (MMDP) as described by Boutilier (1996).”
Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems (2012, background): “This equilibria selection mechanism is a social convention (Boutilier, 1996) which places constraints on the possible action choices of the agents.”
PettingZoo: Gym for Multi-Agent Reinforcement Learning (2020, unknown): “This model is very similar to, and strictly more general than, multi-agent MDPs [Boutilier, 1996], Dec-POMDPs [Bernstein et al., 2002], and Stochastic (“Markov”) games [Shapley, 1953]).”
Collaborative Multiagent Reinforcement Learning by Payoff Propagation (2006, unknown): “This approach differs from other multiagent models, for example, multiagent MDPs (Boutilier, 1996) or decentralized MDPs (Bernstein et al., 2000), in which all agents observe the global reward.”
Graphical Model Inference in Optimal Control of Stochastic Multi-Agent Systems (2008, background): “The usual approach to control in multi-agent systems with stochastic dynamics is to model the system by a Markov Decision Processes (MDP) (Boutilier, 1996; Sadati & Elhamifar, 2006). In principle, these are solved in discrete space and time by backward dynamic programming. However, the discretizatio”
Exploiting Agent and Type Independence in Collaborative Graphical Bayesian Games (2011, background): “centralized algorithm, e.g., when there are multiple maxima. Ensuring that the agents select the same PONE can be accomplished by imposing certain social conventions or through repeated interactions (Boutilier, 1996). In this article, we assume that the game is solved in an off-line centralized plan”
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, background): “When cooperative teams of agents are planning in uncertain domains, they must coordinate to maximise their (joint) team value, in this scenario the multiagent Markov decision processes (MMDPs) (Boutilier, 1996) are useful.”
Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning (2019, background): “We focused on the cooperative setting [24]; however, unlike the fully-cooperative setting, in which all agents optimise a shared reward function [2] our approach introduces a diversity of rewards, which can help with credit-assignment [4, 35] but also introduces elements of competition.”
Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning (2020, background): “In multi-agent environments, Multi-agent MDPs (“MMDPs”) (Boutilier, 1996) extend MDPs by allowing for a set of actions to accommodate multiple agents.”
The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems (1998, background): “Many of our conclusions hold mutatis mutandis for sequential, multiagent Markov decision processes [2] with multiple states; but From: AAAI-98 Proceedings.”
Utile Coordination: Learning Interdependencies Among Cooperative Agents (2005, methodology): “If this framework is constrained such that each agent receives the same reward, it corresponds exactly to the MMDP (multiagent MDP) framework of [Boutilier, 1996].”
Cooperative Multi-Agent Learning: The State of the Art (2005, unknown): “Boutilier [24] describes multi-agent Markov”
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning (2017, background): “When there are several joint actions with the same value, coordination mechanisms are needed to make sure that all agents play according to the same strategy [4].”
Decentralized POMDPs (2012, background): “These range from fully- or individually observable as in a multi-agent MDP (Boutilier, 1996) to non-observable.”
Learning multi-agent state space representations (2010, unknown): “In a special case of the general Markov game framework, the socalled team games or multi-agent MDPs (MMDPs) optimal policies still exist [1] [2].”