Memory-Bounded Dynamic Programming for DEC-POMDPs

By Sven Seuken, Shlomo Zilberstein. 2007

182 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Multi-agent reinforcement learning as a rehearsal for decentralized planning (2016, neutral): “[24,1,10]); however, most of these approaches are centralized and assume the model is known a priori.”
Optimal and Approximate Q-value Functions for Decentralized POMDPs (2008, background): “To counter this growth, Seuken and Zilberstein (2007a) proposed an extension that limits the considered observations during the backup step to the maxObs most likely observations.”
An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs (2014, methodology): “It must also be noted that for this problem, approximate methods like NLP but also other algorithms not depicted here like the “Memory Bound Dynamic Programming” of Seuken and Zilberstein (2007) are able to find the optimal solution.”
Capability Models and Their Applications in Planning (2015, unknown): “Capability models also have connections to macro operators [2], as well as options, their MDP counterparts [21, 20, 9], and the BDI models [17].”
Rollout Sampling Policy Iteration for Decentralized POMDPs (2010, methodology): “Memory-bounded techniques have been developed to combine the top-down heuristics and the bottom-up dynamic programming together, keeping only a bounded number of policies at each iteration [18].”
Scaling Up Optimal Heuristic Search in Dec-POMDPs via Incremental Expansion (2011, methodology): “Optimal solution methods for Dec-POMDPs are of great interest; they are of practical value for smaller or decomposable problems and lie at the basis for most successful approximate methods [EmeryMontemerlo et al., 2004; Seuken and Zilberstein, 2007; Wu et al., 2011].”
A Concise Introduction to Decentralized POMDPs (2016, methodology): “discuss a method called Memory-bounded DP (MBDP), introduced by Seuken and Zilberstein [2007a] , which takes this idea even further.”
Heuristic search for identical payoff Bayesian games (2010, methodology): “Since the number of sub-tree policies that are non-dominated tends to be very large, point-based DP (PBDP) methods [10, 8, 3] have been introduced.”
Planning with macro-actions in decentralized POMDPs (2014, methodology): “Memory-bounded dynamic programming (MBDP) [25] can also be extended to use options as shown in Algorithm 2.”
Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games (2019, unknown): “, [15], [26], [22]).”
Sample Bounded Distributed Reinforcement Learning for Decentralized POMDPs (2012, background): “Even memory bounded techniques (Seuken 2007) maintain multiple policies, although of a bounded total size.”
Point-based incremental pruning heuristic for solving finite-horizon DEC-POMDPs (2009, background): “For the general finite-horizon DEC-POMDPs that we are interested in, MBDP [12] is currently the most successful approximate algorithm.”
Decentralized POMDPs (2012, methodology): “In order to perform the maximization in (17), MBDP loops over the |Qeτ=k| = O(|A†|nMn”
Monte-Carlo Expectation Maximization for Decentralized POMDPs (2013, unknown): “…have been developed to solve DEC- POMDPs optimally [Hansen et al., 2004; Szer et al., 2005; Aras et al., 2007; Spaan et al., 2011] or approximately [Nair et al., 2003; Seuken and Zilberstein, 2007; Bernstein et al., 2009; Wu et al., 2010; Dibangoye et al., 2011; Pajarinen and Peltonen, 2011b].”
Exploiting Agent and Type Independence in Collaborative Graphical Bayesian Games (2011, methodology): “rogramming 14In (Oliehoek et al., 2008c), this problem is referred to as FIREFIGHTINGGRAPH. We use a different name here to distinguish it from GENERALIZED FIRE FIGHTING, which is also graphical. 28 (Seuken and Zilberstein, 2007). In the remainder of this section, we show CGBGs can help the former c”
Incremental Policy Generation for Finite-Horizon DEC-POMDPs (2009, methodology): “These algorithms were tested on the common Box Pushing problem (Seuken and Zilberstein 2007a), the Stochastic Mars Rover problem (Amato and Zilberstein 2009) and a new version of the Meeting in a Grid problem (Bernstein, Hansen, and Zilberstein 2005).”