Solving Partially Observable Stochastic Games with Public Observations

By Karel Horák, Branislav Bosanský. 2019

54 citations recorded by the source corpus.

In many real-world problems, there is a dynamic interaction between competitive agents. Partially observable stochastic games (POSGs) are among the most general formal models that capture such dynamic scenarios. The model captures stochastic events, partial information of players about the environment, and the scenario does not have a fixed horizon. Solving POSGs in the most general setting is intractable.Therefore, the research has been focused on subclasses of POSGs that have a value of the game and admit designing (approximate) optimal algorithms. We propose such a subclass for two-player zero-sum games with discounted-sum objective function—POSGs with public observations (POPOSGs)—where each player is able to reconstruct beliefs of the other player over the unobserved states. Our results include: (1) theoretical analysis of PO-POSGs and their value functions showing convexity (concavity) in beliefs of maximizing (minimizing) player, (2) a novel algorithm for approximating the value of the game, and (3) a practical demonstration of scalability of our algorithm. Experimental results show that our algorithm can closely approximate the value of non-trivial games with hundreds of states.

Subfields: General Decision Sciences, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Decision Sciences

In Management Science and Operations Research

Citations in the corpus

Rethinking Formal Models of Partially Observable Multiagent Decision Making (2019, background): “There are many formalizations and variants of POSGs in the literature (Peshkin et al., 2000; Hansen et al., 2004b; Emery-Montemerlo et al., 2004; Horak and Bosansky, 2019).”