Markovian State and Action Abstractions for MDPs via Hierarchical MCTS

65 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Learning Compositional Neural Programs with Recursive Tree Search and Planning (2019, background): “While AlphaZero does not use hierarchies or recursion, hierarchical MCTS algorithms have been previously proposed for simple hierarchical RL domains [Vien and Toussaint, 2015, Bai et al., 2016]. The current work capitalizes on advances brought in by deep reinforcement learning as well as design choi”
Learning Markov State Abstractions for Deep Reinforcement Learning (2021, background): “This is especially unsatisfying, not just because learning is challenging in POMDPs (Zhang et al., 2012), but because formulating the problem as a Block MDP was supposed to avoid precisely this type of partial observability: an abstraction exists (namely φ = σ − 1 ) that would result in a fully obse”
Spatio-Temporal Abstractions in Reinforcement Learning Through Neural Encoding (2016, background): “First, the action space A is not modified and therefore the planning horizon remains intractable, and second, AMDPs are not necessarily Markovian (Bai et al., 2016).”
Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning (2022, background): “A large body of work has emerged that studies learning abstractions for planning [12, 13, 14], but with a few notable exceptions [15, 16, 17], these efforts assume that the learned abstractions should be “all you need.””