Black-Box Policy Search with Probabilistic Programs
By Jan-Willem van de Meent, Brooks Paige, David Tolpin, Frank D. Wood. 2015
25 citations recorded by the source corpus.
In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.
Subfields: Artificial Intelligence, Reinforcement Learning, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Francisco J. R. Ruiz
- Michalis K. Titsias
- David M. Blei
- Hao Cui
- Roni Khardon
- Qiang Liu
- Stephan Mandt
- Rajesh Ranganath
In Reinforcement Learning
In Probabilistic & Bayesian Methods
- Francisco J. R. Ruiz
- Michalis K. Titsias
- David M. Blei
- Hao Cui
- Roni Khardon
- Qiang Liu
- Rajesh Ranganath
- Richard E. Turner
Citations in the corpus
- The Generalized Reparameterization Gradient (2016, background): “…Paisley et al., 2012; Ranganath et al., 2014; Titsias and Lázaro-Gredilla, 2014), deep neural networks (Neal, 1992; Hin-ton et al., 1995; Mnih and Gregor, 2014; Kingma and Welling, 2014), and probabilistic programming (Wingate and Weber, 2013; Kucukelbir et al., 2015; van de Meent et al., 2016).”
- Lifted Stochastic Planning, Belief Propagation and Marginal MAP (2018, methodology): “…2011; Sabbadin, Peyrard, and Forsell 2012; Liu and Ihler 2012; Cheng et al. 2013; Kiselev and Poupart 2014; Nitti, Belle, and Raedt 2015; van de Meent et al. 2016; Lee, Mari-nescau, and Dechter 2016; Nguyen, Kumar, and Lau 2017) with applications in robotics, scheduling and environmental…”