Sample Efficient Policy Search for Optimal Stopping Domains
By Karan Goel, Christoph Dann, Emma Brunskill. 2017
9 citations recorded by the source corpus.
Optimal stopping problems consider the question of deciding when to stop an observation-generating process in order to maximize a return. We examine the problem of simultaneously learning and planning in such domains, when data is collected directly from the environment. We propose GFSE, a simple and flexible model-free policy search method that reuses data for sample efficiency by leveraging problem structure. We bound the sample complexity of our approach to guarantee uniform convergence of policy value estimates, tightening existing PAC bounds to achieve logarithmic dependence on horizon length for our setting. We also examine the benefit of our method against prevalent model-based and model-free approaches on 3 domains taken from diverse fields.
Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Stefan Wager
- Xinkun Nie
- Emma Brunskill
- Susan Athey
- Lihong Li
- Ziyang Tang
- Nathan Kallus
- Sham M. Kakade
In Computer Networks and Communications
In Management Science and Operations Research
- Stefan Wager
- Xinkun Nie
- Emma Brunskill
- Susan Athey
- Nathan Kallus
- Ian Osband
- Brendan O'Donoghue
- Lihong Li
Citations in the corpus
- Learning When-to-Treat Policies (2019, methodology): “Finally, in the optimal stopping literature (e.g., Van Moerbeke 1976; Jacka 1991; Mordecki 2002; Goel, Dann, and Brunskill 2017), the treatment choices are binary, and the goal is to optimize for a policy for when to start or stop a treatment.”