Faster Rates for Policy Learning
By Alexander Luedtke, Antoine Chambaz. 2017
10 citations recorded by the source corpus.
This article improves the existing proven rates of regret decay in optimal policy estimation. We give a margin-free result showing that the regret decay for estimating a within-class optimal policy is second-order for empirical risk minimizers over Donsker classes, with regret decaying at a faster rate than the standard error of an efficient estimator of the value of an optimal policy. We also give a result from the classification literature that shows that faster regret decay is possible via plug-in estimation provided a margin condition holds. Four examples are considered. In these examples, the regret is expressed in terms of either the mean value or the median value; the number of possible actions is either two or finitely many; and the sampling scheme is either independent and identically distributed or sequential, where the latter represents a contextual bandit sampling scheme.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Xinkun Nie
- Emma Brunskill
- Susan Athey
- Nathan Kallus
- Stefan Wager
- Edward H. Kennedy
- Lihong Li
- Ofir Nachum
Citations in the corpus
- Learning When-to-Treat Policies (2019, background): “In regimes where treatment effects stay constant when the sample size grows, super-efficiency phenomena are unavoidable; see Luedtke and Chambaz (2017) for a formal statement with static decision rules.”
- Efficient Policy Learning (2017, background): “Conversely, if treatment effects dominate the 1/ √ n scale, then in large samples it is all but obvious who should be treated and who should not, and it is possible to get regret bounds that decay at superefficient rates (Luedtke and Chambaz, 2017), again making Theorem 1 loose.”