Policy Learning With Observational Data

By Susan Athey, Stefan Wager. 2017

470 citations recorded by the source corpus.

In many areas, practitioners seek to use observational data to learn a treatment assignment policy that satisfies application‐specific constraints, such as budget, fairness, simplicity, or other functional form constraints. For example, policies may be restricted to take the form of decision trees based on a limited set of easily observable individual characteristics. We propose a new approach to this problem motivated by the theory of semiparametrically efficient estimation. Our method can be used to optimize either binary treatments or infinitesimal nudges to continuous treatments, and can leverage observational data where causal effects are identified using a variety of strategies, including selection on observables and instrumental variables. Given a doubly robust estimator of the causal effect of assigning everyone to treatment, we develop an algorithm for choosing whom to treat, and establish strong guarantees for the asymptotic utilitarian regret of the resulting policy.

Subfields: Statistics and Probability, Reinforcement Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Stefan Wager
2. Xinkun Nie
3. Kevin Guo
4. Jacob Dorn
5. Emma Brunskill
6. Susan Athey
7. Edward H. Kennedy
8. Mihaela van der Schaar

In Reinforcement Learning

1. Xinkun Nie
2. Stefan Wager
3. Emma Brunskill
4. Kevin Guo
5. Jacob Dorn
6. Nathan Kallus
7. Ziyang Tang
8. Lihong Li

In Causal Inference & Discovery

1. Stefan Wager
2. Xinkun Nie
3. Emma Brunskill
4. Kevin Guo
5. Jacob Dorn
6. Susan Athey
7. Edward H. Kennedy
8. Mihaela van der Schaar

Citations in the corpus

• Learning When-to-Treat Policies (2019, methodology): “In the static setting, a popular approach to policy learning starts by first providing an estimator V̂π for the value Vπ of each feasible policy π ∈ , and then sets π̂ = argmax{V̂π : π ∈ } (e.g., Manski 2004; Zhao et al. 2012; Swaminathan and Joachims 2015; Kitagawa and Tetenov 2018; Athey and Wager”
• Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing (2021, methodology): “With minor modification, our identification results can also be applied to more complex estimands, including policy values [3, 27] and weighted average treatment effects.”