Online Decision Making with High-Dimensional Covariates

By Hamsa Bastani, Mohsen Bayati. 2020

433 citations recorded by the source corpus.

Big data has enabled decision-makers to tailor decisions at the individual-level in a variety of domains such as personalized medicine and online advertising. This involves learning a model of decision rewards conditional on individual-specific covariates. In many practical settings, these covariates are high-dimensional; however, typically only a small subset of the observed features are predictive of a decision's success. We formulate this problem as a multi-armed bandit with high-dimensional covariates, and present a new efficient bandit algorithm based on the LASSO estimator. Our regret analysis establishes that our algorithm achieves near-optimal performance in comparison to an oracle that knows all the problem parameters. The key step in our analysis is proving a new oracle inequality that guarantees the convergence of the LASSO estimator despite the non-i.i.d. data induced by the bandit policy. Furthermore, we illustrate the practical relevance of our algorithm by evaluating it on a real-world clinical problem of warfarin dosing. A patient's optimal warfarin dosage depends on the patient's genetic profile and medical records; incorrect initial dosage may result in adverse consequences such as stroke or bleeding. We show that our algorithm outperforms existing bandit methods as well as physicians to correctly dose a majority of patients.

Subfields: Artificial Intelligence, Management Science and Operations Research, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Susan Athey
  2. Stefan Wager
  3. Nathan Kallus
  4. Jack Dunn
  5. Guido W. Imbens
  6. Angela Zhou
  7. Osbert Bastani
  8. Carolyn Kim

In Management Science and Operations Research

  1. Susan Athey
  2. Stefan Wager
  3. Nathan Kallus
  4. Guido W. Imbens
  5. Zhengyuan Zhou
  6. Maria Dimakopoulou
  7. Angela Zhou
  8. Jack Dunn

In Game & Decision Theory

  1. Carolyn Kim
  2. Jack Dunn
  3. Osbert Bastani
  4. Susan Athey
  5. Nathan Kallus
  6. Guido W. Imbens
  7. Zhengyuan Zhou
  8. Maria Dimakopoulou

Citations in the corpus