Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions
By Feiyun Zhu, Jun Guo, Ruoyu Li, Junzhou Huang. 2018
20 citations recorded by the source corpus.
We consider the actor-critic contextual bandit for the mobile health (mHealth) intervention. State-of-the-art decision-making algorithms generally ignore the outliers in the data-set. In this paper, we propose a novel robust contextual bandit method for the mHealth. It can achieve the conflicting goal of reducing the influence of outliers, while seeking for a similar solution compared with the state-of-the-art contextual bandit methods on the datasets without outliers. Such performance relies on two technologies: (1) the capped-L2 norm; (2) a reliable method to set the threshold hyper-parameter, which is inspired by one of the most fundamental techniques in the statistics. Although the model is non-convex and non-differentiable, we propose an effective reweighted algorithm and provide solid theoretical analyses. We prove that the proposed algorithm can sufficiently decrease the objective function value at each iteration and will converge after a finite number of iterations. Extensive experiment results on two datasets demonstrate that our method can achieve almost identical results compared with state-of-the-art contextual bandit methods on the dataset without outliers, and significantly outperform those state-of-the-art methods on the badly noised dataset with outliers in a variety of parameter settings.
Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Electrical and Electronic Engineering
In Management Science and Operations Research
Citations in the corpus
- Reinforcement Learning in Healthcare: A Survey (2019, methodology): “For example, RL has been used to adapt the intervention strategies of smart phones to recommend regular physical activities to people who suffer from diabetes type 2 [84, 246] or who have experienced a cardiac event and been in cardiac rehab [119, 256, 257].”