On-line policy learning and adaptation for real-time personalization of an artificial pancreas

By Mariano De Paula, Gerardo Gabriel Acosta, Ernesto C. Martínez. 2015

21 citations recorded by the source corpus.

The importance of tailoring an artificial pancreas to a given patient is addressed.On-line policy learning integrates reinforcement learning with Gaussian processes.Only relevant data is used to update the control policy.Fast policy adaptation allows dealing with patient-specific glycemic variability. The dynamic complexity of the glucose-insulin metabolism in diabetic patients is the main obstacle towards widespread use of an artificial pancreas. The significant level of subject-specific glycemic variability requires continuously adapting the control policy to successfully face daily changes in patient's metabolism and lifestyle. In this paper, an on-line selective reinforcement learning algorithm that enables real-time adaptation of a control policy based on ongoing interactions with the patient so as to tailor the artificial pancreas is proposed. Adaptation includes two online procedures: on-line sparsification and parameter updating of the Gaussian process used to approximate the control policy. With the proposed sparsification method, the support data dictionary for on-line learning is modified by checking if in the arriving data stream there exists novel information to be added to the dictionary in order to personalize the policy. Results obtained in silico experiments demonstrate that on-line policy learning is both safe and efficient for maintaining blood glucose variability within the normoglycemic range.

Subfields: Control and Systems Engineering, Endocrinology, Diabetes and Metabolism, Genetics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Endocrinology, Diabetes and Metabolism

In Genetics

Citations in the corpus

Model-Free Machine Learning in Biomedicine: Feasibility Study in Type 1 Diabetes (2016, methodology): “An online policy learning algorithm was presented in [34] and performed efficiently.”
Deep Reinforcement Learning for Closed-Loop Blood Glucose Control (2020, methodology): “De Paula et al. (2015) use a policy-iteration framework with Gaussian process approximation and Bayesian active learning.”