ML for Health

Improving Quality and Efficiency of Clinical Processes by Learning from Big Health Data

Program: Scientific and Technological Research Council of Turkey (TÜBİTAK) 1003 Program

Principal Investigator: Cem Tekin

Graduate Student Contributors: Ahmet Alparslan Çelik, Andi Nika, Kerem Bozgan, İlker Demirel

Undergraduate Student Contributors: Alihan Hüyük, Sepehr Elahi, Mehmet Ufuk Özdemir

Project Summary:

Big Health Data (BHD) has a great potential to support clinicians in diagnosis, treatment administration, and patient monitoring. However, BHD includes a vast amount of documents, test results, and medical images related to the patient's past treatments. Leveraging this information requires developing sophisticated statistical learning techniques for diagnosis and treatment recommendation and risk management. This project aims to design trustable machine learning algorithms that will learn to make optimal decisions using BHD and integrate them into new decision support systems that clinicians and patients will use.

In particular, this project focuses on two critical questions: (i) Given a patient with a specific condition, how to extract relevant information from large clinical records that doctors find essential for diagnosis or treatment. (ii) How to optimize treatments over time while minimizing risk to a particular patient.

Combinatorial Bandits for Health Report Recommendations

Electronic Health Record (EHR) contains a vast amount of information about a patient. Due to time constraints, it is not efficient for a clinician to scan the entire record during the visit. IHR aims to show only a small but relevant (to the current visit) subset of the patient record to the clinician in real-time. The clinician can rate each displayed item's relevance, and a learning engine uses this feedback to improve the displayed items over time.

We tackle this problem by building on tools from combinatorial learning and optimization theory. In particular, we propose very general combinatorial sequential decision-making models that can characterize different forms (binary rating, real-valued feedback) and style of feedback (cascading feedback, random feedback) that the doctors can provide to the displayed items.

Relevant items are identified solely by interacting with the user, making our system fully adaptive and personalized. Our learning algorithms are built based on the posterior sampling and optimism in the face of uncertainty principles. Our general mathematical formalism applies to a wide range of other combinatorial learning problems. These include web-based recommender systems, online influence maximization, and cognitive communications.

Related publications:

I. Demirel, C. Tekin, "Combinatorial Gaussian process bandits with probabilistically triggered arms", Proc. 24th International Conference on Artificial Intelligence and Statistics (AISTATS), April 2021.

A. Huyuk, C. Tekin, "Thompson sampling for combinatorial network optimization in unknown environments", IEEE/ACM Transactions on Networking, 28(6): 2836-2849, December 2020.

A. Nika, S. Elahi, C. Tekin, "Contextual combinatorial volatile multi-armed bandit with adaptive discretization", in Proc. 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), August 2020.

A. Huyuk, C. Tekin, "Analysis of Thompson sampling for combinatorial multi-armed bandit with probabilistically triggered arms", in Proc. 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), April 2019.

Optimizing Treatments with Contextual Gaussian Process Bandits

Treatment of chronic diseases requires long term commitment and adaptation to ever evolving conditions. As there is no one-size-fits-all approach, treatments must be adapted based on changing patient characteristics. Within this context, there has been a surge of interest in using machine learning techniques for identifying optimal personalized treatment regimes.

Since management of chronic diseases requires repeatedly making decisions, more data about the patient’s response is accumulated over time. Moreover, data collected from the patient depends on the course of the treatment. Therefore, supervised learning methods---which require offline training data---are unable to produce accurate recommendations in the long run. As the patient characteristics evolve over time, treatment must be adjusted to maximize the benefit while minimizing the risks. This requires an intricate balance between exploration and exploitation. The best treatment under the current context must be identified with sequential experimentation while ensuring safety and efficiency at the same time.

In this work, we model optimization of dynamic treatment regimes as a volatile contextual Gaussian process (GP) bandit. The predictive power and non-parametric flexibility of GPs allow us to accurately model the relationship between treatment and response under different patient contexts. Moreover, sequential experimentation via upper confidence bounds constructed using the posterior mean and covariance functions of the GP allows us perform safe experimentation over a set of admissible treatments calculated based on the current context. Our framework allows flexibility in forming the set of admissible treatments, and enables safe experimentation among a set of treatments identified by clinical guidelines or baseline interpretable formula-based systems. In particular, we focus on using our framework for personalized bolus insulin dose recommendation for type 1 diabetes mellitus (T1DM) patients.

Related publications:

I. Demirel, A. A. Celik, C. Tekin, "ESCADA: Efficient safety and context aware dose allocation for precision medicine".

A. A. Celik, C. Tekin, "Optimizing dynamic treatment regimes via volatile contextual Gaussian process bandits ", ICML 2021 - Reinforcement Learning for Real Life Workshop, July 2021.

I. Demirel, M. U. Ozdemir, C. Tekin, "Safe linear leveling bandits".