Compositional Q-learning for electrolyte repletion with imbalanced patient sub-populations

• URL: http://arxiv.org/abs/2110.02879v2
• Date: Sun, 11 Feb 2024 00:21:07 GMT
• Title: Compositional Q-learning for electrolyte repletion with imbalanced patient sub-populations
• Authors: Aishwarya Mandyam, Andrew Jones, Jiayu Yao, Krzysztof Laudanski, Barbara Engelhardt
• Abstract summary: We introduce Compositional Fitted $Q$-iteration (CFQI) to represent heterogeneous treatment responses in medical care settings. CFQI uses a compositional $Q$-value function with separate modules for each task variant, allowing it to take advantage of shared knowledge. We validate CFQI's performance using a Cartpole environment and use CFQI to recommend electrolyte repletion for patients with and without renal disease.
• Score: 2.9271624996210313
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) is an effective framework for solving sequential decision-making tasks. However, applying RL methods in medical care settings is challenging in part due to heterogeneity in treatment response among patients. Some patients can be treated with standard protocols whereas others, such as those with chronic diseases, need personalized treatment planning. Traditional RL methods often fail to account for this heterogeneity, because they assume that all patients respond to the treatment in the same way (i.e., transition dynamics are shared). We introduce Compositional Fitted $Q$-iteration (CFQI), which uses a compositional task structure to represent heterogeneous treatment responses in medical care settings. A compositional task consists of several variations of the same task, each progressing in difficulty; solving simpler variants of the task can enable efficient solving of harder variants. CFQI uses a compositional $Q$-value function with separate modules for each task variant, allowing it to take advantage of shared knowledge while learning distinct policies for each variant. We validate CFQI's performance using a Cartpole environment and use CFQI to recommend electrolyte repletion for patients with and without renal disease. Our results demonstrate that CFQI is robust even in the presence of class imbalance, enabling effective information usage across patient sub-populations. CFQI exhibits great promise for clinical applications in scenarios characterized by known compositional structures.

Related papers

• Doubly Robust Fusion of Many Treatments for Policy Learning [7.943530359935207]
  We propose a calibration-weighted treatment fusion procedure that robustly balances covariates across treatment groups.<n>We establish theoretical guarantees, including consistency, the oracle property of treatment fusion, and regret bounds.<n>We illustrate the practical utility of our method using a nationwide electronic health record-derived de-identified database.
  arXiv  Detail & Related papers  (2025-05-12T21:55:57Z)
• Uncovering Population PK Covariates from VAE-Generated Latent Spaces [0.24578723416255746]
  We propose a data-driven, model-free framework that integrates Variational Autoencoders (VAEs) deep learning model and LASSO regression.<n>VAE compresses high-dimensional PK signals into a structured latent space, achieving accurate reconstruction with a mean absolute percentage error (MAPE) of 2.26%.
  arXiv  Detail & Related papers  (2025-05-05T09:47:39Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions. We propose a novel approach utilizing structured medical reasoning. Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework [2.5070297884580874]
  This study introduces ConformalDQN, a distribution-free conformal deep Q-learning approach for optimizing mechanical ventilation in intensive care units. We trained and evaluated our model using ICU patient records from the MIMIC-IV database.
  arXiv  Detail & Related papers  (2024-12-17T06:55:20Z)
• Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithm [0.7519918949973486]
  This study proposes a reinforcement learning-based personalized optimal heparin dosing policy. A batch-constrained policy was implemented to minimize out-of-distribution errors in an offline RL environment. This research enhances heparin administration practices and establishes a precedent for the development of sophisticated decision-support tools in medicine.
  arXiv  Detail & Related papers  (2024-09-24T05:20:38Z)
• UniFed: A Universal Federation of a Mixture of Highly Heterogeneous Medical Image Classification Tasks [5.563171090433323]
  We introduce UniFed, a universal federated learning paradigm that aims to classify any disease from any imaging modality. Specifically, by dynamically adjusting both local and global models, UniFed considers the varying task complexities of clients and the server. We demonstrate the superiority of our framework in terms of accuracy, communication cost, and convergence time over relevant benchmarks in diagnosing retina, histopathology, and liver tumour diseases.
  arXiv  Detail & Related papers  (2024-07-29T23:15:15Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Machine learning-based patient selection in an emergency department [0.0]
  This paper investigates the potential of an Machine Learning (ML) based patient selection method. It incorporates a comprehensive state representation of the system and a complex non-linear selection function. Results show that the proposed method significantly outperforms the APQ method for a majority of evaluated settings.
  arXiv  Detail & Related papers  (2022-06-08T08:56:52Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation [81.72197368690031]
  We present a new benchmarking suite designed specifically for medical sequential decision making. The Medkit-Learn(ing) Environment is a publicly available Python package providing simple and easy access to high-fidelity synthetic medical data.
  arXiv  Detail & Related papers  (2021-06-08T10:38:09Z)
• COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching [70.08786840301435]
  We propose CrOss-Modal PseudO-SiamEse network (COMPOSE) to address these challenges for patient-trial matching. Experiment results show COMPOSE can reach 98.0% AUC on patient-criteria matching and 83.7% accuracy on patient-trial matching.
  arXiv  Detail & Related papers  (2020-06-15T21:01:33Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)
• Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale [0.5498849973527224]
  We present an unsupervised framework based on deep learning to process heterogeneous EHRs. We derive patient representations that can efficiently and effectively enable patient stratification at scale.
  arXiv  Detail & Related papers  (2020-03-14T00:04:20Z)
• Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations [114.16762407465427]
  We introduce the Counterfactual Recurrent Network (CRN) to estimate treatment effects over time. CRN uses domain adversarial training to build balancing representations of the patient history. We show how our model achieves lower error in estimating counterfactuals and in choosing the correct treatment and timing of treatment.
  arXiv  Detail & Related papers  (2020-02-10T20:47:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.