Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithm

• URL: http://arxiv.org/abs/2409.15753v1
• Date: Tue, 24 Sep 2024 05:20:38 GMT
• Title: Development and Validation of Heparin Dosing Policies Using an Offline Reinforcement Learning Algorithm
• Authors: Yooseok Lim, Inbeom Park, Sujee Lee,
• Abstract summary: This study proposes a reinforcement learning-based personalized optimal heparin dosing policy. A batch-constrained policy was implemented to minimize out-of-distribution errors in an offline RL environment. This research enhances heparin administration practices and establishes a precedent for the development of sophisticated decision-support tools in medicine.
• Score: 0.7519918949973486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Appropriate medication dosages in the intensive care unit (ICU) are critical for patient survival. Heparin, used to treat thrombosis and inhibit blood clotting in the ICU, requires careful administration due to its complexity and sensitivity to various factors, including patient clinical characteristics, underlying medical conditions, and potential drug interactions. Incorrect dosing can lead to severe complications such as strokes or excessive bleeding. To address these challenges, this study proposes a reinforcement learning (RL)-based personalized optimal heparin dosing policy that guides dosing decisions reliably within the therapeutic range based on individual patient conditions. A batch-constrained policy was implemented to minimize out-of-distribution errors in an offline RL environment and effectively integrate RL with existing clinician policies. The policy's effectiveness was evaluated using weighted importance sampling, an off-policy evaluation method, and the relationship between state representations and Q-values was explored using t-SNE. Both quantitative and qualitative analyses were conducted using the Medical Information Mart for Intensive Care III (MIMIC-III) database, demonstrating the efficacy of the proposed RL-based medication policy. Leveraging advanced machine learning techniques and extensive clinical data, this research enhances heparin administration practices and establishes a precedent for the development of sophisticated decision-support tools in medicine.

Related papers

• OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatment [0.4998632546280975]
  This study focuses on developing a reward function that reflects the clinician's intentions. We learn a parameterized reward function that includes the expert's intentions from limited data. This approach can be broadly utilized not only for the heparin dosing problem but also for RL-based medication dosing tasks in general.
  arXiv  Detail & Related papers  (2024-09-20T07:51:37Z)
• Safe and Interpretable Estimation of Optimal Treatment Regimes [54.257304443780434]
  We operationalize a safe and interpretable framework to identify optimal treatment regimes. Our findings support personalized treatment strategies based on a patient's medical history and pharmacological features.
  arXiv  Detail & Related papers  (2023-10-23T19:59:10Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [46.2482873419289]
  We introduce a deep Q-learning approach to obtain more reliable critical care policies. We evaluate our method in off-policy and offline settings using simulated environments and real health records from intensive care units.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control [11.07500653906137]
  Policy Constraint Q-Learning (PCQL) is a data-driven reinforcement learning algorithm for solving the problem of learning anesthesia strategies on real clinical datasets. PCQL is validated by extensive experiments on a real clinical anesthesia dataset.
  arXiv  Detail & Related papers  (2023-03-17T10:05:20Z)
• Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients [0.0]
  We propose a clinically motivated control objective for critically ill patients, for which the value functions have a simple medical interpretation. We experiment on a large cohort and show that our method produces results consistent with clinical knowledge.
  arXiv  Detail & Related papers  (2022-07-17T00:06:09Z)
• Optimal discharge of patients from intensive care via a data-driven policy learning framework [58.720142291102135]
  It is important that the patient discharge task addresses the nuanced trade-off between decreasing a patient's length of stay and the risk of readmission or even death following the discharge decision. This work introduces an end-to-end general framework for capturing this trade-off to recommend optimal discharge timing decisions. A data-driven approach is used to derive a parsimonious, discrete state space representation that captures a patient's physiological condition.
  arXiv  Detail & Related papers  (2021-12-17T04:39:33Z)
• Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost Functions [80.12620331438052]
  deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features. Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets. We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
  arXiv  Detail & Related papers  (2020-06-25T08:46:37Z)
• Reinforcement learning and Bayesian data assimilation for model-informed precision dosing in oncology [0.0]
  Current strategies comprise model-informed dosing tables or are based on maximum a-posteriori estimates. We propose three novel approaches for MIPD employing Bayesian data assimilation and/or reinforcement learning to control neutropenia. These approaches have the potential to substantially reduce the incidence of life-threatening grade 4 and subtherapeutic grade 0 neutropenia.
  arXiv  Detail & Related papers  (2020-06-01T16:38:27Z)
• Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation [2.908482270923597]
  Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics.
  arXiv  Detail & Related papers  (2020-03-13T20:31:47Z)
• Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions [48.91284724066349]
  Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education. Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding. We develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of policy evaluation estimates.
  arXiv  Detail & Related papers  (2020-02-10T00:26:43Z)
• Contextual Constrained Learning for Dose-Finding Clinical Trials [102.8283665750281]
  C3T-Budget is a contextual constrained clinical trial algorithm for dose-finding under both budget and safety constraints. It recruits patients with consideration of the remaining budget, the remaining time, and the characteristics of each group.
  arXiv  Detail & Related papers  (2020-01-08T11:46:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.