Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes

• URL: http://arxiv.org/abs/2302.10434v2
• Date: Mon, 17 Feb 2025 13:07:21 GMT
• Title: Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes
• Authors: Di Wang, Yao Wang, Shao-Bo Lin,
• Abstract summary: We present a scalable kernel-based distributed Q-learning algorithm for generating dynamic treatment regimes (DTRs)<n>The results demonstrate that our algorithm significantly reduces the computational complexity associated with the state-of-the-art deep reinforcement learning methods.
• Score: 20.62850899223748
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, large amounts of electronic health records (EHRs) concerning chronic diseases have been collected to facilitate medical diagnosis. Modeling the dynamic properties of EHRs related to chronic diseases can be efficiently done using dynamic treatment regimes (DTRs). While reinforcement learning (RL) is a widely used method for creating DTRs, there is ongoing research in developing RL algorithms that can effectively handle large amounts of data. In this paper, we present a scalable kernel-based distributed Q-learning algorithm for generating DTRs. We perform both theoretical assessments and numerical analysis for the proposed approach. The results demonstrate that our algorithm significantly reduces the computational complexity associated with the state-of-the-art deep reinforcement learning methods, while maintaining comparable generalization performance in terms of accumulated rewards across stages, such as survival time or cumulative survival probability.

Related papers

• Exploring Training and Inference Scaling Laws in Generative Retrieval [50.82554729023865]
  We investigate how model size, training data scale, and inference-time compute jointly influence generative retrieval performance. Our experiments show that n-gram-based methods demonstrate strong alignment with both training and inference scaling laws. We find that LLaMA models consistently outperform T5 models, suggesting a particular advantage for larger decoder-only models in generative retrieval.
  arXiv  Detail & Related papers  (2025-03-24T17:59:03Z)
• Censoring-Aware Tree-Based Reinforcement Learning for Estimating Dynamic Treatment Regimes with Censored Outcomes [4.877686100899469]
  Censoring-Aware Tree-Based Reinforcement Learning (CA-TRL) is a novel framework to address the complexities associated with censored data. We demonstrate its effectiveness through extensive simulations and real-world applications using the SANAD epilepsy dataset.
  arXiv  Detail & Related papers  (2025-03-09T16:53:09Z)
• Fine-tuning -- a Transfer Learning approach [0.22344294014777952]
  Missingness in Electronic Health Records (EHRs) is often hampered by the abundance of missing data in this valuable resource. Existing deep imputation methods rely on end-to-end pipelines that incorporate both imputation and downstream analyses. This paper explores the development of a modular, deep learning-based imputation and classification pipeline.
  arXiv  Detail & Related papers  (2024-11-06T14:18:23Z)
• DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regime [18.443316087890324]
  Reinforcement learning (RL) has garnered increasing recognition for its potential to optimise dynamic treatment regimes (DTRs) in personalised medicine. We introduce textitDTR-Bench, a benchmarking platform for simulating diverse healthcare scenarios. We evaluate various state-of-the-art RL algorithms across these settings, particularly highlighting their performance amidst real-world challenges.
  arXiv  Detail & Related papers  (2024-05-28T21:40:00Z)
• Self-STORM: Deep Unrolled Self-Supervised Learning for Super-Resolution Microscopy [55.2480439325792]
  We introduce deep unrolled self-supervised learning, which alleviates the need for such data by training a sequence-specific, model-based autoencoder. Our proposed method exceeds the performance of its supervised counterparts.
  arXiv  Detail & Related papers  (2024-03-25T17:40:32Z)
• Fast Cerebral Blood Flow Analysis via Extreme Learning Machine [4.373558495838564]
  We introduce a rapid and precise analytical approach for analyzing cerebral blood flow (CBF) using Diffuse Correlation spectroscopy (DCS) We assess existing algorithms using synthetic datasets for both semi-infinite and multi-layer models. Results demonstrate that ELM consistently achieves higher fidelity across various noise levels and optical parameters, showcasing robust generalization ability and outperforming iterative fitting algorithms.
  arXiv  Detail & Related papers  (2024-01-10T23:01:35Z)
• Evolutionary Optimization of 1D-CNN for Non-contact Respiration Pattern Classification [0.19999259391104385]
  We present a deep learning-based approach for time-series respiration data classification. We employed a 1D convolutional neural network (1D-CNN) for classification purposes. Genetic algorithm was employed to optimize the 1D-CNN architecture to maximize classification accuracy.
  arXiv  Detail & Related papers  (2023-12-20T13:59:43Z)
• Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicine [20.401805132360654]
  We develop two novel methods for learning optimal dynamic treatment regimes (DTRs) Our methods are based on a data-driven estimation of heterogeneous treatment effects using causal tree methods. We evaluate our proposed methods using synthetic data and then apply them to real-world data from intensive care units.
  arXiv  Detail & Related papers  (2022-04-14T17:27:08Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• Lung Cancer Lesion Detection in Histopathology Images Using Graph-Based Sparse PCA Network [93.22587316229954]
  We propose a graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E) We evaluate the performance of the proposed algorithm on H&E slides obtained from an SVM K-rasG12D lung cancer mouse model using precision/recall rates, F-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC)
  arXiv  Detail & Related papers  (2021-10-27T19:28:36Z)
• Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation [51.66271681532262]
  Online Self-Acquired Knowledge Distillation (OSAKD) is proposed, aiming to improve the performance of any deep neural model in an online manner. We utilize k-nn non-parametric density estimation technique for estimating the unknown probability distributions of the data samples in the output feature space.
  arXiv  Detail & Related papers  (2021-08-26T14:01:04Z)
• Dynamic Mode Decomposition in Adaptive Mesh Refinement and Coarsening Simulations [58.720142291102135]
  Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract coherent schemes. This paper proposes a strategy to enable DMD to extract from observations with different mesh topologies and dimensions.
  arXiv  Detail & Related papers  (2021-04-28T22:14:25Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation [15.451690870640295]
  In some scenarios such as healthcare, usually only few records are available for each patient, impeding the application of currentReinforcement learning algorithms. We propose a data-efficient RL algorithm that exploits structural causal models (SCMs) to model the state dynamics. We show that counterfactual outcomes are identifiable under mild conditions and that Q- learning on the counterfactual-based augmented data set converges to the optimal value function.
  arXiv  Detail & Related papers  (2020-12-16T17:21:13Z)
• Deep Reinforcement Learning using Cyclical Learning Rates [62.19441737665902]
  One of the most influential parameters in optimization procedures based on gradient descent (SGD) is the learning rate. We investigate cyclical learning and propose a method for defining a general cyclical learning rate for various DRL problems. Our experiments show that, utilizing cyclical learning achieves similar or even better results than highly tuned fixed learning rates.
  arXiv  Detail & Related papers  (2020-07-31T10:06:02Z)
• A Systematic Approach to Featurization for Cancer Drug Sensitivity Predictions with Deep Learning [49.86828302591469]
  We train >35,000 neural network models, sweeping over common featurization techniques. We found the RNA-seq to be highly redundant and informative even with subsets larger than 128 features.
  arXiv  Detail & Related papers  (2020-04-30T20:42:17Z)
• Deep Learning Algorithms for Rotating Machinery Intelligent Diagnosis: An Open Source Benchmark Study [0.8497188292342053]
  This paper provides a benchmark study of deep learning algorithms for rotating machinery intelligent diagnosis. We integrate the whole evaluation codes into a code library and release this code library to the public for better development of this field. By these works, we release a unified code framework for comparing and testing models fairly and quickly, emphasize the importance of open source codes, provide the baseline accuracy (a lower bound) to avoid useless improvement, and discuss potential future directions in this field.
  arXiv  Detail & Related papers  (2020-03-06T17:24:43Z)
• Distributed Training of Deep Neural Network Acoustic Models for Automatic Speech Recognition [33.032361181388886]
  We provide an overview of distributed training techniques for deep neural network acoustic models for ASR. Experiments are carried out on a popular public benchmark to study the convergence, speedup and recognition performance of the investigated strategies.
  arXiv  Detail & Related papers  (2020-02-24T19:31:50Z)
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review [62.490310870300746]
  The electrocardiogram (ECG) is one of the most commonly used diagnostic tools in medicine and healthcare. Deep learning methods have achieved promising results on predictive healthcare tasks using ECG signals. This paper presents a systematic review of deep learning methods for ECG data from both modeling and application perspectives.
  arXiv  Detail & Related papers  (2019-12-28T02:44:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.