Deep Reinforcement Learning for Cost-Effective Medical Diagnosis

• URL: http://arxiv.org/abs/2302.10261v1
• Date: Mon, 20 Feb 2023 19:47:25 GMT
• Title: Deep Reinforcement Learning for Cost-Effective Medical Diagnosis
• Authors: Zheng Yu, Yikuan Li, Joseph Kim, Kaixuan Huang, Yuan Luo, Mengdi Wang
• Abstract summary: We use reinforcement learning to find a dynamic policy that selects lab test panels sequentially based on previous observations. We propose a Semi-Model-based Deep Diagnosis Policy Optimization framework that is compatible with end-to-end training and online learning. SM-DDPO is tested on diverse clinical tasks: ferritin abnormality detection, sepsis mortality prediction, and acute kidney injury diagnosis.
• Score: 41.10546022107126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dynamic diagnosis is desirable when medical tests are costly or time-consuming. In this work, we use reinforcement learning (RL) to find a dynamic policy that selects lab test panels sequentially based on previous observations, ensuring accurate testing at a low cost. Clinical diagnostic data are often highly imbalanced; therefore, we aim to maximize the $F_1$ score instead of the error rate. However, optimizing the non-concave $F_1$ score is not a classic RL problem, thus invalidates standard RL methods. To remedy this issue, we develop a reward shaping approach, leveraging properties of the $F_1$ score and duality of policy optimization, to provably find the set of all Pareto-optimal policies for budget-constrained $F_1$ score maximization. To handle the combinatorially complex state space, we propose a Semi-Model-based Deep Diagnosis Policy Optimization (SM-DDPO) framework that is compatible with end-to-end training and online learning. SM-DDPO is tested on diverse clinical tasks: ferritin abnormality detection, sepsis mortality prediction, and acute kidney injury diagnosis. Experiments with real-world data validate that SM-DDPO trains efficiently and identifies all Pareto-front solutions. Across all tasks, SM-DDPO is able to achieve state-of-the-art diagnosis accuracy (in some cases higher than conventional methods) with up to $85\%$ reduction in testing cost. The code is available at [https://github.com/Zheng321/Blood_Panel].

Related papers

• A Progressive Single-Modality to Multi-Modality Classification Framework for Alzheimer's Disease Sub-type Diagnosis [40.838012045852]
  Current Alzheimer's disease (AD) diagnosis involves multiple modalities acquired from multiple diagnosis stages, each with distinct usage and cost. We propose a novel progressive AD sub-type diagnosis framework, aiming to give diagnosis results based on easier-to-access modalities in earlier low-cost stages. We evaluate our proposed framework on large diverse public and in-home datasets and achieve superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-26T02:25:45Z)
• Exploiting Pre-trained Models for Drug Target Affinity Prediction with Nearest Neighbors [58.661454334877256]
  Drug-Target binding Affinity (DTA) prediction is essential for drug discovery. Despite the application of deep learning methods to DTA prediction, the achieved accuracy remain suboptimal. We propose $k$NN-DTA, a non-representation embedding-based retrieval method adopted on a pre-trained DTA prediction model.
  arXiv  Detail & Related papers  (2024-07-21T15:49:05Z)
• Improving Bias Correction Standards by Quantifying its Effects on Treatment Outcomes [54.18828236350544]
  Propensity score matching (PSM) addresses selection biases by selecting comparable populations for analysis. Different matching methods can produce significantly different Average Treatment Effects (ATE) for the same task, even when meeting all validation criteria. To address this issue, we introduce a novel metric, A2A, to reduce the number of valid matches.
  arXiv  Detail & Related papers  (2024-07-20T12:42:24Z)
• Calibrated Diverse Ensemble Entropy Minimization for Robust Test-Time Adaptation in Prostate Cancer Detection [4.123522672804104]
  High resolution micro-ultrasound has demonstrated promise in real-time prostate cancer detection. Deep learning is a prominent tool for learning complex tissue properties reflected on ultrasound. We show that model performance suffers when applied to data from different clinical centers due to variations in data distribution.
  arXiv  Detail & Related papers  (2024-07-17T16:20:10Z)
• DDxT: Deep Generative Transformer Models for Differential Diagnosis [51.25660111437394]
  We show that a generative approach trained with simpler supervised and self-supervised learning signals can achieve superior results on the current benchmark. The proposed Transformer-based generative network, named DDxT, autoregressively produces a set of possible pathologies, i.e., DDx, and predicts the actual pathology using a neural network.
  arXiv  Detail & Related papers  (2023-12-02T22:57:25Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• FIT: a Fast and Accurate Framework for Solving Medical Inquiring and Diagnosing Tasks [10.687562550605739]
  Self-diagnosis provides low-cost and accessible healthcare via an agent that queries the patient and makes predictions about possible diseases. We propose a competitive framework, called FIT, which uses an information-theoretic reward to determine what data to collect next. Our results in two simulated datasets show that FIT can effectively deal with large search space problems, outperforming existing baselines.
  arXiv  Detail & Related papers  (2020-12-02T10:12:49Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)
• Semixup: In- and Out-of-Manifold Regularization for Deep Semi-Supervised Knee Osteoarthritis Severity Grading from Plain Radiographs [3.0969191504482247]
  Knee osteoarthritis (OA) is one of the highest disability factors in the world. Deep learning methods can reliably perform the OA severity assessment according to the gold standard Kellgren-Lawrence (KL) grading system. We propose the Semixup algorithm, a semi-supervised learning (SSL) approach to leverage unlabeled data.
  arXiv  Detail & Related papers  (2020-03-04T08:33:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.