PIE: Simulating Disease Progression via Progressive Image Editing

• URL: http://arxiv.org/abs/2309.11745v2
• Date: Thu, 5 Oct 2023 04:45:21 GMT
• Title: PIE: Simulating Disease Progression via Progressive Image Editing
• Authors: Kaizhao Liang, Xu Cao, Kuei-Da Liao, Tianren Gao, Wenqian Ye, Zhengyu Chen, Jianguo Cao, Tejas Nama, Jimeng Sun
• Abstract summary: Progressive Image Editing (PIE) enables controlled manipulation of disease-related image features. We leverage recent advancements in text-to-image generative models to simulate disease progression accurately and personalize it for each patient. PIE is the first of its kind to generate disease progression images meeting real-world standards.
• Score: 27.658116659009025
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Disease progression simulation is a crucial area of research that has significant implications for clinical diagnosis, prognosis, and treatment. One major challenge in this field is the lack of continuous medical imaging monitoring of individual patients over time. To address this issue, we develop a novel framework termed Progressive Image Editing (PIE) that enables controlled manipulation of disease-related image features, facilitating precise and realistic disease progression simulation. Specifically, we leverage recent advancements in text-to-image generative models to simulate disease progression accurately and personalize it for each patient. We theoretically analyze the iterative refining process in our framework as a gradient descent with an exponentially decayed learning rate. To validate our framework, we conduct experiments in three medical imaging domains. Our results demonstrate the superiority of PIE over existing methods such as Stable Diffusion Walk and Style-Based Manifold Extrapolation based on CLIP score (Realism) and Disease Classification Confidence (Alignment). Our user study collected feedback from 35 veteran physicians to assess the generated progressions. Remarkably, 76.2% of the feedback agrees with the fidelity of the generated progressions. To our best knowledge, PIE is the first of its kind to generate disease progression images meeting real-world standards. It is a promising tool for medical research and clinical practice, potentially allowing healthcare providers to model disease trajectories over time, predict future treatment responses, and improve patient outcomes.

Related papers

• Towards Predicting Temporal Changes in a Patient's Chest X-ray Images based on Electronic Health Records [9.398163873685798]
  We propose a novel framework, EHRXDiff, which predicts future CXR images by integrating previous CXRs with subsequent medical events. We demonstrate that our framework generates high-quality, realistic future images that capture potential temporal changes. This could offer valuable insights for patient monitoring and treatment planning in the medical field.
  arXiv  Detail & Related papers  (2024-09-11T04:49:44Z)
• Exploring Foundation Models for Synthetic Medical Imaging: A Study on Chest X-Rays and Fine-Tuning Techniques [0.49000940389224884]
  Machine learning has significantly advanced healthcare by aiding in disease prevention and treatment identification. However, accessing patient data can be challenging due to privacy concerns and strict regulations. Recent studies suggest that fine-tuning foundation models can produce such data effectively.
  arXiv  Detail & Related papers  (2024-09-06T17:36:08Z)
• Multi-task Learning Approach for Intracranial Hemorrhage Prognosis [0.0]
  We propose a 3D multi-task image model to predict prognosis, Glasgow Coma Scale and age, improving accuracy and interpretability. Our method outperforms current state-of-the-art baseline image models, and demonstrates superior performance in ICH prognosis compared to four board-certified neuroradiologists using only CT scans as input.
  arXiv  Detail & Related papers  (2024-08-16T14:56:17Z)
• Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration [54.76511683427566]
  This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information. A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction. The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures.
  arXiv  Detail & Related papers  (2024-02-16T05:16:20Z)
• BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys [99.7082441544384]
  We present BiomedJourney, a novel method for counterfactual biomedical image generation by instruction-learning. We use GPT-4 to process the corresponding imaging reports and generate a natural language description of disease progression. The resulting triples are then used to train a latent diffusion model for counterfactual biomedical image generation.
  arXiv  Detail & Related papers  (2023-10-16T18:59:31Z)
• Learning Spatio-Temporal Model of Disease Progression with NeuralODEs from Longitudinal Volumetric Data [4.998875488622879]
  We develop a deep learning method that models the evolution of age-related disease by processing a single medical scan. For Geographic Atrophy, the proposed method outperformed the related baseline models in the atrophy growth prediction. For Alzheimer's Disease, the proposed method demonstrated remarkable performance in predicting the brain ventricle changes induced by the disease.
  arXiv  Detail & Related papers  (2022-11-08T13:28:26Z)
• Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations [8.396615243014768]
  We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory. A Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images.
  arXiv  Detail & Related papers  (2022-03-02T22:11:07Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
  A common scenario in developing countries is to have the volume metadata lost due multiple reasons. It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
  arXiv  Detail & Related papers  (2020-10-20T04:30:09Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations [114.16762407465427]
  We introduce the Counterfactual Recurrent Network (CRN) to estimate treatment effects over time. CRN uses domain adversarial training to build balancing representations of the patient history. We show how our model achieves lower error in estimating counterfactuals and in choosing the correct treatment and timing of treatment.
  arXiv  Detail & Related papers  (2020-02-10T20:47:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.