PIE: Simulating Disease Progression via Progressive Image Editing
- URL: http://arxiv.org/abs/2309.11745v2
- Date: Thu, 5 Oct 2023 04:45:21 GMT
- Title: PIE: Simulating Disease Progression via Progressive Image Editing
- Authors: Kaizhao Liang, Xu Cao, Kuei-Da Liao, Tianren Gao, Wenqian Ye, Zhengyu
Chen, Jianguo Cao, Tejas Nama, Jimeng Sun
- Abstract summary: Progressive Image Editing (PIE) enables controlled manipulation of disease-related image features.
We leverage recent advancements in text-to-image generative models to simulate disease progression accurately and personalize it for each patient.
PIE is the first of its kind to generate disease progression images meeting real-world standards.
- Score: 27.658116659009025
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Disease progression simulation is a crucial area of research that has
significant implications for clinical diagnosis, prognosis, and treatment. One
major challenge in this field is the lack of continuous medical imaging
monitoring of individual patients over time. To address this issue, we develop
a novel framework termed Progressive Image Editing (PIE) that enables
controlled manipulation of disease-related image features, facilitating precise
and realistic disease progression simulation. Specifically, we leverage recent
advancements in text-to-image generative models to simulate disease progression
accurately and personalize it for each patient. We theoretically analyze the
iterative refining process in our framework as a gradient descent with an
exponentially decayed learning rate. To validate our framework, we conduct
experiments in three medical imaging domains. Our results demonstrate the
superiority of PIE over existing methods such as Stable Diffusion Walk and
Style-Based Manifold Extrapolation based on CLIP score (Realism) and Disease
Classification Confidence (Alignment). Our user study collected feedback from
35 veteran physicians to assess the generated progressions. Remarkably, 76.2%
of the feedback agrees with the fidelity of the generated progressions. To our
best knowledge, PIE is the first of its kind to generate disease progression
images meeting real-world standards. It is a promising tool for medical
research and clinical practice, potentially allowing healthcare providers to
model disease trajectories over time, predict future treatment responses, and
improve patient outcomes.
Related papers
- Probabilistic Temporal Prediction of Continuous Disease Trajectories and Treatment Effects Using Neural SDEs [6.5527554277858275]
We present the first causal temporal framework to model the continuous temporal evolution of disease progression via Neural Differential Equations (NSDE)
Our results present the first successful uncertainty-based causal Deep Learning (DL) model to accurately predict future patient MS disability (e.g. EDSS) and treatment effects.
arXiv Detail & Related papers (2024-06-18T17:22:55Z) - Adversarial-Robust Transfer Learning for Medical Imaging via Domain
Assimilation [17.46080957271494]
The scarcity of publicly available medical images has led contemporary algorithms to depend on pretrained models grounded on a large set of natural images.
A significant em domain discrepancy exists between natural and medical images, which causes AI models to exhibit heightened em vulnerability to adversarial attacks.
This paper proposes a em domain assimilation approach that introduces texture and color adaptation into transfer learning, followed by a texture preservation component to suppress undesired distortion.
arXiv Detail & Related papers (2024-02-25T06:39:15Z) - Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary
Task Integration [54.76511683427566]
This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information.
A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction.
The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures.
arXiv Detail & Related papers (2024-02-16T05:16:20Z) - BiomedJourney: Counterfactual Biomedical Image Generation by
Instruction-Learning from Multimodal Patient Journeys [99.7082441544384]
We present BiomedJourney, a novel method for counterfactual biomedical image generation by instruction-learning.
We use GPT-4 to process the corresponding imaging reports and generate a natural language description of disease progression.
The resulting triples are then used to train a latent diffusion model for counterfactual biomedical image generation.
arXiv Detail & Related papers (2023-10-16T18:59:31Z) - Learning Spatio-Temporal Model of Disease Progression with NeuralODEs
from Longitudinal Volumetric Data [4.998875488622879]
We develop a deep learning method that models the evolution of age-related disease by processing a single medical scan.
For Geographic Atrophy, the proposed method outperformed the related baseline models in the atrophy growth prediction.
For Alzheimer's Disease, the proposed method demonstrated remarkable performance in predicting the brain ventricle changes induced by the disease.
arXiv Detail & Related papers (2022-11-08T13:28:26Z) - Temporal Context Matters: Enhancing Single Image Prediction with Disease
Progression Representations [8.396615243014768]
We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images.
In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory.
A Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images.
arXiv Detail & Related papers (2022-03-02T22:11:07Z) - Variational Knowledge Distillation for Disease Classification in Chest
X-Rays [102.04931207504173]
We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays.
We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
arXiv Detail & Related papers (2021-03-19T14:13:56Z) - MAVIDH Score: A COVID-19 Severity Scoring using Chest X-Ray Pathology
Features [10.671315906986754]
The application of computer vision for COVID-19 diagnosis is complex and challenging.
The primary value of medical imaging for COVID-19 lies rather on patient prognosis.
A simple method based on lung-pathology interpretable features for scoring disease severity from Chest X-rays is proposed here.
arXiv Detail & Related papers (2020-11-30T16:55:28Z) - Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
A common scenario in developing countries is to have the volume metadata lost due multiple reasons.
It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
arXiv Detail & Related papers (2020-10-20T04:30:09Z) - MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
This paper aims at a unified deep learning approach to predict patient prognosis and therapy response.
We formalize the prognosis modeling as a multi-modal asynchronous time series classification task.
Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
arXiv Detail & Related papers (2020-10-08T15:30:17Z) - Estimating Counterfactual Treatment Outcomes over Time Through
Adversarially Balanced Representations [114.16762407465427]
We introduce the Counterfactual Recurrent Network (CRN) to estimate treatment effects over time.
CRN uses domain adversarial training to build balancing representations of the patient history.
We show how our model achieves lower error in estimating counterfactuals and in choosing the correct treatment and timing of treatment.
arXiv Detail & Related papers (2020-02-10T20:47:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.