Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation

• URL: http://arxiv.org/abs/2410.17918v1
• Date: Wed, 23 Oct 2024 14:34:39 GMT
• Title: Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation
• Authors: Wenfang Yao, Chen Liu, Kejing Yin, William K. Cheung, Jing Qin,
• Abstract summary: We propose DDL-CXR, a method that dynamically generates an up-to-date latent representation of the individualized chest X-ray images. Our approach leverages latent diffusion models for patient-specific generation strategically conditioned on a previous CXR image and EHR time series. Experiments using MIMIC datasets show that the proposed model could effectively address asynchronicity in multimodal fusion and consistently outperform existing methods.
• Score: 14.658627367126009
• License:
• Abstract: Integrating multi-modal clinical data, such as electronic health records (EHR) and chest X-ray images (CXR), is particularly beneficial for clinical prediction tasks. However, in a temporal setting, multi-modal data are often inherently asynchronous. EHR can be continuously collected but CXR is generally taken with a much longer interval due to its high cost and radiation dose. When clinical prediction is needed, the last available CXR image might have been outdated, leading to suboptimal predictions. To address this challenge, we propose DDL-CXR, a method that dynamically generates an up-to-date latent representation of the individualized CXR images. Our approach leverages latent diffusion models for patient-specific generation strategically conditioned on a previous CXR image and EHR time series, providing information regarding anatomical structures and disease progressions, respectively. In this way, the interaction across modalities could be better captured by the latent CXR generation process, ultimately improving the prediction performance. Experiments using MIMIC datasets show that the proposed model could effectively address asynchronicity in multimodal fusion and consistently outperform existing methods.

Related papers

• Towards Predicting Temporal Changes in a Patient's Chest X-ray Images based on Electronic Health Records [9.398163873685798]
  We propose a novel framework, EHRXDiff, which predicts future CXR images by integrating previous CXRs with subsequent medical events. We demonstrate that our framework generates high-quality, realistic future images that capture potential temporal changes. This could offer valuable insights for patient monitoring and treatment planning in the medical field.
  arXiv  Detail & Related papers  (2024-09-11T04:49:44Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generation [7.586632627817609]
  Radiologists face high burnout rates, partly due to the increasing volume of Chest X-rays (CXRs) requiring interpretation and reporting. Our proposed CXR report generator integrates elements of the workflow and introduces a novel reward for reinforcement learning. Results from our study demonstrate that the proposed model generates reports that are more aligned with radiologists' reports than state-of-the-art models.
  arXiv  Detail & Related papers  (2023-07-19T05:41:14Z)
• tdCoxSNN: Time-Dependent Cox Survival Neural Network for Continuous-time Dynamic Prediction [19.38247205641199]
  We propose a time-dependent Cox survival neural network (tdCoxSNN) to predict its progression using longitudinal fundus images. We evaluate and compare our proposed method with joint modeling and landmarking approaches through extensive simulations.
  arXiv  Detail & Related papers  (2023-07-12T03:03:40Z)
• End-To-End Prediction of Knee Osteoarthritis Progression With Multi-Modal Transformers [2.9822184411723645]
  Knee Osteoarthritis (KOA) is a highly prevalent chronic musculoskeletal condition with no currently available treatment. We leveraged recent advances in Deep Learning and developed a unified framework for the multi-modal fusion of knee imaging data. Our follow-up analysis generally shows that prediction from the imaging data is more accurate for post-traumatic subjects.
  arXiv  Detail & Related papers  (2023-07-03T09:10:57Z)
• Early Diagnosis of Chronic Obstructive Pulmonary Disease from Chest X-Rays using Transfer Learning and Fusion Strategies [1.234198411367205]
  Chronic obstructive pulmonary disease (COPD) is one of the most common chronic illnesses in the world and the third leading cause of mortality worldwide. It is often underdiagnosed or not diagnosed until later in the disease course. Spirometry tests are the gold standard for diagnosing COPD but can be difficult to obtain, especially in resource-poor countries. Chest X-rays (CXRs) are readily available and may serve as a screening tool to identify patients with COPD who should undergo further testing.
  arXiv  Detail & Related papers  (2022-11-13T15:12:22Z)
• Optimising Chest X-Rays for Image Analysis by Identifying and Removing Confounding Factors [49.005337470305584]
  During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. The variable quality of clinically-acquired CXRs within publicly available datasets could have a profound effect on algorithm performance. We propose a simple and effective step-wise approach to pre-processing a COVID-19 chest X-ray dataset to remove undesired biases.
  arXiv  Detail & Related papers  (2022-08-22T13:57:04Z)
• SurvLatent ODE : A Neural ODE based time-to-event model with competing risks for longitudinal data improves cancer-associated Deep Vein Thrombosis (DVT) prediction [68.8204255655161]
  We propose a generative time-to-event model, SurvLatent ODE, which parameterizes a latent representation under irregularly sampled data. Our model then utilizes the latent representation to flexibly estimate survival times for multiple competing events without specifying shapes of event-specific hazard function. SurvLatent ODE outperforms the current clinical standard Khorana Risk scores for stratifying DVT risk groups.
  arXiv  Detail & Related papers  (2022-04-20T17:28:08Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.