CXR-TFT: Multi-Modal Temporal Fusion Transformer for Predicting Chest X-ray Trajectories

• URL: http://arxiv.org/abs/2507.14766v1
• Date: Sat, 19 Jul 2025 22:42:26 GMT
• Title: CXR-TFT: Multi-Modal Temporal Fusion Transformer for Predicting Chest X-ray Trajectories
• Abstract summary: In intensive care units (ICUs), patients with complex clinical conditions require vigilant monitoring and prompt interventions.<n>Existing tools for CXR interpretation are constrained by cross-sectional analysis, failing to capture temporal dynamics.<n>We introduce CXR-TFT, a novel multi-modal framework that integrates temporally sparse CXR imaging and radiology reports with high-frequency clinical data.
• Score: 1.681259205454531
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In intensive care units (ICUs), patients with complex clinical conditions require vigilant monitoring and prompt interventions. Chest X-rays (CXRs) are a vital diagnostic tool, providing insights into clinical trajectories, but their irregular acquisition limits their utility. Existing tools for CXR interpretation are constrained by cross-sectional analysis, failing to capture temporal dynamics. To address this, we introduce CXR-TFT, a novel multi-modal framework that integrates temporally sparse CXR imaging and radiology reports with high-frequency clinical data, such as vital signs, laboratory values, and respiratory flow sheets, to predict the trajectory of CXR findings in critically ill patients. CXR-TFT leverages latent embeddings from a vision encoder that are temporally aligned with hourly clinical data through interpolation. A transformer model is then trained to predict CXR embeddings at each hour, conditioned on previous embeddings and clinical measurements. In a retrospective study of 20,000 ICU patients, CXR-TFT demonstrated high accuracy in forecasting abnormal CXR findings up to 12 hours before they became radiographically evident. This predictive capability in clinical data holds significant potential for enhancing the management of time-sensitive conditions like acute respiratory distress syndrome, where early intervention is crucial and diagnoses are often delayed. By providing distinctive temporal resolution in prognostic CXR analysis, CXR-TFT offers actionable 'whole patient' insights that can directly improve clinical outcomes.

Related papers

• Mr.Dec: Daily-Scale Longitudinal Multimodal Modeling for 30-Day Readmission Prediction [5.15235367352057]
  We propose Mr.Dec (Multimodal Readmission-risk prediction Decoder), which models each admission as a natural chronological sequence of daily multimodal events.<n>By leveraging a Transformer Decoder, Mr.Dec integrates daily Electronic Health Record(EHR) updates and intermittent Chest X-ray(CXR) findings in a time-aligned stream.<n>Our model identifies "Critical Days" within an admission, providing actionable and clinically grounded interpretations for real-time risk stratification.
  arXiv  Detail & Related papers  (2026-08-05T05:59:42Z)
• Prospective evaluation of multimodal respiratory failure prediction: Do chest X-rays improve performance beyond EHR signals? [4.250825424649631]
  We develop a gated multimodal framework that integrates structured EHR time-series data with CXR foundation-model representations.<n>We evaluate the framework for predicting invasive mechanical ventilation within 24 hours in ICU patients.
  arXiv  Detail & Related papers  (2026-05-25T18:25:47Z)
• A Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking [1.2938481923855734]
  MDS ICU is a unified machine learning framework that fuses routinely collected data including demographics, biometrics, vital signs, laboratory values, ECG waveforms, surgical procedures, and medical device usage to provide continuous predictive support during ICU stays.<n>We trained a deep learning architecture that combines structured state space S4 encoders for ECG waveforms with multilayer perceptron RealMLP encoders for tabular data to jointly predict 33 clinically relevant outcomes spanning mortality, organ dysfunction, medication needs, and acute deterioration.<n>Analysis showed close agreement between predicted and observed risks, with consistent gains from ECG waveform integration.
  arXiv  Detail & Related papers  (2026-01-10T18:11:12Z)
• CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation [62.0150409256153]
  We propose CXRAgent, a director-orchestrated, multi-stage agent for CXR interpretation.<n>The agent strategically orchestrates a set of CXR-analysis tools, with outputs normalized and verified by the Evidence-driven Validator.<n>Experiments on various CXR interpretation tasks show that CXRAgent delivers strong performance, providing visual evidence and generalizes well to clinical tasks of different complexity.
  arXiv  Detail & Related papers  (2025-10-24T10:31:30Z)
• Revisiting Performance Claims for Chest X-Ray Models Using Clinical Context [12.0757445616255]
  Using discharge summaries recorded prior to each CXR, we derive a prior'' or pre-test'' probability of each CXR label.<n>For several diagnostic labels, CXR models tend to perform best on cases where the pre-test probability is very low.<n>We find that performance drops sharply on a balanced test set where this shortcut does not exist.
  arXiv  Detail & Related papers  (2025-09-24T01:10:35Z)
• CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-ray [64.2434525370243]
  The CXR-LT series is a community-driven initiative designed to enhance lung disease classification using chest X-rays.<n>The CXR-LT 2024 expands the dataset to 377,110 chest X-rays (CXRs) and 45 disease labels, including 19 new rare disease findings.<n>This paper provides an overview of CXR-LT 2024, detailing the data curation process and consolidating state-of-the-art solutions.
  arXiv  Detail & Related papers  (2025-06-09T17:53:31Z)
• Chronic Obstructive Pulmonary Disease Prediction Using Deep Convolutional Network [0.0]
  This study presents a Deep Convolutional Neural Network (CNN)-based approach for the analysis of respiratory sound data to detect Chronic Obstructive Pulmonary Disease (COPD)<n>The system also classifies disease severity as mild, moderate, or severe.<n> Evaluation on the ICBHI database achieved 96% accuracy using 10-fold cross-validation and 90% accuracy without cross-validation.
  arXiv  Detail & Related papers  (2024-11-03T05:01:49Z)
• Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation [14.658627367126009]
  We propose DDL-CXR, a method that dynamically generates an up-to-date latent representation of the individualized chest X-ray images. Our approach leverages latent diffusion models for patient-specific generation strategically conditioned on a previous CXR image and EHR time series. Experiments using MIMIC datasets show that the proposed model could effectively address asynchronicity in multimodal fusion and consistently outperform existing methods.
  arXiv  Detail & Related papers  (2024-10-23T14:34:39Z)
• Towards Predicting Temporal Changes in a Patient's Chest X-ray Images based on Electronic Health Records [9.398163873685798]
  We propose a novel framework, EHRXDiff, which predicts future CXR images by integrating previous CXRs with subsequent medical events.<n>Results show that our framework generates high-quality, realistic future images that effectively capture potential temporal changes.
  arXiv  Detail & Related papers  (2024-09-11T04:49:44Z)
• Expert Uncertainty and Severity Aware Chest X-Ray Classification by Multi-Relationship Graph Learning [48.29204631769816]
  We re-extract disease labels from CXR reports to make them more realistic by considering disease severity and uncertainty in classification. Our experimental results show that models considering disease severity and uncertainty outperform previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-06T19:19:41Z)
• Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generation [7.586632627817609]
  Radiologists face high burnout rates, partly due to the increasing volume of Chest X-rays (CXRs) requiring interpretation and reporting. Our proposed CXR report generator integrates elements of the workflow and introduces a novel reward for reinforcement learning. Results from our study demonstrate that the proposed model generates reports that are more aligned with radiologists' reports than state-of-the-art models.
  arXiv  Detail & Related papers  (2023-07-19T05:41:14Z)
• Revisiting Computer-Aided Tuberculosis Diagnosis [56.80999479735375]
  Tuberculosis (TB) is a major global health threat, causing millions of deaths annually. Computer-aided tuberculosis diagnosis (CTD) using deep learning has shown promise, but progress is hindered by limited training data. We establish a large-scale dataset, namely the Tuberculosis X-ray (TBX11K) dataset, which contains 11,200 chest X-ray (CXR) images with corresponding bounding box annotations for TB areas. This dataset enables the training of sophisticated detectors for high-quality CTD.
  arXiv  Detail & Related papers  (2023-07-06T08:27:48Z)
• Optimising Chest X-Rays for Image Analysis by Identifying and Removing Confounding Factors [49.005337470305584]
  During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. The variable quality of clinically-acquired CXRs within publicly available datasets could have a profound effect on algorithm performance. We propose a simple and effective step-wise approach to pre-processing a COVID-19 chest X-ray dataset to remove undesired biases.
  arXiv  Detail & Related papers  (2022-08-22T13:57:04Z)
• COVID-Net CXR-S: Deep Convolutional Neural Network for Severity Assessment of COVID-19 Cases from Chest X-ray Images [74.77272804752306]
  We introduce COVID-Net CXR-S, a convolutional neural network for predicting the airspace severity of a SARS-CoV-2 positive patient based on a CXR image of the patient's chest. We leveraged transfer learning to transfer representational knowledge gained from over 16,000 CXR images from a multinational cohort of over 15,000 patient cases into a custom network architecture for severity assessment. The proposed COVID-Net CXR-S has potential to be a powerful tool for computer-aided severity assessment of CXR images of COVID-19 positive patients.
  arXiv  Detail & Related papers  (2021-05-01T14:15:12Z)
• AIforCOVID: predicting the clinical outcomes in patients with COVID-19 applying AI to chest-X-rays. An Italian multicentre study [7.456548336226919]
  We investigate whether chest X-ray (CXR) can be used as a possible tool for the early identification of patients at risk of severe outcome, like intensive care or death. CXR is a radiological technique that compared to computed tomography (CT) it is simpler, faster, more widespread and it induces lower radiation dose. We present a dataset including data collected from 820 patients by six Italian hospitals in spring 2020 during the first COVID-19 emergency.
  arXiv  Detail & Related papers  (2020-12-11T18:03:08Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Integrative Analysis for COVID-19 Patient Outcome Prediction [53.11258640541513]
  We combine radiomics of lung opacities and non-imaging features from demographic data, vital signs, and laboratory findings to predict need for intensive care unit admission. Our methods may also be applied to other lung diseases including but not limited to community acquired pneumonia.
  arXiv  Detail & Related papers  (2020-07-20T19:08:50Z)
• AI-Driven CT-based quantification, staging and short-term outcome prediction of COVID-19 pneumonia [3.672093204122992]
  Chest computed tomography (CT) is widely used for the management of Coronavirus disease 2019 (COVID-19) pneumonia. CT has a prognostic role by allowing visually evaluating the extent of COVID-19 lung abnormalities.
  arXiv  Detail & Related papers  (2020-04-20T12:24:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.