TMF-RSE: Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty for Lung Severity Scoring

• URL: http://arxiv.org/abs/2607.06356v1
• Date: Tue, 07 Jul 2026 14:54:40 GMT
• Title: TMF-RSE: Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty for Lung Severity Scoring
• Abstract summary: We propose a tri-modal deep learning framework, TMF-RSE, that combines appearance features from two-dimensional chest inputs, structural features from lung segmentation masks, and semantic features from vision-language models for severity quantification.<n>Our approach employs complementary fusion mechanisms that integrate semantic guidance, structural priors, and hierarchical interactions across modalities.<n> Experiments on the Per-COVID-19 CT and RALO datasets show that TMF-RSE outperforms recent transformer-based baselines.
• Score: 11.785230718509196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate quantification of lung disease severity from chest imaging is critical for clinical decision-making and resource allocation. We propose a tri-modal deep learning framework, TMF-RSE (Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty), that combines appearance features from two-dimensional chest inputs, structural features from lung segmentation masks, and semantic features from vision-language models (VLMs) for severity quantification. Our approach employs complementary fusion mechanisms that integrate semantic guidance, structural priors, and hierarchical interactions across modalities. The model employs evidential regression to provide both severity predictions and uncertainty estimates. Experiments on the Per-COVID-19 CT and RALO datasets show that TMF-RSE outperforms recent transformer-based baselines, achieving MAE of 4.02 and Pearson correlation of 0.9629 on Per-COVID-19 validation, and 0.339 MAE / 0.973 PC on RALO geographic extent.

Related papers

• Understanding Synergistic Interactions among Pathology Foundation Models via Adaptive Fusion [49.550545038402184]
  Pathology foundation models (PFMs) provide strong tile-level representations via self-supervised pre-training on large-scale pathology images.<n>We present AdaFusion, a lightweight adaptive fusion framework that integrates complementary signals from multiple frozen PFMs.<n>AdaFusion consistently outperforms individual PFMs and other fusion baselines, while providing interpretable tissue visualisation.
  arXiv  Detail & Related papers  (2026-08-02T16:44:45Z)
• Spatio-Temporal Mixture-of-Modality-Experts Diffusion for Quantitative DCE-MRI Synthesis from Incomplete MR Sequences [2.4827516038838064]
  ST-MoME is a conditional diffusion framework that synthesizes 3D DCE parameter maps from diverse subsets multimodal MRI.<n>It performs diffusion directly in image space with 3D patch-based training and a Swin-based backbone.<n>It achieves the lowest mean Normalized Mean Square Error (NMSE) aggregated across all three DCE parameters, with leading performance on $v_p$ and $v_e$, competitive results on $Kmathrmtrans$, and the lowest reconstruction error within the clinically critical tumor region.
  arXiv  Detail & Related papers  (2026-06-24T08:10:27Z)
• Adaptive Temporal Gating of Longitudinal Magnetic Resonance Imaging for Alzheimer's Prediction [4.549831511476248]
  We introduce the Temporal Adaptive Fusion Network (TAF-Net), a hybrid CNN-Transformer architecture that models longitudinal 3D MRI scans.<n> evaluated on the Alzheimer's Disease Neuroimaging Initiative cohort for three-year MCI-to-AD conversion prediction.
  arXiv  Detail & Related papers  (2026-05-27T12:33:46Z)
• D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities [54.87947751720332]
  We propose a novel segmentation model D3Seg to maintain stable performance under missing-modality settings.<n>The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing modality configurations.
  arXiv  Detail & Related papers  (2026-05-21T09:55:42Z)
• UniField: A Unified Field-Aware MRI Enhancement Framework [45.03230466428695]
  methodname is a unified framework integrating multiple modalities and enhancement tasks.<n>We exploit comprehensive 3D volumetric information by leveraging pre-trained 3D foundation models.<n>We organize and publicly release a comprehensive paired multi-field MRI dataset.
  arXiv  Detail & Related papers  (2026-03-10T05:45:12Z)
• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• CONFIDE: Hallucination Assessment for Reliable Biomolecular Structure Prediction and Design [46.12506067241116]
  We present CODE (Chain of Diffusion Embeddings), a self evaluating metric to quantify topological frustration.<n>We propose CONFIDE, a unified evaluation framework that combines energetic and topological perspectives.<n>By combining data driven embeddings with theoretical insight, CODE and CONFIDE outperform existing metrics across a wide range of biomolecular systems.
  arXiv  Detail & Related papers  (2025-11-20T03:38:46Z)
• Transparent Early ICU Mortality Prediction with Clinical Transformer and Per-Case Modality Attribution [42.85462513661566]
  We present a lightweight, transparent multimodal ensemble that fuses physiological time-series measurements with unstructured clinical notes from the first 48 hours of an ICU stay.<n>A logistic regression model combines predictions from two modality-specific models: a bidirectional LSTM for vitals and a finetuned ClinicalModernBERT transformer for notes.<n>On the MIMIC-III benchmark, our late-fusion ensemble improves discrimination over the best single model while maintaining well-calibrated predictions.
  arXiv  Detail & Related papers  (2025-11-19T20:11:49Z)
• PF-DAformer: Proximal Femur Segmentation via Domain Adaptive Transformer for Dual-Center QCT [8.358409792893278]
  We develop a domain-adaptive transformer segmentation framework tailored for multi-institutional Quantitative computed tomography (QCT)<n>Our model is trained and validated on one of the largest hip fracture related research cohorts to date, comprising 1,024 QCT images scans from Tulane University and 384 scans from Rochester, Minnesota for proximal femur segmentation.
  arXiv  Detail & Related papers  (2025-10-30T18:07:56Z)
• AdaFusion: Prompt-Guided Inference with Adaptive Fusion of Pathology Foundation Models [49.550545038402184]
  We propose AdaFusion, a novel prompt-guided inference framework.<n>Our method compresses and aligns tile-level features from diverse models.<n>AdaFusion consistently surpasses individual PFMs across both classification and regression tasks.
  arXiv  Detail & Related papers  (2025-08-07T07:09:31Z)
• HWA-UNETR: Hierarchical Window Aggregate UNETR for 3D Multimodal Gastric Lesion Segmentation [11.180283823428711]
  HWA-UNETR is a novel 3D segmentation framework that employs an original HWA block with learnable window aggregation layers.<n>Our framework surpasses existing methods by up to 1.68% in the Dice score while maintaining solid robustness.
  arXiv  Detail & Related papers  (2025-05-15T16:18:00Z)
• Multi-modal Evidential Fusion Network for Trustworthy PET/CT Tumor Segmentation [5.839660501978193]
  In clinical settings, the quality of PET and CT images often varies significantly, leading to uncertainty in the modality information extracted by networks.<n>We propose a novel Multi-modal Evidential Fusion Network (MEFN), which consists of two core stages: Cross-Modal Feature Learning (CFL) and Multi-modal Trustworthy Fusion (MTF)<n>Our model can provide radiologists with credible uncertainty of the segmentation results for their decision in accepting or rejecting the automatic segmentation results.
  arXiv  Detail & Related papers  (2024-06-26T13:14:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.