Trimodal Deep Learning for Glioma Survival Prediction: A Feasibility Study Integrating Histopathology, Gene Expression, and MRI

• URL: http://arxiv.org/abs/2603.29968v1
• Date: Tue, 31 Mar 2026 16:32:22 GMT
• Title: Trimodal Deep Learning for Glioma Survival Prediction: A Feasibility Study Integrating Histopathology, Gene Expression, and MRI
• Abstract summary: This pilot study extends a bimodal framework by incorporating Fluid Attenuated Inversion Recovery (FLAIR) MRI as a third modality.<n>Using the TCGA-GBMLGG cohort (664 patients), we evaluate three unimodal models, nine bimodal configurations, and three trimodal configurations across early, late, and joint fusion strategies.<n>MRI achieves reasonable unimodal discrimination (CS = 0.755) but does not substantially improve bimodal pairs, while providing measurable uplift in the three-way combination.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal deep learning has improved prognostic accuracy for brain tumours by integrating histopathology and genomic data, yet the contribution of volumetric MRI within unified survival frameworks remains unexplored. This pilot study extends a bimodal framework by incorporating Fluid Attenuated Inversion Recovery (FLAIR) MRI from BraTS2021 as a third modality. Using the TCGA-GBMLGG cohort (664 patients), we evaluate three unimodal models, nine bimodal configurations, and three trimodal configurations across early, late, and joint fusion strategies. In this small cohort setting, trimodal early fusion achieves an exploratory Composite Score (CS = 0.854), with a controlled $Δ$CS of +0.011 over the bimodal baseline on identical patients, though this difference is not statistically significant (p = 0.250, permutation test). MRI achieves reasonable unimodal discrimination (CS = 0.755) but does not substantially improve bimodal pairs, while providing measurable uplift in the three-way combination. All MRI containing experiments are constrained to 19 test patients, yielding wide bootstrap confidence intervals (e.g. [0.400,1.000]) that preclude definitive conclusions. These findings provide preliminary evidence that a third imaging modality may add prognostic value even with limited sample sizes, and that additional modalities require sufficient multimodal context to contribute effectively.

Related papers

• Spatio-Temporal Mixture-of-Modality-Experts Diffusion for Quantitative DCE-MRI Synthesis from Incomplete MR Sequences [2.4827516038838064]
  ST-MoME is a conditional diffusion framework that synthesizes 3D DCE parameter maps from diverse subsets multimodal MRI.<n>It performs diffusion directly in image space with 3D patch-based training and a Swin-based backbone.<n>It achieves the lowest mean Normalized Mean Square Error (NMSE) aggregated across all three DCE parameters, with leading performance on $v_p$ and $v_e$, competitive results on $Kmathrmtrans$, and the lowest reconstruction error within the clinically critical tumor region.
  arXiv  Detail & Related papers  (2026-06-24T08:10:27Z)
• D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities [54.87947751720332]
  We propose a novel segmentation model D3Seg to maintain stable performance under missing-modality settings.<n>The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing modality configurations.
  arXiv  Detail & Related papers  (2026-05-21T09:55:42Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• A Systematic Benchmark of GAN Architectures for MRI-to-CT Synthesis [38.31064949197421]
  Generative Adversarial Network (GAN) architectures have been proposed for MRI-to-CT translation.<n>We present a benchmark of ten GAN evaluated on the SynthRAD2025 dataset across three anatomical districts.
  arXiv  Detail & Related papers  (2026-03-13T18:54:20Z)
• Quantitative mapping from conventional MRI using self-supervised physics-guided deep learning: applications to a large-scale, clinically heterogeneous dataset [32.995373978092665]
  This study presents a self-supervised physics-guided deep learning framework to infer quantitative T1, T2, and proton-density maps.<n>The framework was trained and evaluated on a large-scale, clinically heterogeneous dataset.
  arXiv  Detail & Related papers  (2026-01-08T16:08:58Z)
• Placenta Accreta Spectrum Detection using Multimodal Deep Learning [0.6192567865226507]
  Placenta Accreta Spectrum (PAS) is a life-threatening complication involving abnormal placental invasion into the uterine wall.<n>This study aimed to develop and validate a deep learning framework that enhances PAS detection by integrating multiple imaging modalities.
  arXiv  Detail & Related papers  (2025-12-31T23:55:56Z)
• Pattern-Aware Diffusion Synthesis of fMRI/dMRI with Tissue and Microstructural Refinement [34.55493442995441]
  We propose PDS, a pattern-aware dual-modal 3D diffusion framework for cross-modality learning.<n>We also introduce a tissue refinement network integrated with a efficient microstructure refinement to maintain structural fidelity and fine details.<n>PDS achieves state-of-the-art results, with PSNR/SSIM scores of 29.83 dB/90.84% for fMRI synthesis and 30.00 dB/77.55% for dMRI synthesis.
  arXiv  Detail & Related papers  (2025-11-07T03:51:00Z)
• Evaluation of Machine Learning Reconstruction Techniques for Accelerated Brain MRI Scans [0.0]
  DeepFoqus-Accelerate enables robust fourfold brain MRI acceleration with 75% reduced scan time.<n>These findings demonstrate that DeepFoqus-Accelerate enables robust fourfold brain MRI acceleration with 75% reduced scan time.
  arXiv  Detail & Related papers  (2025-09-08T20:20:24Z)
• $M^{2}$Fusion: Bayesian-based Multimodal Multi-level Fusion on Colorectal Cancer Microsatellite Instability Prediction [37.210658002943525]
  Colorectal cancer (CRC) micro-satellite instability (MSI) prediction on histopathology images is a challenging weakly supervised learning task. We propose $M2$Fusion: a Bayesian-based multimodal multi-level fusion pipeline for CRC MSI prediction. Our approach is validated on cross-validation of 352 cases and outperforms either feature-level (0.8177 vs. 0.7908) or decision-level fusion strategy (0.8177 vs. 0.7289) on AUC score.
  arXiv  Detail & Related papers  (2024-01-15T17:28:58Z)
• M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities [29.455215925816187]
  Multimodal magnetic resonance imaging (MRI) provides complementary information for sub-region analysis of brain tumors. It is common to have one or more modalities missing due to image corruption, artifacts, acquisition protocols, allergy to contrast agents, or simply cost. We propose a novel two-stage framework for brain tumor segmentation with missing modalities.
  arXiv  Detail & Related papers  (2023-03-09T14:54:30Z)
• Moving from 2D to 3D: volumetric medical image classification for rectal cancer staging [62.346649719614]
  preoperative discrimination between T2 and T3 stages is arguably both the most challenging and clinically significant task for rectal cancer treatment. We present a volumetric convolutional neural network to accurately discriminate T2 from T3 stage rectal cancer with rectal MR volumes.
  arXiv  Detail & Related papers  (2022-09-13T07:10:14Z)
• Perfusion imaging in deep prostate cancer detection from mp-MRI: can we take advantage of it? [0.0]
  We evaluate strategies to integrate information from perfusion imaging in deep neural architectures. Perfusion maps from dynamic contrast enhanced MR exams are shown to positively impact segmentation and grading performance of PCa lesions.
  arXiv  Detail & Related papers  (2022-07-06T07:55:46Z)
• Severity classification in cases of Collagen VI-related myopathy with Convolutional Neural Networks and handcrafted texture features [0.34998703934432684]
  Three methods are proposed to classify target muscles in Collagen VI-related myopathy cases. The best results were obtained with the hybrid model, resulting in a global accuracy of 93.8%.
  arXiv  Detail & Related papers  (2022-02-28T15:09:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.