Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation

• URL: http://arxiv.org/abs/2604.10702v3
• Date: Sat, 18 Apr 2026 13:05:33 GMT
• Title: Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation
• Abstract summary: Multi-weighted prostate MRI combines T2-parametric (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences.<n>In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W.<n>We propose Modality-Isolated Gated Fusion (MIGF) to maintain separate modality-specific encoding streams before a learned gating stage.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-parametric prostate MRI combines T2-weighted (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences for non-invasive detection of clinically significant prostate cancer. In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W, making robust fusion of these channels the clinically relevant requirement. We propose Modality-Isolated Gated Fusion (MIGF), an architecture-agnostic module that maintains separate modality-specific encoding streams before a learned gating stage, combined with modality dropout training to enforce compensation under incomplete inputs. We benchmark six backbones and assess MIGF-equipped models under seven missing-modality and artifact scenarios on PI-CAI (1,500 studies, fold-0 split, five seeds). MIGF improved ideal-scenario Ranking Score for UNet, nnUNet, and Mamba by 2.8%, 4.6%, and 13.4%; the best model (MIGFNet-nnUNet, gating + ModDrop, no deep supervision) achieved 0.7304 +/- 0.056. MIGF primarily improved tolerance to HBV/ADC degradation, while missing T2W remained a shared failure mode across all architectures. Mechanistic analysis shows that robustness gains arise from strict modality isolation and dropout-driven compensation rather than adaptive per-sample quality routing: the gate converged to a stable modality prior, and deep supervision was beneficial only for the largest backbone. External evaluation on Prostate158 (n=158) revealed substantial domain shift driven primarily by ADC map incompatibility across institutions; modality-isolation analysis confirmed that removing ADC improved external performance, identifying computed diffusion maps as inherently less portable than raw-acquired sequences.

Related papers

• A cross-modal generative model for incomplete and degraded prostate MRI with multicentre clinical validation [48.14053205497089]
  MSCNet is a sequence-conditioned cross-modal generative framework for reconstructing unavailable contrasts and restoring degraded acquisitions.<n>Overall image quality met the prespecified non-inferiority criterion for DWI, ADC and T2W completion, but not T1W.
  arXiv  Detail & Related papers  (2026-08-17T08:11:25Z)
• Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification [12.244907298771585]
  We present cUPMI, a class-conditional Gaussian augmentation of a combiner's log-probability meta-features.<n>We find cUPMI adds limited value to properly regularized L2-logistic binary classification stacks, but consistently regularizes higher-capacity tree combiners.<n>Its cleanest ordinal benefit appears for XGBoost on an 8-stream radiomics task.
  arXiv  Detail & Related papers  (2026-08-11T22:17:32Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities [54.87947751720332]
  We propose a novel segmentation model D3Seg to maintain stable performance under missing-modality settings.<n>The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing modality configurations.
  arXiv  Detail & Related papers  (2026-05-21T09:55:42Z)
• Plug-and-Play Logit Fusion for Heterogeneous Pathology Foundation Models [28.682541563931593]
  We present LogitProd, a lightweight and novel model fusion strategy.<n>It treats independently trained FM-based predictors as fixed experts and learns sample-adaptive fusion weights over slide-level outputs.<n>LogitProd ranks first on 20/22 tasks and improves the average performance across all tasks by 3% over the strongest single expert.
  arXiv  Detail & Related papers  (2026-04-09T04:10:54Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• Automated Prostate Gland Segmentation in MRI Using nnU-Net [0.14777718769290524]
  We propose a dedicated deep learning-based approach for automatic prostate gland segmentation using the nnUNet v2 framework.<n>The proposed model achieved a mean Dice score of 0.96 +/- 0.00 in cross-validation and 0.82 on the external test set, demonstrating strong generalization despite domain shift.
  arXiv  Detail & Related papers  (2026-04-02T12:28:03Z)
• Maximizing T2-Only Prostate Cancer Localization from Expected Diffusion Weighted Imaging [68.57480103120561]
  Multiparametric MRI is increasingly recommended as a first-line noninvasive approach to detect and localize prostate cancer.<n>Early machine learning attempts using only T2w images have shown promising diagnostic performance in segmenting radiologist-annotated lesions.<n>This work investigates an arguably more challenging application using only T2w at inference, but to localize individual cancers based on independent histopathology labels.
  arXiv  Detail & Related papers  (2026-04-01T14:50:18Z)
• Less is More in Semantic Space: Intrinsic Decoupling via Clifford-M for Fundus Image Classification [13.23226865033351]
  We propose Clifford-M, a lightweight backbone that replaces feed-forward expansion and frequency-splitting modules with sparse geometric interaction.<n>Without pre-training, Clifford-M achieves a mean AUC-ROC of 0.8142 and a mean macro-F1 of 0.5481 on ODIR-5K using only 0.85M parameters.
  arXiv  Detail & Related papers  (2026-03-21T13:00:05Z)
• UniField: A Unified Field-Aware MRI Enhancement Framework [45.03230466428695]
  methodname is a unified framework integrating multiple modalities and enhancement tasks.<n>We exploit comprehensive 3D volumetric information by leveraging pre-trained 3D foundation models.<n>We organize and publicly release a comprehensive paired multi-field MRI dataset.
  arXiv  Detail & Related papers  (2026-03-10T05:45:12Z)
• Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognition [60.20529806857076]
  Multimodal emotion recognition fuses cues such as text, video, and audio to understand individual emotional states.<n>Prior methods face two main limitations: mechanically relying on independent unimodal performance, and coarse-grained fusion conflicting with the fine-grained representations required by emotion tasks.<n>We propose the Complementarity-Supervised Multi-Band Expert Network, named Atsuko, to model fine-grained complementary features via multi-scale band decomposition and expert collaboration.
  arXiv  Detail & Related papers  (2026-03-07T03:58:48Z)
• NodMAISI: Nodule-Oriented Medical AI for Synthetic Imaging [21.25110830915782]
  We introduce NodMAISI, an anatomically constrained, nodule-oriented CT synthesis and augmentation framework trained on a unified multi-source cohort.<n>Across six public test datasets, NodMAISI improved distributional fidelity relative to MAISI-v2 (real-to-synthetic FID range 1.18 to 2.99 vs 1.69 to 5.21)<n>NodMAISI augmentation improved AUC by 0.07 to 0.11 at =20% clinical data and by 0.12 to 0.21 at 10%, consistently narrowing the performance gap under data scarcity.
  arXiv  Detail & Related papers  (2025-12-19T20:11:30Z)
• PathCo-LatticE: Pathology-Constrained Lattice-Of Experts Framework for Fully-supervised Few-Shot Cardiac MRI Segmentation [0.0]
  Few-shot learning mitigates data scarcity in cardiac MRI segmentation.<n>PathCo-LatticE replaces unlabeled data with pathology-guided synthetic supervision.
  arXiv  Detail & Related papers  (2025-12-10T15:59:43Z)
• Cancer-Net PCa-MultiSeg: Multimodal Enhancement of Prostate Cancer Lesion Segmentation Using Synthetic Correlated Diffusion Imaging [55.62977326180104]
  Current deep learning approaches for prostate cancer lesion segmentation achieve limited performance.<n>We investigate synthetic correlated diffusion imaging (CDI$s$) as an enhancement to standard diffusion-based protocols.<n>Our results establish validated integration pathways for CDI$s$ as a practical drop-in enhancement for PCa lesion segmentation tasks.
  arXiv  Detail & Related papers  (2025-11-11T04:16:12Z)
• Improved mmFormer for Liver Fibrosis Staging via Missing-Modality Compensation [8.687370165870613]
  We propose a multimodal MRI classification model based on the mmFormer architecture with an adaptive module for handling arbitrary combinations of missing modalities.<n>This method is evaluated on the test set of Comprehensive Analysis & Computing of REal-world medical images (CARE 2025 challenge)<n>For Cirrhosis Detection and Substantial Fibrosis Detection on in-distribution vendors, our model obtains accuracies of 66.67%, and 74.17%, and corresponding area under the curve (AUC) scores of 71.73% and 68.48%, respectively.
  arXiv  Detail & Related papers  (2025-09-19T21:31:05Z)
• UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation [104.59740403500132]
  Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance.<n>We propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC)<n>Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels.
  arXiv  Detail & Related papers  (2025-09-19T17:29:25Z)
• impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction [75.43342771863837]
  We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy.<n>It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches.<n>Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets.
  arXiv  Detail & Related papers  (2025-08-08T10:01:16Z)
• A Foundation Model Framework for Multi-View MRI Classification of Extramural Vascular Invasion and Mesorectal Fascia Invasion in Rectal Cancer [3.9216006287929086]
  Extramural vascular invasion (EVI) and mesorectal fascia invasion (MFI) is pivotal for risk-stratified management of rectal cancer.<n>Foundation model features, harmonization, and multi-view fusion significantly enhance diagnostic performance in rectal MRI.
  arXiv  Detail & Related papers  (2025-05-23T16:04:27Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Segmentation of the Myocardium on Late-Gadolinium Enhanced MRI based on 2.5 D Residual Squeeze and Excitation Deep Learning Model [55.09533240649176]
  The aim of this work is to develop an accurate automatic segmentation method based on deep learning models for the myocardial borders on LGE-MRI. A total number of 320 exams (with a mean number of 6 slices per exam) were used for training and 28 exams used for testing. The performance analysis of the proposed ensemble model in the basal and middle slices was similar as compared to intra-observer study and slightly lower at apical slices.
  arXiv  Detail & Related papers  (2020-05-27T20:44:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.