Brain MR Image Synthesis with Multi-contrast Self-attention GAN

• URL: http://arxiv.org/abs/2604.00070v1
• Date: Tue, 31 Mar 2026 13:43:53 GMT
• Title: Brain MR Image Synthesis with Multi-contrast Self-attention GAN
• Abstract summary: 3D-MC-SAGAN is a unified 3D multi-contrast synthesis framework.<n>It generates high-fidelity missing modalities from a single T2 input while preserving tumour characteristics.<n>It achieves state-of-the-art quantitative performance and generates visually coherent, anatomically plausible contrasts.
• Score: 0.2291770711277359
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate and complete multi-modal Magnetic Resonance Imaging (MRI) is essential for neuro-oncological assessment, as each contrast provides complementary anatomical and pathological information. However, acquiring all modalities (e.g., T1c, T1n, T2, T2f) for every patient is often impractical due to time, cost, and patient discomfort, potentially limiting comprehensive tumour evaluation. We propose 3D-MC-SAGAN (3D Multi-Contrast Self-Attention generative adversarial network), a unified 3D multi-contrast synthesis framework that generates high-fidelity missing modalities from a single T2 input while explicitly preserving tumour characteristics. The model employs a multi-scale 3D encoder-decoder generator with residual connections and a novel Memory-Bounded Hybrid Attention (MBHA) block to capture long-range dependencies efficiently, and is trained with a WGAN-GP critic and an auxiliary contrast-conditioning branch to produce T2f, T1n, and T1c volumes within a single unified network. A frozen 3D U-Net-based segmentation module introduces a segmentation-consistency constraint to preserve lesion morphology. The composite objective integrates adversarial, reconstruction, perceptual, structural similarity, contrast-classification, and segmentation-guided losses to align global realism with tumour-preserving structure. Extensive evaluation on 3D brain MRI datasets demonstrates that 3D-MC-SAGAN achieves state-of-the-art quantitative performance and generates visually coherent, anatomically plausible contrasts with improved distribution-level realism. Moreover, it maintains tumour segmentation accuracy comparable to fully acquired multi-modal inputs, highlighting its potential to reduce acquisition burden while preserving clinically meaningful information.

Related papers

• Recover Semantics First, Generate Better: Improved Latent Modeling for 3D MRI Reconstruction and Cross-Contrast Synthesis [21.111566412869713]
  We propose a semantics-first latent modeling framework for 3D MRI reconstruction and cross-contrast synthesis.<n>We introduce a Latent Harmonization (LHE) to capture global anatomical dependencies, ensuring coherent representations.<n>We further design a Semantic Recovery Block (SRB) that injects high-level priors from a self-supervised semantic teacher, enhancing contrast-aware separability in the latent space.
  arXiv  Detail & Related papers  (2026-06-16T14:41:15Z)
• D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities [54.87947751720332]
  We propose a novel segmentation model D3Seg to maintain stable performance under missing-modality settings.<n>The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing modality configurations.
  arXiv  Detail & Related papers  (2026-05-21T09:55:42Z)
• Degradation-Aware Blur-Segmentation of Brain Tumor [6.772154747032075]
  Degradation-Aware Blur-Segmentation Net (DABSeg) is a synchronous deblurring 3D multimodal MRI segmentation network.<n>We propose a feature-domain motion-deblurring stem to compensate for blur and rebalance intensity.<n>System embeds a blur-aware cross-modal cross-attention module and multi-scale residual aggregation to yield effective modality complementarity.
  arXiv  Detail & Related papers  (2026-05-15T06:47:37Z)
• PHASOR: Anatomy- and Phase-Consistent Volumetric Diffusion for CT Virtual Contrast Enhancement [60.81732730684265]
  This paper introduces PHASOR, a volumetric diffusion framework for high-fidelity virtual contrast enhancement (VCE)<n>By treating CT volumes as coherent sequences, we leverage a video diffusion model to enhance structural coherence and volumetric accuracy.<n>To ensure anatomy-phase consistent synthesis, we introduce two complementary modules. First, anatomy-routed mixture-of-experts (AR-MoE) anchors distinct enhancement patterns to anatomical semantics, with organ-specific memory to capture salient details.<n>Second, intensity-phase aware representation alignment (IP-REPA) highlights intricate contrast signals while mitigating the impact of imperfect spatial alignment.
  arXiv  Detail & Related papers  (2026-04-01T15:57:18Z)
• MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models [59.180043227905294]
  We propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework for efficient 3D medical image understanding.<n>We show that MedPruner enables models such as MedGemma to maintain or even exceed their original performance while retaining fewer than 5% of visual tokens.
  arXiv  Detail & Related papers  (2026-03-12T07:37:00Z)
• UniField: A Unified Field-Aware MRI Enhancement Framework [45.03230466428695]
  methodname is a unified framework integrating multiple modalities and enhancement tasks.<n>We exploit comprehensive 3D volumetric information by leveraging pre-trained 3D foundation models.<n>We organize and publicly release a comprehensive paired multi-field MRI dataset.
  arXiv  Detail & Related papers  (2026-03-10T05:45:12Z)
• Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis [9.857855424798732]
  We propose CoPeDiT, a latent diffusion model equipped with completeness perception for unified synthesis of 3D MRIs.<n>CoPeDiT significantly outperforms state-of-the-art methods, achieving superior robustness, generalizability, and flexibility.
  arXiv  Detail & Related papers  (2026-02-20T18:05:39Z)
• Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification [69.87877580725768]
  Multimodal Visual Surrogate Compression (MVSC) learns to compress and adapt large 3D sMRI volumes into compact 2D features.<n>MVSC has two key components: a Volume Context that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner.
  arXiv  Detail & Related papers  (2026-01-29T13:05:46Z)
• RL-U$^2$Net: A Dual-Branch UNet with Reinforcement Learning-Assisted Multimodal Feature Fusion for Accurate 3D Whole-Heart Segmentation [0.624829068285122]
  We propose a dual-branch U-Net architecture enhanced by reinforcement learning for feature alignment.<n>The model employs a dual-branch U-shaped network to process CT and MRI patches in parallel, and introduces a novel RL-XAlign module.<n> Experimental results on the publicly available MM-WHS 2017 dataset demonstrate that the proposed RL-U$2$Net outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-04T16:12:06Z)
• Multimodal Fusion at Three Tiers: Physics-Driven Data Generation and Vision-Language Guidance for Brain Tumor Segmentation [8.695435245976482]
  This paper proposes a three-tier fusion architecture that achieves precise brain tumor segmentation.<n>The method processes information progressively at the pixel, feature, and semantic levels.<n>We validated the method on the Brain Tumor (BraTS) 2020, 2021 and 2023 datasets.
  arXiv  Detail & Related papers  (2025-07-14T06:32:59Z)
• ReCoGNet: Recurrent Context-Guided Network for 3D MRI Prostate Segmentation [11.248082139905865]
  We propose a hybrid architecture that models MRI sequences as annotated data.<n>Our method uses a deep, preserving pretrained DeepVLab3 backbone to extract high-level semantic features from each MRI slice and a recurrent convolutional head, built with ConvLSTM layers, to integrate information across slices.<n>Compared to state-of-the-art 2D and 3D segmentation models, our approach demonstrates superior performance in terms of precision, recall, Intersection over Union (IoU), Dice Similarity Coefficient (DSC) and robustness.
  arXiv  Detail & Related papers  (2025-06-24T14:56:55Z)
• Towards Synergistic Deep Learning Models for Volumetric Cirrhotic Liver Segmentation in MRIs [1.5228650878164722]
  Liver cirrhosis, a leading cause of global mortality, requires precise segmentation of ROIs for effective disease monitoring and treatment planning. Existing segmentation models often fail to capture complex feature interactions and generalize across diverse datasets. We propose a novel synergistic theory that leverages complementary latent spaces for enhanced feature interaction modeling.
  arXiv  Detail & Related papers  (2024-08-08T14:41:32Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• Self-calibrated convolution towards glioma segmentation [45.74830585715129]
  We evaluate self-calibrated convolutions in different parts of the nnU-Net network to demonstrate that self-calibrated modules in skip connections can significantly improve the enhanced-tumor and tumor-core segmentation accuracy.
  arXiv  Detail & Related papers  (2024-02-07T19:51:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.