Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation

• URL: http://arxiv.org/abs/2606.08421v1
• Date: Sun, 07 Jun 2026 02:41:24 GMT
• Title: Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation
• Abstract summary: We propose a synthesis-centric, segmentation-assisted closed-loop framework with retrieval augmentation synthesis.<n>The proposed method is effective in handling medical brain images synthesis tasks and shows superior performance compared to previous methods.
• Score: 7.213968557982194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-contrast brain MRI provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-contrast images. While current approaches excel in image synthesis, they often struggle to synthesize critical tumor regions and exploit contextual information in multi-contrast brain MRI effectively. To address this issue, we propose a synthesis-centric, segmentation-assisted closed-loop framework with retrieval augmentation synthesis. Our method overall takes a generative adversarial architecture, which aims to synthesize missing contrasts from any combination of available ones with a single model. To explicitly capture tumor semantics and focus synthesis on tumor regions, we add an auxiliary segmentation branch that predicts tumor masks and feeds them back as semantic conditioning in synthesis branch, thereby learning tumor-aware representations in the model and improving synthesis fidelity. Furthermore, we propose a dual-bank retrieval augmentation strategy. It dynamically queries two external knowledge bases, namely a tumor masks memory bank for crucial tumor context and cross-image contrast feature memory bank for global style information, to augment synthesis. Verified on two public multi-contrast magnetic resonance brain datasets: BraTs2020 and UCSF-BMSR, the proposed method is effective in handling medical brain images synthesis tasks and shows superior performance compared to previous methods. Code is available at:https://github.com/iBizzard/SSCF.git

Related papers

• Brain-WM: Brain Glioblastoma World Model [56.45393372490125]
  We present Brain-WM, a pioneering GBM world model that unifies next-step treatment prediction and future MRI generation.<n>Brain-WM encodes latent dynamics into a shared latent space for joint autoregressive treatment prediction and flow-based future MRI generation.<n>Ultimately, Brain-WM offers a robust clinical sandbox for optimizing patient healthcare.
  arXiv  Detail & Related papers  (2026-03-08T09:54:08Z)
• Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training [4.672822120456554]
  Most segmentation models for multimodal brain MRI are restricted to fixed modalities.<n>Some models generalize to unseen modalities but may lose modality-specific information.<n>This work aims to develop a model that can perform inference on data that contain image modalities unseen during training.
  arXiv  Detail & Related papers  (2025-09-11T09:25:30Z)
• BrainMRDiff: A Diffusion Model for Anatomically Consistent Brain MRI Synthesis [11.73947657846282]
  BrainMRDiff is a novel topology-preserving, anatomy-guided diffusion model for brain MRI.<n>To achieve this, we introduce two key modules: Tumor+Structure Aggregation (TSA) and Topology-Guided Anatomy Preservation (TGAP)<n>TSA integrates diverse anatomical structures with tumor information, forming a comprehensive conditioning mechanism for the diffusion process.<n>Results demonstrate that BrainMRDiff surpasses existing baselines, achieving performance improvements of 23.33% on the BraTS-AG dataset and 33.33% on the BraTS-Met dataset.
  arXiv  Detail & Related papers  (2025-04-06T16:16:50Z)
• Two-Stage Approach for Brain MR Image Synthesis: 2D Image Synthesis and 3D Refinement [1.5683566370372715]
  It is crucial to synthesize the missing MR images that reflect the unique characteristics of the absent modality with precise tumor representation.<n>We propose a two-stage approach that first synthesizes MR images from 2D slices using a novel intensity encoding method and then refines the synthesized MRI.
  arXiv  Detail & Related papers  (2024-10-14T08:21:08Z)
• Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation [51.28453192441364]
  Multimodal brain magnetic resonance (MR) imaging is indispensable in neuroscience and neurology. Current MR image synthesis approaches are typically trained on independent datasets for specific tasks. We present TUMSyn, a Text-guided Universal MR image Synthesis model, which can flexibly generate brain MR images.
  arXiv  Detail & Related papers  (2024-09-25T11:14:47Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Generative Adversarial Networks for Brain Images Synthesis: A Review [2.609784101826762]
  In medical imaging, image synthesis is the estimation process of one image (sequence, modality) from another image (sequence, modality) generative adversarial network (GAN) as one of the most popular generative-based deep learning methods. We summarized the recent developments of GANs for cross-modality brain image synthesis including CT to PET, CT to MRI, MRI to PET, and vice versa.
  arXiv  Detail & Related papers  (2023-05-16T17:28:06Z)
• The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn) [9.082208613256295]
  We present the establishment of the Brain MR Image Synthesis Benchmark (BraSyn) in conjunction with the Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2023. The primary objective of this challenge is to evaluate image synthesis methods that can realistically generate missing MRI modalities when multiple available images are provided.
  arXiv  Detail & Related papers  (2023-05-15T20:49:58Z)
• Joint fMRI Decoding and Encoding with Latent Embedding Alignment [77.66508125297754]
  We introduce a unified framework that addresses both fMRI decoding and encoding. Our model concurrently recovers visual stimuli from fMRI signals and predicts brain activity from images within a unified framework.
  arXiv  Detail & Related papers  (2023-03-26T14:14:58Z)
• BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding [51.911473457195555]
  BrainCLIP is a task-agnostic fMRI-based brain decoding model. It bridges the modality gap between brain activity, image, and text. BrainCLIP can reconstruct visual stimuli with high semantic fidelity.
  arXiv  Detail & Related papers  (2023-02-25T03:28:54Z)
• Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution [55.52779466954026]
  Multi-contrast super-resolution (SR) reconstruction is promising to yield SR images with higher quality. Existing methods lack effective mechanisms to match and fuse these features for better reconstruction. We propose a novel network to address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques.
  arXiv  Detail & Related papers  (2022-03-26T01:42:59Z)
• Cross-Modality Neuroimage Synthesis: A Survey [71.27193056354741]
  Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. An alternative solution is to explore unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data.
  arXiv  Detail & Related papers  (2022-02-14T19:29:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.