MRGen: Diffusion-based Controllable Data Engine for MRI Segmentation towards Unannotated Modalities

• URL: http://arxiv.org/abs/2412.04106v1
• Date: Wed, 04 Dec 2024 16:34:22 GMT
• Title: MRGen: Diffusion-based Controllable Data Engine for MRI Segmentation towards Unannotated Modalities
• Authors: Haoning Wu, Ziheng Zhao, Ya Zhang, Weidi Xie, Yanfeng Wang,
• Abstract summary: This paper investigates a new paradigm for leveraging generative models in medical applications.<n>We propose a diffusion-based data engine, termed MRGen, which enables generation conditioned on text prompts and masks.
• Score: 59.61465292965639
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation has recently demonstrated impressive progress with deep neural networks, yet the heterogeneous modalities and scarcity of mask annotations limit the development of segmentation models on unannotated modalities. This paper investigates a new paradigm for leveraging generative models in medical applications: controllably synthesizing data for unannotated modalities, without requiring registered data pairs. Specifically, we make the following contributions in this paper: (i) we collect and curate a large-scale radiology image-text dataset, MedGen-1M, comprising modality labels, attributes, region, and organ information, along with a subset of organ mask annotations, to support research in controllable medical image generation; (ii) we propose a diffusion-based data engine, termed MRGen, which enables generation conditioned on text prompts and masks, synthesizing MR images for diverse modalities lacking mask annotations, to train segmentation models on unannotated modalities; (iii) we conduct extensive experiments across various modalities, illustrating that our data engine can effectively synthesize training samples and extend MRI segmentation towards unannotated modalities.

Related papers

• SeLIP: Similarity Enhanced Contrastive Language Image Pretraining for Multi-modal Head MRI [6.714491893348051]
  We propose to develop a foundation model for multi-model head MRI by using contrastive learning on the images and the corresponding radiology findings. Our proposed similarity enhanced contrastive language image pretraining (SeLIP) is able to effectively extract more useful features.
  arXiv  Detail & Related papers  (2025-03-25T16:09:45Z)
• An Ensemble Approach for Brain Tumor Segmentation and Synthesis [0.12777007405746044]
  The integration of machine learning in magnetic resonance imaging (MRI) is proving to be incredibly effective. Deep learning models utilize multiple layers of processing to capture intricate details of complex data. We propose a deep learning framework that ensembles state-of-the-art architectures to achieve accurate segmentation.
  arXiv  Detail & Related papers  (2024-11-26T17:28:51Z)
• Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation [51.28453192441364]
  Multimodal brain magnetic resonance (MR) imaging is indispensable in neuroscience and neurology. Current MR image synthesis approaches are typically trained on independent datasets for specific tasks. We present TUMSyn, a Text-guided Universal MR image Synthesis model, which can flexibly generate brain MR images.
  arXiv  Detail & Related papers  (2024-09-25T11:14:47Z)
• Discriminative Hamiltonian Variational Autoencoder for Accurate Tumor Segmentation in Data-Scarce Regimes [2.8498944632323755]
  We propose an end-to-end hybrid architecture for medical image segmentation. We use Hamiltonian Variational Autoencoders (HVAE) and a discriminative regularization to improve the quality of generated images. Our architecture operates on a slice-by-slice basis to segment 3D volumes, capitilizing on the richly augmented dataset.
  arXiv  Detail & Related papers  (2024-06-17T15:42:08Z)
• 3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes [2.8498944632323755]
  We propose a novel slice-based latent diffusion architecture to address the complexities of volumetric data generation. This approach extends the joint distribution modeling of medical images and their associated masks, allowing a simultaneous generation of both under data-scarce regimes. Our architecture can be conditioned by tumor characteristics, including size, shape, and relative position, thereby providing a diverse range of tumor variations.
  arXiv  Detail & Related papers  (2024-06-08T09:53:45Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models [69.9178140563928]
  Colonoscopy analysis is essential for assisting clinical diagnosis and treatment. The scarcity of annotated data limits the effectiveness and generalization of existing methods. We propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks.
  arXiv  Detail & Related papers  (2023-09-03T07:55:46Z)
• Controllable Mind Visual Diffusion Model [58.83896307930354]
  Brain signal visualization has emerged as an active research area, serving as a critical interface between the human visual system and computer vision models. We propose a novel approach, referred to as Controllable Mind Visual Model Diffusion (CMVDM) CMVDM extracts semantic and silhouette information from fMRI data using attribute alignment and assistant networks. We then leverage a control model to fully exploit the extracted information for image synthesis, resulting in generated images that closely resemble the visual stimuli in terms of semantics and silhouette.
  arXiv  Detail & Related papers  (2023-05-17T11:36:40Z)
• M-GenSeg: Domain Adaptation For Target Modality Tumor Segmentation With Annotation-Efficient Supervision [4.023899199756184]
  M-GenSeg is a new semi-supervised generative training strategy for cross-modality tumor segmentation. We evaluate the performance on a brain tumor segmentation dataset composed of four different contrast sequences. Unlike the prior art, M-GenSeg also introduces the ability to train with a partially annotated source modality.
  arXiv  Detail & Related papers  (2022-12-14T15:19:06Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Realistic Adversarial Data Augmentation for MR Image Segmentation [17.951034264146138]
  We propose an adversarial data augmentation method for training neural networks for medical image segmentation. Our model generates plausible and realistic signal corruptions, which models the intensity inhomogeneities caused by a common type of artefacts in MR imaging: bias field. We show that such an approach can improve the ability generalization and robustness of models as well as provide significant improvements in low-data scenarios.
  arXiv  Detail & Related papers  (2020-06-23T20:43:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.