DiffAtlas: GenAI-fying Atlas Segmentation via Image-Mask Diffusion

• URL: http://arxiv.org/abs/2503.06748v1
• Date: Sun, 09 Mar 2025 20:06:40 GMT
• Title: DiffAtlas: GenAI-fying Atlas Segmentation via Image-Mask Diffusion
• Authors: Hantao Zhang, Yuhe Liu, Jiancheng Yang, Weidong Guo, Xinyuan Wang, Pascal Fua,
• Abstract summary: We introduce DiffAtlas, a novel generative framework that models both images and masks through diffusion during training.<n>During testing, the model is guided to generate a specific target image-mask pair, from which the corresponding mask is obtained.<n>Our approach outperforms existing methods, particularly in limited-data and zero-shot modality segmentation.
• Score: 47.43502676919903
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate medical image segmentation is crucial for precise anatomical delineation. Deep learning models like U-Net have shown great success but depend heavily on large datasets and struggle with domain shifts, complex structures, and limited training samples. Recent studies have explored diffusion models for segmentation by iteratively refining masks. However, these methods still retain the conventional image-to-mask mapping, making them highly sensitive to input data, which hampers stability and generalization. In contrast, we introduce DiffAtlas, a novel generative framework that models both images and masks through diffusion during training, effectively ``GenAI-fying'' atlas-based segmentation. During testing, the model is guided to generate a specific target image-mask pair, from which the corresponding mask is obtained. DiffAtlas retains the robustness of the atlas paradigm while overcoming its scalability and domain-specific limitations. Extensive experiments on CT and MRI across same-domain, cross-modality, varying-domain, and different data-scale settings using the MMWHS and TotalSegmentator datasets demonstrate that our approach outperforms existing methods, particularly in limited-data and zero-shot modality segmentation. Code is available at https://github.com/M3DV/DiffAtlas.

Related papers

• PathSegDiff: Pathology Segmentation using Diffusion model representations [63.20694440934692]
  We propose PathSegDiff, a novel approach for histopathology image segmentation that leverages Latent Diffusion Models (LDMs) as pre-trained featured extractors. Our method utilizes a pathology-specific LDM, guided by a self-supervised encoder, to extract rich semantic information from H&E stained histopathology images. Our experiments demonstrate significant improvements over traditional methods on the BCSS and GlaS datasets.
  arXiv  Detail & Related papers  (2025-04-09T14:58:21Z)
• Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation [54.96563068182733]
  We propose Modality Adaptation with text-to-image Diffusion Models (MADM) for semantic segmentation task. MADM utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model's cross-modality capabilities. We show that MADM achieves state-of-the-art adaptation performance across various modality tasks, including images to depth, infrared, and event modalities.
  arXiv  Detail & Related papers  (2024-10-29T03:49:40Z)
• HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation [16.906987804797975]
  HiDiff is a hybrid diffusion framework for medical image segmentation. It can synergize the strengths of existing discriminative segmentation models and new generative diffusion models. It excels at segmenting small objects and generalizing to new datasets.
  arXiv  Detail & Related papers  (2024-07-03T23:59:09Z)
• Pathological Primitive Segmentation Based on Visual Foundation Model with Zero-Shot Mask Generation [3.5177988631063486]
  We present a novel approach that adapts pre-trained natural image encoders of SAM for detection-based region proposals. The entire base framework, SAM, requires no additional training or fine-tuning but could produce an end-to-end result for two fundamental segmentation tasks in pathology.
  arXiv  Detail & Related papers  (2024-04-12T16:29:49Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models [52.3015009878545]
  We develop an image segmentor capable of generating fine-grained segmentation maps without any additional training. Our framework identifies semantic correspondences between image pixels and spatial locations of low-dimensional feature maps. In extensive experiments, the produced segmentation maps are demonstrated to be well delineated and capture detailed parts of the images.
  arXiv  Detail & Related papers  (2024-01-22T07:34:06Z)
• DG-TTA: Out-of-domain medical image segmentation through Domain Generalization and Test-Time Adaptation [43.842694540544194]
  We propose to combine domain generalization and test-time adaptation to create a highly effective approach for reusing pre-trained models in unseen target domains. We demonstrate that our method, combined with pre-trained whole-body CT models, can effectively segment MR images with high accuracy.
  arXiv  Detail & Related papers  (2023-12-11T10:26:21Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation [86.44683367028914]
  Aerial imagery segmentation has some unique challenges, the most critical one among which lies in foreground-background imbalance. We propose Adaptive Focus Framework (AF$), which adopts a hierarchical segmentation procedure and focuses on adaptively utilizing multi-scale representations. AF$ has significantly improved the accuracy on three widely used aerial benchmarks, as fast as the mainstream method.
  arXiv  Detail & Related papers  (2022-02-18T10:14:45Z)
• Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation [10.634870214944055]
  In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. We propose a novel volumetric self-supervised learning for data augmentation capable of synthesizing volumetric image-segmentation pairs. Our work's central tenet benefits from a combined view of one-shot generative learning and the proposed self-supervised training strategy.
  arXiv  Detail & Related papers  (2021-10-05T15:28:42Z)
• Cross-Domain Segmentation with Adversarial Loss and Covariate Shift for Biomedical Imaging [2.1204495827342438]
  This manuscript aims to implement a novel model that can learn robust representations from cross-domain data by encapsulating distinct and shared patterns from different modalities. The tests on CT and MRI liver data acquired in routine clinical trials show that the proposed model outperforms all other baseline with a large margin.
  arXiv  Detail & Related papers  (2020-06-08T07:35:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.