SegDiff: Image Segmentation with Diffusion Probabilistic Models
- URL: http://arxiv.org/abs/2112.00390v1
- Date: Wed, 1 Dec 2021 10:17:25 GMT
- Title: SegDiff: Image Segmentation with Diffusion Probabilistic Models
- Authors: Tomer Amit, Eliya Nachmani, Tal Shaharbany, Lior Wolf
- Abstract summary: Diffusion Probabilistic Methods are employed for state-of-the-art image generation.
We present a method for extending such models for performing image segmentation.
The method learns end-to-end, without relying on a pre-trained backbone.
- Score: 81.16986859755038
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Diffusion Probabilistic Methods are employed for state-of-the-art image
generation. In this work, we present a method for extending such models for
performing image segmentation. The method learns end-to-end, without relying on
a pre-trained backbone. The information in the input image and in the current
estimation of the segmentation map is merged by summing the output of two
encoders. Additional encoding layers and a decoder are then used to iteratively
refine the segmentation map using a diffusion model. Since the diffusion model
is probabilistic, it is applied multiple times and the results are merged into
a final segmentation map. The new method obtains state-of-the-art results on
the Cityscapes validation set, the Vaihingen building segmentation benchmark,
and the MoNuSeg dataset.
Related papers
- Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models [55.2480439325792]
We propose a new pretext task, which is to perform simultaneously image denoising and mask prediction on the first domain.
We show that fine-tuning a model pretrained using this approach leads to better results than fine-tuning a similar model trained using either supervised or unsupervised pretraining.
arXiv Detail & Related papers (2024-08-06T20:19:06Z) - HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation [16.906987804797975]
HiDiff is a hybrid diffusion framework for medical image segmentation.
It can synergize the strengths of existing discriminative segmentation models and new generative diffusion models.
It excels at segmenting small objects and generalizing to new datasets.
arXiv Detail & Related papers (2024-07-03T23:59:09Z) - EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models [52.3015009878545]
We develop an image segmentor capable of generating fine-grained segmentation maps without any additional training.
Our framework identifies semantic correspondences between image pixels and spatial locations of low-dimensional feature maps.
In extensive experiments, the produced segmentation maps are demonstrated to be well delineated and capture detailed parts of the images.
arXiv Detail & Related papers (2024-01-22T07:34:06Z) - SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete
Diffusion Process [102.18226145874007]
We propose a model-agnostic solution called SegRefiner to enhance the quality of object masks produced by different segmentation models.
SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process.
It consistently improves both the segmentation metrics and boundary metrics across different types of coarse masks.
arXiv Detail & Related papers (2023-12-19T18:53:47Z) - Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for
Pixel-Level Semantic Segmentation [6.82236459614491]
We propose a novel method for generating pixel-level semantic segmentation labels using the text-to-image generative model Stable Diffusion.
By utilizing the text prompts, cross-attention, and self-attention of SD, we introduce three new techniques: class-prompt appending, class-prompt cross-attention, and self-attention exponentiation.
These techniques enable us to generate segmentation maps corresponding to synthetic images.
arXiv Detail & Related papers (2023-09-25T17:19:26Z) - DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion
Models [61.906934570771256]
We present a generic dataset generation model that can produce diverse synthetic images and perception annotations.
Our method builds upon the pre-trained diffusion model and extends text-guided image synthesis to perception data generation.
We show that the rich latent code of the diffusion model can be effectively decoded as accurate perception annotations using a decoder module.
arXiv Detail & Related papers (2023-08-11T14:38:11Z) - Stochastic Segmentation with Conditional Categorical Diffusion Models [3.8168879948759953]
We propose a conditional categorical diffusion model (CCDM) for semantic segmentation based on Denoising Diffusion Probabilistic Models.
Our results show that CCDM achieves state-of-the-art performance on LIDC, and outperforms established baselines on the classical segmentation dataset Cityscapes.
arXiv Detail & Related papers (2023-03-15T19:16:47Z) - Variational multichannel multiclass segmentation using unsupervised
lifting with CNNs [0.0]
We implement a flexible multiclass segmentation method that divides a given image into $K$ different regions.
We use convolutional neural networks (CNNs) targeting a pre-decomposition of the image.
Special emphasis is given to the extraction of informative feature maps serving as a starting point for the segmentation.
arXiv Detail & Related papers (2023-02-04T18:01:47Z) - Diffusion Models for Implicit Image Segmentation Ensembles [1.444701913511243]
We present a novel semantic segmentation method based on diffusion models.
By modifying the training and sampling scheme, we show that diffusion models can perform lesion segmentation of medical images.
Compared to state-of-the-art segmentation models, our approach yields good segmentation results and, additionally, meaningful uncertainty maps.
arXiv Detail & Related papers (2021-12-06T16:28:15Z) - BoundarySqueeze: Image Segmentation as Boundary Squeezing [104.43159799559464]
We propose a novel method for fine-grained high-quality image segmentation of both objects and scenes.
Inspired by dilation and erosion from morphological image processing techniques, we treat the pixel level segmentation problems as squeezing object boundary.
Our method yields large gains on COCO, Cityscapes, for both instance and semantic segmentation and outperforms previous state-of-the-art PointRend in both accuracy and speed under the same setting.
arXiv Detail & Related papers (2021-05-25T04:58:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.