Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution

• URL: http://arxiv.org/abs/2505.07071v1
• Date: Sun, 11 May 2025 17:45:05 GMT
• Title: Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution
• Authors: Zihang Liu, Zhenyu Zhang, Hao Tang,
• Abstract summary: Diffusion-based image super-resolution (SR) methods have demonstrated remarkable performance.<n>Recent advancements have introduced deterministic sampling processes that reduce inference from 15 iterative steps to a single step.<n>We propose SAMSR, a semantic-guided diffusion framework that incorporates semantic segmentation masks into the sampling process.
• Score: 13.187007344274662
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based image super-resolution (SR) methods have demonstrated remarkable performance. Recent advancements have introduced deterministic sampling processes that reduce inference from 15 iterative steps to a single step, thereby significantly improving the inference speed of existing diffusion models. However, their efficiency remains limited when handling complex semantic regions due to the single-step inference. To address this limitation, we propose SAMSR, a semantic-guided diffusion framework that incorporates semantic segmentation masks into the sampling process. Specifically, we introduce the SAM-Noise Module, which refines Gaussian noise using segmentation masks to preserve spatial and semantic features. Furthermore, we develop a pixel-wise sampling strategy that dynamically adjusts the residual transfer rate and noise strength based on pixel-level semantic weights, prioritizing semantically rich regions during the diffusion process. To enhance model training, we also propose a semantic consistency loss, which aligns pixel-wise semantic weights between predictions and ground truth. Extensive experiments on both real-world and synthetic datasets demonstrate that SAMSR significantly improves perceptual quality and detail recovery, particularly in semantically complex images. Our code is released at https://github.com/Liu-Zihang/SAMSR.

Related papers

• CAM-Seg: A Continuous-valued Embedding Approach for Semantic Image Generation [11.170848285659572]
  Autoencoder accuracy on segmentation mask using quantized embeddings is 8% lower than continuous-valued embeddings.<n>We propose a continuous-valued embedding framework for semantic segmentation.<n>Our approach eliminates the need for discrete latent representations while preserving fine-grained semantic details.
  arXiv  Detail & Related papers  (2025-03-19T18:06:54Z)
• HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior [62.04939047885834]
  We present HoliSDiP, a framework that leverages semantic segmentation to provide both precise textual and spatial guidance for Real-ISR.<n>Our method employs semantic labels as concise text prompts while introducing dense semantic guidance through segmentation masks and our proposed spatial-CLIP Map.
  arXiv  Detail & Related papers  (2024-11-27T15:22:44Z)
• SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution [46.85622647257876]
  We propose the SAM-DiffSR model, which can utilize the fine-grained structure information from SAM in the process of sampling noise to improve the image quality without additional computational cost during inference.<n> Experimental results demonstrate the effectiveness of our proposed method, showcasing superior performance in suppressing artifacts, and surpassing existing diffusion-based methods by 0.74 dB at the maximum in terms of PSNR on DIV2K dataset.
  arXiv  Detail & Related papers  (2024-02-27T01:57:02Z)
• SinSR: Diffusion-Based Image Super-Resolution in a Single Step [119.18813219518042]
  Super-resolution (SR) methods based on diffusion models exhibit promising results. But their practical application is hindered by the substantial number of required inference steps. We propose a simple yet effective method for achieving single-step SR generation, named SinSR.
  arXiv  Detail & Related papers  (2023-11-23T16:21:29Z)
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting [70.83632337581034]
  Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed. We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps. Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
  arXiv  Detail & Related papers  (2023-07-23T15:10:02Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• Semantic Image Synthesis via Diffusion Models [174.24523061460704]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.<n>Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches.<n>We propose a novel framework based on DDPM for semantic image synthesis.
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.