Dynamic Attention-Guided Diffusion for Image Super-Resolution

• URL: http://arxiv.org/abs/2308.07977v4
• Date: Fri, 22 Nov 2024 05:05:17 GMT
• Title: Dynamic Attention-Guided Diffusion for Image Super-Resolution
• Authors: Brian B. Moser, Stanislav Frolov, Federico Raue, Sebastian Palacio, Andreas Dengel,
• Abstract summary: Diffusion models in image Super-Resolution (SR) treat all image regions uniformly, which risks compromising the overall image quality. We propose You Only Diffuse Areas'' (YODA), a dynamic attention-guided diffusion process for image SR. We empirically validate YODA by extending leading diffusion-based methods SR3, DiffBIR, and SRDiff. Our experiments demonstrate new state-of-the-art performances in face and general SR tasks across PSNR, SSIM, and LPIPS metrics.
• Score: 9.398135472047132
• License:
• Abstract: Diffusion models in image Super-Resolution (SR) treat all image regions uniformly, which risks compromising the overall image quality by potentially introducing artifacts during denoising of less-complex regions. To address this, we propose ``You Only Diffuse Areas'' (YODA), a dynamic attention-guided diffusion process for image SR. YODA selectively focuses on spatial regions defined by attention maps derived from the low-resolution images and the current denoising time step. This time-dependent targeting enables a more efficient conversion to high-resolution outputs by focusing on areas that benefit the most from the iterative refinement process, i.e., detail-rich objects. We empirically validate YODA by extending leading diffusion-based methods SR3, DiffBIR, and SRDiff. Our experiments demonstrate new state-of-the-art performances in face and general SR tasks across PSNR, SSIM, and LPIPS metrics. As a side effect, we find that YODA reduces color shift issues and stabilizes training with small batches.

Related papers

• Latent Diffusion, Implicit Amplification: Efficient Continuous-Scale Super-Resolution for Remote Sensing Images [7.920423405957888]
  E$2$DiffSR achieves superior objective metrics and visual quality compared to the state-of-the-art SR methods. It reduces the inference time of diffusion-based SR methods to a level comparable to that of non-diffusion methods.
  arXiv  Detail & Related papers  (2024-10-30T09:14:13Z)
• AdaDiffSR: Adaptive Region-aware Dynamic Acceleration Diffusion Model for Real-World Image Super-Resolution [14.2500092850787]
  Diffusion models (DMs) have shown promising results on single-image super-resolution and other image-to-image translation tasks. We propose AdaDiffSR, a DMs-based super-resolution pipeline with dynamic timesteps sampling strategy (DTSS) Our experiments show that our AdaDiffSR achieves comparable performance over current state-of-the-art DMs-based SR methods.
  arXiv  Detail & Related papers  (2024-10-23T10:29:18Z)
• One Step Diffusion-based Super-Resolution with Time-Aware Distillation [60.262651082672235]
  Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. Recent techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. We propose a time-aware diffusion distillation method, named TAD-SR, to accomplish effective and efficient image super-resolution.
  arXiv  Detail & Related papers  (2024-08-14T11:47:22Z)
• PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment [91.38256332633544]
  PASTA is a Progressively Aggregated Spatio-Temporal Alignment framework for HDR deghosting. Our approach achieves effectiveness and efficiency by harnessing hierarchical representation during feature distanglement. Experimental results showcase PASTA's superiority over current SOTA methods in both visual quality and performance metrics.
  arXiv  Detail & Related papers  (2024-03-15T15:05:29Z)
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting [70.83632337581034]
  Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed. We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps. Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
  arXiv  Detail & Related papers  (2023-07-23T15:10:02Z)
• Single Image LDR to HDR Conversion using Conditional Diffusion [18.466814193413487]
  Digital imaging aims to replicate realistic scenes, but Low Dynamic Range (LDR) cameras cannot represent the wide dynamic range of real scenes. This paper presents a deep learning-based approach for recovering intricate details from shadows and highlights. We incorporate a deep-based autoencoder in our proposed framework to enhance the quality of the latent representation of LDR image used for conditioning.
  arXiv  Detail & Related papers  (2023-07-06T07:19:47Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• High-Frequency aware Perceptual Image Enhancement [0.08460698440162888]
  We introduce a novel deep neural network suitable for multi-scale analysis and propose efficient model-agnostic methods. Our model can be applied to multi-scale image enhancement problems including denoising, deblurring and single image super-resolution.
  arXiv  Detail & Related papers  (2021-05-25T07:33:14Z)
• Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors [21.264746234523678]
  Image Super-Resolution (SR) provides a promising technique to enhance the image quality of low-resolution optical sensors. In this paper, we attempt to leverage complementary information from a low-cost channel (visible/depth) to boost image quality of an expensive channel (thermal) using fewer parameters.
  arXiv  Detail & Related papers  (2020-12-07T02:15:28Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.