Dynamic Attention-Guided Diffusion for Image Super-Resolution
- URL: http://arxiv.org/abs/2308.07977v3
- Date: Thu, 7 Mar 2024 15:24:03 GMT
- Title: Dynamic Attention-Guided Diffusion for Image Super-Resolution
- Authors: Brian B. Moser, Stanislav Frolov, Federico Raue, Sebastian Palacio and
Andreas Dengel
- Abstract summary: "You Only Diffuse Areas" (YODA) is a dynamic attention-guided diffusion method for image Super-Resolution (SR)
We empirically validate YODA by extending leading diffusion-based methods SR3 and SRDiff.
Our experiments demonstrate new state-of-the-art performance in face and general SR across PSNR, SSIM, and LPIPS metrics.
- Score: 10.082751617396474
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Diffusion models in image Super-Resolution (SR) treat all image regions with
uniform intensity, which risks compromising the overall image quality. To
address this, we introduce "You Only Diffuse Areas" (YODA), a dynamic
attention-guided diffusion method for image SR. YODA selectively focuses on
spatial regions using attention maps derived from the low-resolution image and
the current time step in the diffusion process. This time-dependent targeting
enables a more efficient conversion to high-resolution outputs by focusing on
areas that benefit the most from the iterative refinement process, i.e.,
detail-rich objects. We empirically validate YODA by extending leading
diffusion-based methods SR3 and SRDiff. Our experiments demonstrate new
state-of-the-art performance in face and general SR across PSNR, SSIM, and
LPIPS metrics. A notable finding is YODA's stabilization effect by reducing
color shifts, especially when training with small batch sizes.
Related papers
- PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment [91.38256332633544]
PASTA is a Progressively Aggregated Spatio-Temporal Alignment framework for HDR deghosting.
Our approach achieves effectiveness and efficiency by harnessing hierarchical representation during feature distanglement.
Experimental results showcase PASTA's superiority over current SOTA methods in both visual quality and performance metrics.
arXiv Detail & Related papers (2024-03-15T15:05:29Z) - Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
This study proposes a novel and efficient diffusion model for image restoration.
Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration.
Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
arXiv Detail & Related papers (2024-03-12T05:06:07Z) - Improving the Stability of Diffusion Models for Content Consistent
Super-Resolution [17.2713480052151]
generative priors of pre-trained latent diffusion models have demonstrated great potential to enhance the perceptual quality of image super-resolution (SR) results.
We propose to employ the diffusion models to refine image structures, while employing the generative adversarial training to enhance image fine details.
Specifically, we propose a non-uniform timestep learning strategy to train a compact diffusion network, which has high efficiency and stability to reproduce the image main structures.
arXiv Detail & Related papers (2023-12-30T10:22:59Z) - Global Structure-Aware Diffusion Process for Low-Light Image Enhancement [64.69154776202694]
This paper studies a diffusion-based framework to address the low-light image enhancement problem.
We advocate for the regularization of its inherent ODE-trajectory.
Experimental evaluations reveal that the proposed framework attains distinguished performance in low-light enhancement.
arXiv Detail & Related papers (2023-10-26T17:01:52Z) - ResShift: Efficient Diffusion Model for Image Super-resolution by
Residual Shifting [70.83632337581034]
Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed.
We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps.
Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
arXiv Detail & Related papers (2023-07-23T15:10:02Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration.
We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
arXiv Detail & Related papers (2023-06-01T03:08:28Z) - Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image
Super-Resolution [4.255342416942236]
This paper presents a novel Diffusion-Wavelet (DiWa) approach for Single-Image Super-Resolution (SISR)
It leverages the strengths of Denoising Diffusion Probabilistic Models (DDPMs) and Discrete Wavelet Transformation (DWT)
By enabling DDPMs to operate in the DWT domain, our DDPM models effectively hallucinate high-frequency information for super-resolved images on the wavelet spectrum.
arXiv Detail & Related papers (2023-04-04T17:52:49Z) - Guided Depth Super-Resolution by Deep Anisotropic Diffusion [18.445649181582823]
We propose a novel approach which combines guided anisotropic diffusion with a deep convolutional network.
We achieve unprecedented results in three commonly used benchmarks for guided depth super-resolution.
arXiv Detail & Related papers (2022-11-21T15:48:13Z) - Boosting Image Super-Resolution Via Fusion of Complementary Information
Captured by Multi-Modal Sensors [21.264746234523678]
Image Super-Resolution (SR) provides a promising technique to enhance the image quality of low-resolution optical sensors.
In this paper, we attempt to leverage complementary information from a low-cost channel (visible/depth) to boost image quality of an expensive channel (thermal) using fewer parameters.
arXiv Detail & Related papers (2020-12-07T02:15:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.