Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models

• URL: http://arxiv.org/abs/2207.14626v1
• Date: Fri, 29 Jul 2022 11:52:41 GMT
• Title: Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models
• Authors: Ozan \"Ozdenizci, Robert Legenstein
• Abstract summary: We present a novel patch-based image restoration algorithm based on denoising diffusion probabilistic models. We demonstrate our approach to achieve state-of-the-art performances on both weather-specific and multi-weather image restoration.
• Score: 8.122270502556374
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image restoration under adverse weather conditions has been of significant interest for various computer vision applications. Recent successful methods rely on the current progress in deep neural network architectural designs (e.g., with vision transformers). Motivated by the recent progress achieved with state-of-the-art conditional generative models, we present a novel patch-based image restoration algorithm based on denoising diffusion probabilistic models. Our patch-based diffusion modeling approach enables size-agnostic image restoration by using a guided denoising process with smoothed noise estimates across overlapping patches during inference. We empirically evaluate our model on benchmark datasets for image desnowing, combined deraining and dehazing, and raindrop removal. We demonstrate our approach to achieve state-of-the-art performances on both weather-specific and multi-weather image restoration, and qualitatively show strong generalization to real-world test images.

Related papers

• Gradient-Guided Conditional Diffusion Models for Private Image Reconstruction: Analyzing Adversarial Impacts of Differential Privacy and Denoising [21.30726250408398]
  Current gradient-based reconstruction methods struggle with high-resolution images due to computational complexity and prior knowledge requirements. We propose two novel methods that require minimal modifications to the diffusion model's generation process and eliminate the need for prior knowledge. We conduct a comprehensive theoretical analysis of the impact of differential privacy noise on the quality of reconstructed images, revealing the relationship among noise magnitude, the architecture of attacked models, and the attacker's reconstruction capability.
  arXiv  Detail & Related papers  (2024-11-05T12:39:21Z)
• Edge-based Denoising Image Compression [10.477417679208845]
  Deep learning-based image compression has emerged as a pivotal area of research. We propose a novel compression model that incorporates a denoising step with diffusion models. Our model achieves superior or comparable results in terms of image quality and compression efficiency.
  arXiv  Detail & Related papers  (2024-09-17T08:20:26Z)
• ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
  We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:52:08Z)
• Reconstruct-and-Generate Diffusion Model for Detail-Preserving Image Denoising [16.43285056788183]
  We propose a novel approach called the Reconstruct-and-Generate Diffusion Model (RnG) Our method leverages a reconstructive denoising network to recover the majority of the underlying clean signal. It employs a diffusion algorithm to generate residual high-frequency details, thereby enhancing visual quality.
  arXiv  Detail & Related papers  (2023-09-19T16:01:20Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey [96.99328714941657]
  We present a comprehensive review of recent diffusion model-based methods on image restoration. We classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR. We propose five potential and challenging directions for the future research of diffusion model-based IR.
  arXiv  Detail & Related papers  (2023-08-18T08:40:38Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Anomaly Detection with Conditioned Denoising Diffusion Models [32.37548329437798]
  We introduce Denoising Diffusion Anomaly Detection (DDAD), a novel denoising process for image reconstruction conditioned on a target image. Our anomaly detection framework employs the conditioning mechanism, where the target image is set as the input image to guide the denoising process. DDAD achieves state-of-the-art results of (99.8 %) and (98.9 %) image-level AUROC respectively.
  arXiv  Detail & Related papers  (2023-05-25T11:54:58Z)
• A comparison of different atmospheric turbulence simulation methods for image restoration [64.24948495708337]
  Atmospheric turbulence deteriorates the quality of images captured by long-range imaging systems. Various deep learning-based atmospheric turbulence mitigation methods have been proposed in the literature. We systematically evaluate the effectiveness of various turbulence simulation methods on image restoration.
  arXiv  Detail & Related papers  (2022-04-19T16:21:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.