Beyond Image Prior: Embedding Noise Prior into Conditional Denoising Transformer

• URL: http://arxiv.org/abs/2407.09094v1
• Date: Fri, 12 Jul 2024 08:43:11 GMT
• Title: Beyond Image Prior: Embedding Noise Prior into Conditional Denoising Transformer
• Authors: Yuanfei Huang, Hua Huang,
• Abstract summary: Existing learning-based denoising methods typically train models to generalize the image prior from large-scale datasets. We propose a new perspective on the denoising challenge by highlighting the distinct separation between noise and image priors. We introduce a Locally Noise Prior Estimation algorithm, which accurately estimates the noise prior directly from a single raw noisy image.
• Score: 17.430622649002427
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Existing learning-based denoising methods typically train models to generalize the image prior from large-scale datasets, suffering from the variability in noise distributions encountered in real-world scenarios. In this work, we propose a new perspective on the denoising challenge by highlighting the distinct separation between noise and image priors. This insight forms the basis for our development of conditional optimization framework, designed to overcome the constraints of traditional denoising framework. To this end, we introduce a Locally Noise Prior Estimation (LoNPE) algorithm, which accurately estimates the noise prior directly from a single raw noisy image. This estimation acts as an explicit prior representation of the camera sensor's imaging environment, distinct from the image prior of scenes. Additionally, we design an auxiliary learnable LoNPE network tailored for practical application to sRGB noisy images. Leveraging the estimated noise prior, we present a novel Conditional Denoising Transformer (Condformer), by incorporating the noise prior into a conditional self-attention mechanism. This integration allows the Condformer to segment the optimization process into multiple explicit subspaces, significantly enhancing the model's generalization and flexibility. Extensive experimental evaluations on both synthetic and real-world datasets, demonstrate that the proposed method achieves superior performance over current state-of-the-art methods. The source code is available at https://github.com/YuanfeiHuang/Condformer.

Related papers

• Score Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoising [14.486289176696438]
  We propose a score priors-guided deep variational inference, namely ScoreDVI, for practical real-world denoising. We exploit a Non-$i.i.d$ Gaussian mixture model and variational noise posterior to model the real-world noise. Our method outperforms other single image-based real-world denoising methods and achieves comparable performance to dataset-based unsupervised methods.
  arXiv  Detail & Related papers  (2023-08-09T03:26:58Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Representing Noisy Image Without Denoising [91.73819173191076]
  Fractional-order Moments in Radon space (FMR) is designed to derive robust representation directly from noisy images. Unlike earlier integer-order methods, our work is a more generic design taking such classical methods as special cases.
  arXiv  Detail & Related papers  (2023-01-18T10:13:29Z)
• NLIP: Noise-robust Language-Image Pre-training [95.13287735264937]
  We propose a principled Noise-robust Language-Image Pre-training framework (NLIP) to stabilize pre-training via two schemes: noise-harmonization and noise-completion. Our NLIP can alleviate the common noise effects during image-text pre-training in a more efficient way.
  arXiv  Detail & Related papers  (2022-12-14T08:19:30Z)
• CFNet: Conditional Filter Learning with Dynamic Noise Estimation for Real Image Denoising [37.29552796977652]
  This paper considers real noise approximated by heteroscedastic Gaussian/Poisson Gaussian distributions with in-camera signal processing pipelines. We propose a novel conditional filter in which the optimal kernels for different feature positions can be adaptively inferred by local features from the image and the noise map. Also, we bring the thought that alternatively performs noise estimation and non-blind denoising into CNN structure, which continuously updates noise prior to guide the iterative feature denoising.
  arXiv  Detail & Related papers  (2022-11-26T14:28:54Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
  We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise. Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-24T18:11:31Z)
• FINO: Flow-based Joint Image and Noise Model [23.9749061109964]
  Flow-based joint Image and NOise model (FINO) We propose a novel Flow-based joint Image and NOise model (FINO) that distinctly decouples the image and noise in the latent space and losslessly reconstructs them via a series of invertible transformations.
  arXiv  Detail & Related papers  (2021-11-11T02:51:54Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.