Self-Supervised Image Restoration with Blurry and Noisy Pairs

• URL: http://arxiv.org/abs/2211.07317v1
• Date: Mon, 14 Nov 2022 12:57:41 GMT
• Title: Self-Supervised Image Restoration with Blurry and Noisy Pairs
• Authors: Zhilu Zhang, Rongjian Xu, Ming Liu, Zifei Yan, Wangmeng Zuo
• Abstract summary: Images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion. Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either full- or self-supervision. We propose jointly leveraging the short-exposure noisy image and the long-exposure blurry image for better image restoration.
• Score: 66.33313180767428
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When taking photos under an environment with insufficient light, the exposure time and the sensor gain usually require to be carefully chosen to obtain images with satisfying visual quality. For example, the images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion. Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either full- or self-supervision. However, the real-world training pairs are difficult to collect, and the self-supervised methods merely rely on blurry or noisy images are limited in performance. In this work, we tackle this problem by jointly leveraging the short-exposure noisy image and the long-exposure blurry image for better image restoration. Such setting is practically feasible due to that short-exposure and long-exposure images can be either acquired by two individual cameras or synthesized by a long burst of images. Moreover, the short-exposure images are hardly blurry, and the long-exposure ones have negligible noise. Their complementarity makes it feasible to learn restoration model in a self-supervised manner. Specifically, the noisy images can be used as the supervision information for deblurring, while the sharp areas in the blurry images can be utilized as the auxiliary supervision information for self-supervised denoising. By learning in a collaborative manner, the deblurring and denoising tasks in our method can benefit each other. Experiments on synthetic and real-world images show the effectiveness and practicality of the proposed method. Codes are available at https://github.com/cszhilu1998/SelfIR.

Related papers

• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Dual-Camera Joint Deblurring-Denoising [24.129908866882346]
  We propose a novel dual-camera method for obtaining a high-quality image. Our method uses a synchronized burst of short exposure images captured by one camera and a long exposure image simultaneously captured by another. Our method is able to achieve state-of-the-art results on synthetic dual-camera images from the GoPro dataset with five times fewer training parameters compared to the next best method.
  arXiv  Detail & Related papers  (2023-09-16T00:58:40Z)
• Contrastive Learning for Low-light Raw Denoising [2.929093799984392]
  We introduce a new denoising contrastive regularization (DCR) to exploit the information of noisy images and clean images. In the feature space, DCR makes the denoised image closer to the clean image and far away from the noisy image. In addition, we build a new feature embedding network called Wnet, which is more effective to extract high-frequency information.
  arXiv  Detail & Related papers  (2023-05-05T08:13:53Z)
• Learn to See Faster: Pushing the Limits of High-Speed Camera with Deep Underexposed Image Denoising [12.507566152678857]
  The ability to record high-fidelity videos at high acquisition rates is central to the study of fast moving phenomena. The difficulty of imaging fast moving scenes lies in a trade-off between motion blur and underexposure noise. We propose to address this trade-off by treating the problem of high-speed imaging as an underexposed image denoising problem.
  arXiv  Detail & Related papers  (2022-11-29T09:10:50Z)
• Robust Scene Inference under Noise-Blur Dual Corruptions [20.0721386176278]
  Scene inference under low-light is a challenging problem due to severe noise in the captured images. With the rise of cameras capable of capturing multiple exposures of the same scene simultaneously, it is possible to overcome this trade-off. We propose a method to leverage these multi exposure captures for robust inference under low-light and motion.
  arXiv  Detail & Related papers  (2022-07-24T02:52:00Z)
• Exposure Correction Model to Enhance Image Quality [70.7848787073901]
  We propose an end-to-end exposure correction model to handle both under- and overexposure errors. Our model contains an image encoder, consecutive residual blocks, and image decoder to synthesize the corrected image. We show that after applying exposure correction with the proposed model, the portrait matting quality increases significantly.
  arXiv  Detail & Related papers  (2022-04-22T11:38:52Z)
• Improving Blind Spot Denoising for Microscopy [73.94017852757413]
  We present a novel way to improve the quality of self-supervised denoising. We assume the clean image to be the result of a convolution with a point spread function (PSF) and explicitly include this operation at the end of our neural network.
  arXiv  Detail & Related papers  (2020-08-19T13:06:24Z)
• Deep Bilateral Retinex for Low-Light Image Enhancement [96.15991198417552]
  Low-light images suffer from poor visibility caused by low contrast, color distortion and measurement noise. This paper proposes a deep learning method for low-light image enhancement with a particular focus on handling the measurement noise. The proposed method is very competitive to the state-of-the-art methods, and has significant advantage over others when processing images captured in extremely low lighting conditions.
  arXiv  Detail & Related papers  (2020-07-04T06:26:44Z)
• Learning Multi-Scale Photo Exposure Correction [51.57836446833474]
  Capturing photographs with wrong exposures remains a major source of errors in camera-based imaging. We propose a coarse-to-fine deep neural network (DNN) model, trainable in an end-to-end manner, that addresses each sub-problem separately. Our method achieves results on par with existing state-of-the-art methods on underexposed images.
  arXiv  Detail & Related papers  (2020-03-25T19:33:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.