CycleISP: Real Image Restoration via Improved Data Synthesis

• URL: http://arxiv.org/abs/2003.07761v1
• Date: Tue, 17 Mar 2020 15:20:25 GMT
• Title: CycleISP: Real Image Restoration via Improved Data Synthesis
• Authors: Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao
• Abstract summary: We present a framework that models camera imaging pipeline in forward and reverse directions. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets.
• Score: 166.17296369600774
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The availability of large-scale datasets has helped unleash the true potential of deep convolutional neural networks (CNNs). However, for the single-image denoising problem, capturing a real dataset is an unacceptably expensive and cumbersome procedure. Consequently, image denoising algorithms are mostly developed and evaluated on synthetic data that is usually generated with a widespread assumption of additive white Gaussian noise (AWGN). While the CNNs achieve impressive results on these synthetic datasets, they do not perform well when applied on real camera images, as reported in recent benchmark datasets. This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline. In this paper, we present a framework that models camera imaging pipeline in forward and reverse directions. It allows us to produce any number of realistic image pairs for denoising both in RAW and sRGB spaces. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets. The parameters in our model are ~5 times lesser than the previous best method for RAW denoising. Furthermore, we demonstrate that the proposed framework generalizes beyond image denoising problem e.g., for color matching in stereoscopic cinema. The source code and pre-trained models are available at https://github.com/swz30/CycleISP.

Related papers

• NIR-Assisted Image Denoising: A Selective Fusion Approach and A Real-World Benchmark Dataset [53.79524776100983]
  Leveraging near-infrared (NIR) images to assist visible RGB image denoising shows the potential to address this issue. Existing works still struggle with taking advantage of NIR information effectively for real-world image denoising. We propose an efficient Selective Fusion Module (SFM), which can be plug-and-played into the advanced denoising networks.
  arXiv  Detail & Related papers  (2024-04-12T14:54:26Z)
• RViDeformer: Efficient Raw Video Denoising Transformer with a Larger Benchmark Dataset [16.131438855407175]
  There is no large dataset with realistic motions for supervised raw video denoising. We construct a video denoising dataset (named as ReCRVD) with 120 groups of noisy-clean videos. We propose an efficient raw video denoising transformer network (RViDeformer) that explores both short and long-distance correlations.
  arXiv  Detail & Related papers  (2023-05-01T11:06:58Z)
• FSID: Fully Synthetic Image Denoising via Procedural Scene Generation [12.277286575812441]
  We develop a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results.
  arXiv  Detail & Related papers  (2022-12-07T21:21:55Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Robust Deep Ensemble Method for Real-world Image Denoising [62.099271330458066]
  We propose a simple yet effective Bayesian deep ensemble (BDE) method for real-world image denoising. Our BDE achieves +0.28dB PSNR gain over the state-of-the-art denoising method. Our BDE can be extended to other image restoration tasks, and achieves +0.30dB, +0.18dB and +0.12dB PSNR gains on benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-08T06:19:30Z)
• Modeling sRGB Camera Noise with Normalizing Flows [35.29066692454865]
  We propose a new sRGB-domain noise model based on normalizing flows that is capable of learning the complex noise distribution found in sRGB images under various ISO levels. Our normalizing flows-based approach outperforms other models by a large margin in noise modeling and synthesis tasks.
  arXiv  Detail & Related papers  (2022-06-02T00:56:34Z)
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
  We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise. Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-24T18:11:31Z)
• IDR: Self-Supervised Image Denoising via Iterative Data Refinement [66.5510583957863]
  We present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance. Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising. To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
  arXiv  Detail & Related papers  (2021-11-29T07:22:53Z)
• Rethinking Noise Synthesis and Modeling in Raw Denoising [75.55136662685341]
  We introduce a new perspective to synthesize noise by directly sampling from the sensor's real noise. It inherently generates accurate raw image noise for different camera sensors.
  arXiv  Detail & Related papers  (2021-10-10T10:45:24Z)
• Adaptive Denoising via GainTuning [17.72738152112575]
  Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. We propose "GainTuning", in which CNN models pre-trained on large datasets are adaptively and selectively adjusted for individual test images. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set.
  arXiv  Detail & Related papers  (2021-07-27T13:35:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.