Modeling Dual-Exposure Quad-Bayer Patterns for Joint Denoising and Deblurring

• URL: http://arxiv.org/abs/2412.07256v1
• Date: Tue, 10 Dec 2024 07:35:26 GMT
• Title: Modeling Dual-Exposure Quad-Bayer Patterns for Joint Denoising and Deblurring
• Authors: Yuzhi Zhao, Lai-Man Po, Xin Ye, Yongzhe Xu, Qiong Yan,
• Abstract summary: Single-image solutions face an inherent tradeoff between noise reduction and motion blur. We propose a physical-model-based image restoration approach leveraging a novel dual-exposure Quad-Bayer pattern sensor. We design a hierarchical convolutional neural network called QRNet to recover high-quality RGB images.
• Score: 22.82877719326985
• License:
• Abstract: Image degradation caused by noise and blur remains a persistent challenge in imaging systems, stemming from limitations in both hardware and methodology. Single-image solutions face an inherent tradeoff between noise reduction and motion blur. While short exposures can capture clear motion, they suffer from noise amplification. Long exposures reduce noise but introduce blur. Learning-based single-image enhancers tend to be over-smooth due to the limited information. Multi-image solutions using burst mode avoid this tradeoff by capturing more spatial-temporal information but often struggle with misalignment from camera/scene motion. To address these limitations, we propose a physical-model-based image restoration approach leveraging a novel dual-exposure Quad-Bayer pattern sensor. By capturing pairs of short and long exposures at the same starting point but with varying durations, this method integrates complementary noise-blur information within a single image. We further introduce a Quad-Bayer synthesis method (B2QB) to simulate sensor data from Bayer patterns to facilitate training. Based on this dual-exposure sensor model, we design a hierarchical convolutional neural network called QRNet to recover high-quality RGB images. The network incorporates input enhancement blocks and multi-level feature extraction to improve restoration quality. Experiments demonstrate superior performance over state-of-the-art deblurring and denoising methods on both synthetic and real-world datasets. The code, model, and datasets are publicly available at https://github.com/zhaoyuzhi/QRNet.

Related papers

• bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction [57.199618102578576]
  We propose bit2bit, a new method for reconstructing high-quality image stacks at original resolution from sparse binary quantatemporal image data. Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data. We present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions.
  arXiv  Detail & Related papers  (2024-10-30T17:30:35Z)
• Exposure Bracketing Is All You Need For A High-Quality Image [50.822601495422916]
  Multi-exposure images are complementary in denoising, deblurring, high dynamic range imaging, and super-resolution. We propose to utilize exposure bracketing photography to get a high-quality image by combining these tasks in this work. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
  High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. DNNs still generate ghosting artifacts when LDR images have saturation and large motion. We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
  arXiv  Detail & Related papers  (2023-11-02T01:53:55Z)
• Joint Demosaicing and Denoising with Double Deep Image Priors [5.3686304202729]
  Demosaicing and denoising RAW images are crucial steps in the processing pipeline of modern digital cameras. Recent deep neural-network-based approaches have shown the effectiveness of joint demosaicing and denoising to mitigate such challenges. We propose a novel joint demosaicing and denoising method, dubbed JDD-DoubleDIP, which operates directly on a single RAW image without requiring any training data.
  arXiv  Detail & Related papers  (2023-09-18T01:53:10Z)
• Make Explicit Calibration Implicit: Calibrate Denoiser Instead of the Noise Model [83.9497193551511]
  We introduce Lighting Every Darkness (LED), which is effective regardless of the digital gain or the camera sensor. LED eliminates the need for explicit noise model calibration, instead utilizing an implicit fine-tuning process that allows quick deployment and requires minimal data. LED also allows researchers to focus more on deep learning advancements while still utilizing sensor engineering benefits.
  arXiv  Detail & Related papers  (2023-08-07T10:09:11Z)
• Inheriting Bayer's Legacy-Joint Remosaicing and Denoising for Quad Bayer Image Sensor [10.36002828484577]
  Pixel binning based Quad sensors have emerged as a promising solution to overcome the hardware limitations of compact cameras in low-light imaging. We propose a dual-head joint remosaicing and denoising network (DJRD), which enables the conversion of noisy Quad Bayer and standard noise-free Bayer pattern. Our proposed model outperforms competing models by approximately 3dB, without additional complexity in hardware or software.
  arXiv  Detail & Related papers  (2023-03-23T16:16:50Z)
• Self-Supervised Image Restoration with Blurry and Noisy Pairs [66.33313180767428]
  Images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion. Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either full- or self-supervision. We propose jointly leveraging the short-exposure noisy image and the long-exposure blurry image for better image restoration.
  arXiv  Detail & Related papers  (2022-11-14T12:57:41Z)
• D2HNet: Joint Denoising and Deblurring with Hierarchical Network for Robust Night Image Restoration [17.6005530147501]
  Night imaging with modern smartphone cameras is troublesome due to low photon count and unavoidable noise in the imaging system. We develop a D2HNet framework to recover a high-quality image by deblurring and enhancing a long-exposure image under the guidance of a short-exposure image. The results on the validation set and real photos demonstrate our methods achieve better visual quality and state-of-the-art quantitative scores.
  arXiv  Detail & Related papers  (2022-07-07T13:42:05Z)
• HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization [39.82945546614887]
  We propose a novel learning-based approach using a spatially dynamic encoder-decoder network, HDRUNet, to learn an end-to-end mapping for single image HDR reconstruction. Our method achieves the state-of-the-art performance in quantitative comparisons and visual quality.
  arXiv  Detail & Related papers  (2021-05-27T12:12:34Z)
• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution [134.9023380383406]
  Single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images. This paper proposes to design a more complex but practical degradation model that consists of randomly shuffled blur, downsampling and noise degradations.
  arXiv  Detail & Related papers  (2021-03-25T17:40:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.