Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data

• URL: http://arxiv.org/abs/2012.03255v1
• Date: Sun, 6 Dec 2020 13:12:43 GMT
• Title: Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data
• Authors: Abdullah Abuolaim, Mauricio Delbracio, Damien Kelly, Michael S. Brown, Peyman Milanfar
• Abstract summary: Recent work has shown impressive results on data-driven deblurring using the two-image views available on modern dual-pixel (DP) sensors. Despite many cameras having DP sensors, only a limited number provide access to the low-level DP sensor images. We propose a procedure to generate realistic DP data synthetically.
• Score: 42.06108142009718
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work has shown impressive results on data-driven defocus deblurring using the two-image views available on modern dual-pixel (DP) sensors. One significant challenge in this line of research is access to DP data. Despite many cameras having DP sensors, only a limited number provide access to the low-level DP sensor images. In addition, capturing training data for defocus deblurring involves a time-consuming and tedious setup requiring the camera's aperture to be adjusted. Some cameras with DP sensors (e.g., smartphones) do not have adjustable apertures, further limiting the ability to produce the necessary training data. We address the data capture bottleneck by proposing a procedure to generate realistic DP data synthetically. Our synthesis approach mimics the optical image formation found on DP sensors and can be applied to virtual scenes rendered with standard computer software. Leveraging these realistic synthetic DP images, we introduce a new recurrent convolutional network (RCN) architecture that can improve deblurring results and is suitable for use with single-frame and multi-frame data captured by DP sensors. Finally, we show that our synthetic DP data is useful for training DNN models targeting video deblurring applications where access to DP data remains challenging.

Related papers

• Adaptive Domain Learning for Cross-domain Image Denoising [57.4030317607274]
  We present a novel adaptive domain learning scheme for cross-domain image denoising. We use existing data from different sensors (source domain) plus a small amount of data from the new sensor (target domain) The ADL training scheme automatically removes the data in the source domain that are harmful to fine-tuning a model for the target domain. Also, we introduce a modulation module to adopt sensor-specific information (sensor type and ISO) to understand input data for image denoising.
  arXiv  Detail & Related papers  (2024-11-03T08:08:26Z)
• Disparity Estimation Using a Quad-Pixel Sensor [12.34044154078824]
  A quad-pixel (QP) sensor is increasingly integrated into commercial mobile cameras. We propose a QP disparity estimation network (QPDNet) We present a synthetic pipeline to generate a training dataset from an existing RGB-Depth dataset.
  arXiv  Detail & Related papers  (2024-09-01T08:50:32Z)
• Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging [25.851398356458425]
  Single-shot 3D sensing is useful in many application areas such as microscopy, medical imaging, surgical navigation, and autonomous driving. We propose CADS (Coded Aperture Dual-Pixel Sensing), in which we use a coded aperture in the imaging lens along with a DP sensor. Our resulting CADS imaging system demonstrates improvement of >1.5dB PSNR in all-in-focus (AIF) estimates and 5-6% in depth estimation quality over naive DP sensing.
  arXiv  Detail & Related papers  (2024-02-28T06:45:47Z)
• Practical cross-sensor color constancy using a dual-mapping strategy [0.0]
  The proposed method uses a dual-mapping strategy and only requires a simple white point from a test sensor under a D65 condition. In the second mapping phase, we transform the re-constructed image data into sparse features, which are then optimized with a lightweight multi-layer perceptron (MLP) model. This approach effectively reduces sensor discrepancies and delivers performance on par with leading cross-sensor methods.
  arXiv  Detail & Related papers  (2023-11-20T13:58:59Z)
• Learning Dual-Pixel Alignment for Defocus Deblurring [73.80328094662976]
  We propose a Dual-Pixel Alignment Network (DPANet) for defocus deblurring. It is notably superior to state-of-the-art deblurring methods in reducing defocus blur while recovering visually plausible sharp structures and textures.
  arXiv  Detail & Related papers  (2022-04-26T07:02:58Z)
• Facial Depth and Normal Estimation using Single Dual-Pixel Camera [81.02680586859105]
  We introduce a DP-oriented Depth/Normal network that reconstructs the 3D facial geometry. It contains the corresponding ground-truth 3D models including depth map and surface normal in metric scale. It achieves state-of-the-art performances over recent DP-based depth/normal estimation methods.
  arXiv  Detail & Related papers  (2021-11-25T05:59:27Z)
• Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning [48.063176079878055]
  We propose a single-image deblurring network that incorporates the two sub-aperture views into a multi-task framework. Our experiments show this multi-task strategy achieves +1dB PSNR improvement over state-of-the-art defocus deblurring methods. These high-quality DP views can be used for other DP-based applications, such as reflection removal.
  arXiv  Detail & Related papers  (2021-08-11T14:45:15Z)
• Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration [77.1056200937214]
  We study the formation of the DP pair which links the blur and the depth information. We propose an end-to-end DDDNet (DP-based Depth and De Network) to jointly estimate the depth and restore the image.
  arXiv  Detail & Related papers  (2020-12-01T06:53:57Z)
• Defocus Deblurring Using Dual-Pixel Data [41.201653787083735]
  Defocus blur arises in images that are captured with a shallow depth of field due to the use of a wide aperture. We propose an effective defocus deblurring method that exploits data available on dual-pixel (DP) sensors found on most modern cameras.
  arXiv  Detail & Related papers  (2020-05-01T10:38:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.