Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning

• URL: http://arxiv.org/abs/2108.05251v1
• Date: Wed, 11 Aug 2021 14:45:15 GMT
• Title: Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning
• Authors: Abdullah Abuolaim, Mahmoud Afifi, Michael S. Brown
• Abstract summary: We propose a single-image deblurring network that incorporates the two sub-aperture views into a multi-task framework. Our experiments show this multi-task strategy achieves +1dB PSNR improvement over state-of-the-art defocus deblurring methods. These high-quality DP views can be used for other DP-based applications, such as reflection removal.
• Score: 48.063176079878055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many camera sensors use a dual-pixel (DP) design that operates as a rudimentary light field providing two sub-aperture views of a scene in a single capture. The DP sensor was developed to improve how cameras perform autofocus. Since the DP sensor's introduction, researchers have found additional uses for the DP data, such as depth estimation, reflection removal, and defocus deblurring. We are interested in the latter task of defocus deblurring. In particular, we propose a single-image deblurring network that incorporates the two sub-aperture views into a multi-task framework. Specifically, we show that jointly learning to predict the two DP views from a single blurry input image improves the network's ability to learn to deblur the image. Our experiments show this multi-task strategy achieves +1dB PSNR improvement over state-of-the-art defocus deblurring methods. In addition, our multi-task framework allows accurate DP-view synthesis (e.g., ~ 39dB PSNR) from the single input image. These high-quality DP views can be used for other DP-based applications, such as reflection removal. As part of this effort, we have captured a new dataset of 7,059 high-quality images to support our training for the DP-view synthesis task. Our dataset, code, and trained models will be made publicly available at https://github.com/Abdullah-Abuolaim/multi-task-defocus-deblurring-dual-pixel-nimat

Related papers

• Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging [25.851398356458425]
  Single-shot 3D sensing is useful in many application areas such as microscopy, medical imaging, surgical navigation, and autonomous driving. We propose CADS (Coded Aperture Dual-Pixel Sensing), in which we use a coded aperture in the imaging lens along with a DP sensor. Our resulting CADS imaging system demonstrates improvement of >1.5dB PSNR in all-in-focus (AIF) estimates and 5-6% in depth estimation quality over naive DP sensing.
  arXiv  Detail & Related papers  (2024-02-28T06:45:47Z)
• Learning Dual-Pixel Alignment for Defocus Deblurring [73.80328094662976]
  We propose a Dual-Pixel Alignment Network (DPANet) for defocus deblurring. It is notably superior to state-of-the-art deblurring methods in reducing defocus blur while recovering visually plausible sharp structures and textures.
  arXiv  Detail & Related papers  (2022-04-26T07:02:58Z)
• Facial Depth and Normal Estimation using Single Dual-Pixel Camera [81.02680586859105]
  We introduce a DP-oriented Depth/Normal network that reconstructs the 3D facial geometry. It contains the corresponding ground-truth 3D models including depth map and surface normal in metric scale. It achieves state-of-the-art performances over recent DP-based depth/normal estimation methods.
  arXiv  Detail & Related papers  (2021-11-25T05:59:27Z)
• Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image [54.10957300181677]
  We present a method that takes as input a single dual-pixel image, and simultaneously estimates the image's defocus map. Our approach improves upon prior works for both defocus map estimation and blur removal, despite being entirely unsupervised.
  arXiv  Detail & Related papers  (2021-10-12T00:09:07Z)
• Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data [42.06108142009718]
  Recent work has shown impressive results on data-driven deblurring using the two-image views available on modern dual-pixel (DP) sensors. Despite many cameras having DP sensors, only a limited number provide access to the low-level DP sensor images. We propose a procedure to generate realistic DP data synthetically.
  arXiv  Detail & Related papers  (2020-12-06T13:12:43Z)
• Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration [77.1056200937214]
  We study the formation of the DP pair which links the blur and the depth information. We propose an end-to-end DDDNet (DP-based Depth and De Network) to jointly estimate the depth and restore the image.
  arXiv  Detail & Related papers  (2020-12-01T06:53:57Z)
• Defocus Blur Detection via Depth Distillation [64.78779830554731]
  We introduce depth information into DBD for the first time. In detail, we learn the defocus blur from ground truth and the depth distilled from a well-trained depth estimation network. Our approach outperforms 11 other state-of-the-art methods on two popular datasets.
  arXiv  Detail & Related papers  (2020-07-16T04:58:09Z)
• Defocus Deblurring Using Dual-Pixel Data [41.201653787083735]
  Defocus blur arises in images that are captured with a shallow depth of field due to the use of a wide aperture. We propose an effective defocus deblurring method that exploits data available on dual-pixel (DP) sensors found on most modern cameras.
  arXiv  Detail & Related papers  (2020-05-01T10:38:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.