Deep Camera Obscura: An Image Restoration Pipeline for Lensless Pinhole Photography

• URL: http://arxiv.org/abs/2108.05563v1
• Date: Thu, 12 Aug 2021 07:03:00 GMT
• Title: Deep Camera Obscura: An Image Restoration Pipeline for Lensless Pinhole Photography
• Authors: Joshua D. Rego, Huaijin Chen, Shuai Li, Jinwei Gu, Suren Jayasuriya
• Abstract summary: pinhole camera is perhaps the earliest and simplest form of an imaging system using only a pinhole-sized aperture in place of a lens. In this paper, we explore an image restoration pipeline using deep learning and domain-knowledge of the pinhole system to enhance the pinhole image quality through a joint denoise and deblur approach. Our approach allows for more practical exposure times for hand-held photography and provides higher image quality, making it more suitable for daily photography compared to other lensless cameras while keeping size and cost low.
• Score: 18.19703711805033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The lensless pinhole camera is perhaps the earliest and simplest form of an imaging system using only a pinhole-sized aperture in place of a lens. They can capture an infinite depth-of-field and offer greater freedom from optical distortion over their lens-based counterparts. However, the inherent limitations of a pinhole system result in lower sharpness from blur caused by optical diffraction and higher noise levels due to low light throughput of the small aperture, requiring very long exposure times to capture well-exposed images. In this paper, we explore an image restoration pipeline using deep learning and domain-knowledge of the pinhole system to enhance the pinhole image quality through a joint denoise and deblur approach. Our approach allows for more practical exposure times for hand-held photography and provides higher image quality, making it more suitable for daily photography compared to other lensless cameras while keeping size and cost low. This opens up the potential of pinhole cameras to be used in smaller devices, such as smartphones.

Related papers

• Let There Be Light: Robust Lensless Imaging Under External Illumination With Deep Learning [7.368155086339779]
  Lensless cameras relax the design constraints of traditional cameras by shifting image formation from analog optics to digital post-processing. While new camera designs and applications can be enabled, lensless imaging is very sensitive to unwanted interference (other sources, noise, etc.)
  arXiv  Detail & Related papers  (2024-09-25T09:24:53Z)
• Enhancing Low-Light Images Using Infrared-Encoded Images [81.8710581927427]
  Previous arts mainly focus on the low-light images captured in the visible spectrum using pixel-wise loss. We propose a novel approach to increase the visibility of images captured under low-light environments by removing the in-camera infrared (IR) cut-off filter.
  arXiv  Detail & Related papers  (2023-07-09T08:29:19Z)
• Bokeh Rendering Based on Adaptive Depth Calibration Network [13.537088629080122]
  Bokeh rendering is a popular technique used in photography to create an aesthetically pleasing effect. Mobile phones are not able to capture natural shallow depth-of-field photos. We propose a novel method for bokeh rendering using the Vision Transformer, a recent and powerful deep learning architecture.
  arXiv  Detail & Related papers  (2023-02-21T16:33:51Z)
• Self-Supervised Image Restoration with Blurry and Noisy Pairs [66.33313180767428]
  Images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion. Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either full- or self-supervision. We propose jointly leveraging the short-exposure noisy image and the long-exposure blurry image for better image restoration.
  arXiv  Detail & Related papers  (2022-11-14T12:57:41Z)
• Direct Handheld Burst Imaging to Simulated Defocus [1.7403133838762446]
  A shallow depth-of-field image keeps the subject in focus, and the foreground and background contexts blurred. We present a learning-based method to synthesize the defocus blur in shallow depth-of-field images from handheld bursts. Our method does not suffer from artifacts due to inaccurate or ambiguous depth estimation, and it is well-suited to portrait photography.
  arXiv  Detail & Related papers  (2022-07-09T01:59:36Z)
• LenslessPiCam: A Hardware and Software Platform for Lensless Computational Imaging with a Raspberry Pi [14.690546891460235]
  LenslessPiCam provides a framework to enable researchers, hobbyists, and students to implement and explore lensless imaging. We provide detailed guides and exercises so that LenslessPiCam can be used as an educational resource, and point to results from our graduate-level signal processing course.
  arXiv  Detail & Related papers  (2022-06-03T07:39:21Z)
• Learning Spatially Varying Pixel Exposures for Motion Deblurring [49.07867902677453]
  We present a novel approach of leveraging spatially varying pixel exposures for motion deblurring. Our work illustrates the promising role that focal-plane sensor--processors can play in the future of computational imaging.
  arXiv  Detail & Related papers  (2022-04-14T23:41:49Z)
• Photon-Starved Scene Inference using Single Photon Cameras [14.121328731553868]
  We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels. We develop training techniques that push images with different illumination levels closer to each other in feature representation space. Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks.
  arXiv  Detail & Related papers  (2021-07-23T02:27:03Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• A learning-based view extrapolation method for axial super-resolution [52.748944517480155]
  Axial light field resolution refers to the ability to distinguish features at different depths by refocusing. We propose a learning-based method to extrapolate novel views from axial volumes of sheared epipolar plane images.
  arXiv  Detail & Related papers  (2021-03-11T07:22:13Z)
• Rendering Natural Camera Bokeh Effect with Deep Learning [95.86933125733673]
  Bokeh is an important artistic effect used to highlight the main object of interest on the photo. Mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics. We propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras.
  arXiv  Detail & Related papers  (2020-06-10T07:28:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.