Curriculum Learning for ab initio Deep Learned Refractive Optics

• URL: http://arxiv.org/abs/2302.01089v4
• Date: Wed, 7 Aug 2024 05:22:57 GMT
• Title: Curriculum Learning for ab initio Deep Learned Refractive Optics
• Authors: Xinge Yang, Qiang Fu, Wolfgang Heidrich,
• Abstract summary: DeepLens is able to learn optical designs of compound ab initio from randomlytuning surfaces without human intervention. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens.
• Score: 17.52983714236245
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep optical optimization has recently emerged as a new paradigm for designing computational imaging systems using only the output image as the objective. However, it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element (DOE) or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a DeepLens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens in a cellphone-style form factor, with highly aspheric surfaces and a short back focal length.

Related papers

• Global Search Optics: Automatically Exploring Optimal Solutions to Compact Computational Imaging Systems [15.976326291076377]
  The popularity of mobile vision creates a demand for advanced compact computational imaging systems. Joint design pipelines come to the forefront, where the two significant components are simultaneously optimized via data-driven learning. In this work, we present Global Search Optimization (GSO) to design compact computational imaging systems.
  arXiv  Detail & Related papers  (2024-04-30T01:59:25Z)
• Neural Lithography: Close the Design-to-Manufacturing Gap in Computational Optics with a 'Real2Sim' Learned Photolithography Simulator [2.033983045970252]
  We introduce neural lithography to address the 'design-to-manufacturing' gap in computational optics. We propose a fully differentiable design framework that integrates a pre-trained photolithography simulator into the model-based optical design loop.
  arXiv  Detail & Related papers  (2023-09-29T15:50:26Z)
• Revealing the preference for correcting separated aberrations in joint optic-image design [19.852225245159598]
  We characterize the optics with separated aberrations to achieve efficient joint design of complex systems such as smartphones and drones. An image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. Experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last.
  arXiv  Detail & Related papers  (2023-09-08T14:12:03Z)
• Compound eye inspired flat lensless imaging with spatially-coded Voronoi-Fresnel phase [32.914536774672925]
  We report a lensless camera with spatially-coded Voronoi-Fresnel phase, partly inspired by biological apposition compound eye, to achieve superior image quality. We demonstrate and verify the imaging performance with a prototype Voronoi-Fresnel lensless camera on a 1.6-megapixel image sensor in various illumination conditions.
  arXiv  Detail & Related papers  (2021-09-28T13:13:58Z)
• Deep Reparametrization of Multi-Frame Super-Resolution and Denoising [167.42453826365434]
  We propose a deep reparametrization of the maximum a posteriori formulation commonly employed in multi-frame image restoration tasks. Our approach is derived by introducing a learned error metric and a latent representation of the target image. We validate our approach through comprehensive experiments on burst denoising and burst super-resolution datasets.
  arXiv  Detail & Related papers  (2021-08-18T17:57:02Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• A learning-based view extrapolation method for axial super-resolution [52.748944517480155]
  Axial light field resolution refers to the ability to distinguish features at different depths by refocusing. We propose a learning-based method to extrapolate novel views from axial volumes of sheared epipolar plane images.
  arXiv  Detail & Related papers  (2021-03-11T07:22:13Z)
• Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces [103.08512487830669]
  This paper presents an uncalibrated deep neural network framework for the photometric stereo problem. Existing neural network-based methods either require exact light directions or ground-truth surface normals of the object or both. We propose an uncalibrated neural inverse rendering approach to this problem.
  arXiv  Detail & Related papers  (2020-12-12T10:33:08Z)
• Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics [72.9038524082252]
  We propose a compact single-shot monocular hyperspectral-depth (HS-D) imaging method. Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum. To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager.
  arXiv  Detail & Related papers  (2020-09-01T14:19:35Z)
• Learning Light Field Angular Super-Resolution via a Geometry-Aware Network [101.59693839475783]
  We propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$times$.
  arXiv  Detail & Related papers  (2020-02-26T02:36:57Z)
• Learning Wavefront Coding for Extended Depth of Field Imaging [4.199844472131922]
  Extended depth of field (EDoF) imaging is a challenging ill-posed problem. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element. We demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.
  arXiv  Detail & Related papers  (2019-12-31T17:00:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.