The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection

• URL: http://arxiv.org/abs/2212.04441v2
• Date: Mon, 27 Mar 2023 18:16:47 GMT
• Title: The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection
• Authors: Geoffroi C\^ot\'e, Fahim Mannan, Simon Thibault, Jean-Fran\c{c}ois Lalonde, Felix Heide
• Abstract summary: Most camera lens systems are designed in isolation, separately from downstream computer methods. We propose an optimization strategy to address the challenges to lens design. Specifically, we introduce quantized glass variables to facilitate the optimization of glass materials in an end-to-end context.
• Score: 42.00621716076439
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Most camera lens systems are designed in isolation, separately from downstream computer vision methods. Recently, joint optimization approaches that design lenses alongside other components of the image acquisition and processing pipeline -- notably, downstream neural networks -- have achieved improved imaging quality or better performance on vision tasks. However, these existing methods optimize only a subset of lens parameters and cannot optimize glass materials given their categorical nature. In this work, we develop a differentiable spherical lens simulation model that accurately captures geometrical aberrations. We propose an optimization strategy to address the challenges of lens design -- notorious for non-convex loss function landscapes and many manufacturing constraints -- that are exacerbated in joint optimization tasks. Specifically, we introduce quantized continuous glass variables to facilitate the optimization and selection of glass materials in an end-to-end design context, and couple this with carefully designed constraints to support manufacturability. In automotive object detection, we report improved detection performance over existing designs even when simplifying designs to two- or three-element lenses, despite significantly degrading the image quality.

Related papers

• VICAN: Very Efficient Calibration Algorithm for Large Camera Networks [49.17165360280794]
  We introduce a novel methodology that extends Pose Graph Optimization techniques. We consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step. Our framework retains compatibility with traditional PGO solvers, but its efficacy benefits from a custom-tailored optimization scheme.
  arXiv  Detail & Related papers  (2024-03-25T17:47:03Z)
• Revealing the preference for correcting separated aberrations in joint optic-image design [19.852225245159598]
  We characterize the optics with separated aberrations to achieve efficient joint design of complex systems such as smartphones and drones. An image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. Experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last.
  arXiv  Detail & Related papers  (2023-09-08T14:12:03Z)
• Computational Optics for Mobile Terminals in Mass Production [17.413494778377565]
  We construct the perturbed lens system model to illustrate the relationship between the system parameters and the deviated frequency response measured from photographs. An optimization framework is proposed based on this model to build proxy cameras from the machining samples' SFRs. Engaging with the proxy cameras, we synthetic data pairs, which encode the optical aberrations and the random manufacturing biases, for training the aberration-based algorithms.
  arXiv  Detail & Related papers  (2023-05-10T04:17:33Z)
• Neural Lens Modeling [50.57409162437732]
  NeuroLens is a neural lens model for distortion and vignetting that can be used for point projection and ray casting. It can be used to perform pre-capture calibration using classical calibration targets, and can later be used to perform calibration or refinement during 3D reconstruction. The model generalizes across many lens types and is trivial to integrate into existing 3D reconstruction and rendering systems.
  arXiv  Detail & Related papers  (2023-04-10T20:09:17Z)
• Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models [55.04969603431266]
  This paper proposes a method for generating images of customized objects specified by users. The method is based on a general framework that bypasses the lengthy optimization required by previous approaches. We demonstrate through experiments that our proposed method is able to synthesize images with compelling output quality, appearance diversity, and object fidelity.
  arXiv  Detail & Related papers  (2023-04-05T17:59:32Z)
• Learning rich optical embeddings for privacy-preserving lensless image classification [17.169529483306103]
  We exploit the unique multiplexing property of casting the optics as an encoder that produces learned embeddings directly at the camera sensor. We do so in the context of image classification, where we jointly optimize the encoder's parameters and those of an image classifier in an end-to-end fashion. Our experiments show that jointly learning the lensless optical encoder and the digital processing allows for lower resolution embeddings at the sensor, and hence better privacy as it is much harder to recover meaningful images from these measurements.
  arXiv  Detail & Related papers  (2022-06-03T07:38:09Z)
• Coded Illumination for Improved Lensless Imaging [22.992552346745523]
  We propose to use coded illumination to improve the quality of images reconstructed with lensless cameras. In our imaging model, the scene/object is illuminated by multiple coded illumination patterns as the lensless camera records sensor measurements. We propose a fast and low-complexity recovery algorithm that exploits the separability and block-diagonal structure in our system.
  arXiv  Detail & Related papers  (2021-11-25T01:22:40Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.