Global Search Optics: Automatically Exploring Optimal Solutions to Compact Computational Imaging Systems

• URL: http://arxiv.org/abs/2404.19201v1
• Date: Tue, 30 Apr 2024 01:59:25 GMT
• Title: Global Search Optics: Automatically Exploring Optimal Solutions to Compact Computational Imaging Systems
• Authors: Yao Gao, Qi Jiang, Shaohua Gao, Lei Sun, Kailun Yang, Kaiwei Wang,
• Abstract summary: The popularity of mobile vision creates a demand for advanced compact computational imaging systems. Joint design pipelines come to the forefront, where the two significant components are simultaneously optimized via data-driven learning. In this work, we present Global Search Optimization (GSO) to design compact computational imaging systems.
• Score: 15.976326291076377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The popularity of mobile vision creates a demand for advanced compact computational imaging systems, which call for the development of both a lightweight optical system and an effective image reconstruction model. Recently, joint design pipelines come to the research forefront, where the two significant components are simultaneously optimized via data-driven learning to realize the optimal system design. However, the effectiveness of these designs largely depends on the initial setup of the optical system, complicated by a non-convex solution space that impedes reaching a globally optimal solution. In this work, we present Global Search Optics (GSO) to automatically design compact computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications; and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution. Extensive experimental results on the design of three-piece (3P) sphere computational imaging systems illustrate that the GSO serves as a transformative end-to-end lens design paradigm for superior global optimal structure searching ability, which provides compact computational imaging systems with higher imaging quality compared to traditional methods. The source code will be made publicly available at https://github.com/wumengshenyou/GSO.

Related papers

• Tolerance-Aware Deep Optics [15.445359232123133]
  Deep optics has emerged as a promising approach by co-designing optical elements with deep learning algorithms. We present the first end-to-end tolerance-aware optimization framework that incorporates multiple tolerance types into the deep optics design pipeline. Our method combines physics-informed modelling with data-driven training to enhance optical design by accounting for and compensating for structural deviations in manufacturing and assembly.
  arXiv  Detail & Related papers  (2025-02-07T07:42:25Z)
• Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information [9.527960631238173]
  We introduce a precise optical simulation model, and every operation in pipeline is differentiable. To efficiently address various degradation, we design a joint optimization procedure that leverages field information.
  arXiv  Detail & Related papers  (2024-12-19T07:49:40Z)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models [50.40325411764262]
  Large-scale Mixture of Experts (MoE) models offer enhanced model capacity and computational efficiency through conditional computation. deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency. This survey analyzes optimization techniques for MoE models across the entire system stack.
  arXiv  Detail & Related papers  (2024-12-18T14:11:15Z)
• A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization [19.83939112821776]
  End-to-end optimization has emerged as a powerful data-driven method for computational imaging system design. It is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics. We propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics.
  arXiv  Detail & Related papers  (2024-12-13T00:57:47Z)
• Revealing the preference for correcting separated aberrations in joint optic-image design [19.852225245159598]
  We characterize the optics with separated aberrations to achieve efficient joint design of complex systems such as smartphones and drones. An image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. Experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last.
  arXiv  Detail & Related papers  (2023-09-08T14:12:03Z)
• Image-specific Convolutional Kernel Modulation for Single Image Super-resolution [85.09413241502209]
  In this issue, we propose a novel image-specific convolutional modulation kernel (IKM) We exploit the global contextual information of image or feature to generate an attention weight for adaptively modulating the convolutional kernels. Experiments on single image super-resolution show that the proposed methods achieve superior performances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-16T11:05:10Z)
• A Framework for Discovering Optimal Solutions in Photonic Inverse Design [0.0]
  Photonic inverse design has emerged as an indispensable engineering tool for complex optical systems. Finding solutions approaching global optimum may present a computationally intractable task. We develop a framework that allows expediting the search of solutions close to global optimum on complex optimization spaces.
  arXiv  Detail & Related papers  (2021-06-03T22:11:03Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Optimization-Inspired Learning with Architecture Augmentations and Control Mechanisms for Low-Level Vision [74.9260745577362]
  This paper proposes a unified optimization-inspired learning framework to aggregate Generative, Discriminative, and Corrective (GDC) principles. We construct three propagative modules to effectively solve the optimization models with flexible combinations. Experiments across varied low-level vision tasks validate the efficacy and adaptability of GDC.
  arXiv  Detail & Related papers  (2020-12-10T03:24:53Z)
• End-to-end Full Projector Compensation [81.19324259967742]
  Full projector compensation aims to modify a projector input image to compensate for both geometric and photometric disturbance of the projection surface. In this paper, we propose the first end-to-end differentiable solution, named CompenNeSt++, to solve the two problems jointly.
  arXiv  Detail & Related papers  (2020-07-30T18:23:52Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.