Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information

• URL: http://arxiv.org/abs/2412.14603v2
• Date: Mon, 23 Dec 2024 11:40:38 GMT
• Title: Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information
• Authors: Zheng Ren, Jingwen Zhou, Wenguan Zhang, Jiapu Yan, Bingkun Chen, Huajun Feng, Shiqi Chen,
• Abstract summary: We introduce a precise optical simulation model, and every operation in pipeline is differentiable. To efficiently address various degradation, we design a joint optimization procedure that leverages field information.
• Score: 9.527960631238173
• License:
• Abstract: Recently, the joint design of optical systems and downstream algorithms is showing significant potential. However, existing rays-described methods are limited to optimizing geometric degradation, making it difficult to fully represent the optical characteristics of complex, miniaturized lenses constrained by wavefront aberration or diffraction effects. In this work, we introduce a precise optical simulation model, and every operation in pipeline is differentiable. This model employs a novel initial value strategy to enhance the reliability of intersection calculation on high aspherics. Moreover, it utilizes a differential operator to reduce memory consumption during coherent point spread function calculations. To efficiently address various degradation, we design a joint optimization procedure that leverages field information. Guided by a general restoration network, the proposed method not only enhances the image quality, but also successively improves the optical performance across multiple lenses that are already in professional level. This joint optimization pipeline offers innovative insights into the practical design of sophisticated optical systems and post-processing algorithms. The source code will be made publicly available at https://github.com/Zrr-ZJU/Successive-optimization

Related papers

• Tolerance-Aware Deep Optics [15.445359232123133]
  Deep optics has emerged as a promising approach by co-designing optical elements with deep learning algorithms. We present the first end-to-end tolerance-aware optimization framework that incorporates multiple tolerance types into the deep optics design pipeline. Our method combines physics-informed modelling with data-driven training to enhance optical design by accounting for and compensating for structural deviations in manufacturing and assembly.
  arXiv  Detail & Related papers  (2025-02-07T07:42:25Z)
• A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization [19.83939112821776]
  End-to-end optimization has emerged as a powerful data-driven method for computational imaging system design. It is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics. We propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics.
  arXiv  Detail & Related papers  (2024-12-13T00:57:47Z)
• Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration [59.744840744491945]
  We reformulate the trajectory optimization of this kind of method, focusing on enhancing both reconstruction quality and efficiency. We propose cost-aware trajectory distillation to streamline complex paths into several manageable steps with adaptable sizes. Experiments showcase the significant superiority of the proposed method, achieving a maximum PSNR improvement of 2.1 dB over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-07T07:46:08Z)
• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Rethink Predicting the Optical Flow with the Kinetics Perspective [1.7901503554839604]
  Optical flow estimation is one of the fundamental tasks in low-level computer vision. From the apparent aspect, the optical flow can be viewed as the correlation between the pixels in consecutive frames. We propose a method combining the apparent and kinetics information from this motivation.
  arXiv  Detail & Related papers  (2024-05-21T05:47:42Z)
• Revealing the preference for correcting separated aberrations in joint optic-image design [19.852225245159598]
  We characterize the optics with separated aberrations to achieve efficient joint design of complex systems such as smartphones and drones. An image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. Experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last.
  arXiv  Detail & Related papers  (2023-09-08T14:12:03Z)
• Characterization of multi-mode linear optical networks [0.0]
  We formulate efficient procedures for the characterization of optical circuits in the presence of imperfections. We show the viability of this approach in an experimentally relevant scenario, defined by a tunable integrated photonic circuit. Our findings can find application in a wide range of optical setups, based both on bulk and integrated configurations.
  arXiv  Detail & Related papers  (2023-04-13T13:09:14Z)
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo [97.07453889070574]
  We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors. We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
  arXiv  Detail & Related papers  (2021-09-02T17:54:31Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Rapid characterisation of linear-optical networks via PhaseLift [51.03305009278831]
  Integrated photonics offers great phase-stability and can rely on the large scale manufacturability provided by the semiconductor industry. New devices, based on such optical circuits, hold the promise of faster and energy-efficient computations in machine learning applications. We present a novel technique to reconstruct the transfer matrix of linear optical networks.
  arXiv  Detail & Related papers  (2020-10-01T16:04:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.