OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design

• URL: http://arxiv.org/abs/2602.23761v1
• Date: Fri, 27 Feb 2026 07:38:31 GMT
• Title: OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design
• Authors: Yuyu Geng, Lei Sun, Yao Gao, Xinxin Hu, Zhonghua Yi, Xiaolong Qian, Weijian Hu, Jian Bai, Kaiwei Wang,
• Abstract summary: Optical design is the process of configuring optical elements to precisely manipulate light for high-fidelity imaging.<n>This work represents the first attempt to bridge the gap between large language models and formal optical design algorithms.<n>Our model integrates with specialized optical optimization routines for end-to-end fine-tuning and precision refinement.
• Score: 9.777936085725033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical design is the process of configuring optical elements to precisely manipulate light for high-fidelity imaging. It is inherently a highly non-convex optimization problem that relies heavily on human heuristic expertise and domain-specific knowledge. While Large Language Models (LLMs) possess extensive optical knowledge, their capabilities in leveraging the knowledge in designing lens system remain significantly constrained. This work represents the first attempt to employ LLMs in the field of optical design. We bridge the expertise gap by enabling users without formal optical training to successfully develop functional lens systems. Concretely, we curate a comprehensive dataset, named OptiDesignQA, which encompasses both classical lens systems sourced from standard optical textbooks and novel configurations generated by automated design algorithms for training and evaluation. Furthermore, we inject domain-specific optical expertise into the LLM through a hybrid objective of full-system synthesis and lens completion. To align the model with optical principles, we employ Group Relative Policy Optimization Done Right (DrGRPO) guided by Optical Lexicographic Reward for physics-driven policy alignment. This reward system incorporates structural format rewards, physical feasibility rewards, light-manipulation accuracy, and LLM-based heuristics. Finally, our model integrates with specialized optical optimization routines for end-to-end fine-tuning and precision refinement. We benchmark our proposed method against both traditional optimization-based automated design algorithms and LLM counterparts, and experimental results show the superiority of our method.

Related papers

• Learning to Remove Lens Flare in Event Camera [56.9171469873838]
  We present E-DeflareDeflare, the first framework for removing lens flare from event camera data.<n>We first establish the theoretical foundation by deriving a physics-grounded forward model of the non-linear suppression mechanism.<n> Empowered by this benchmark, we design E-DeflareNet, which achieves state-of-the-art restoration performance.
  arXiv  Detail & Related papers  (2025-12-09T18:59:57Z)
• OmniLens++: Blind Lens Aberration Correction via Large LensLib Pre-Training and Latent PSF Representation [72.72583225885636]
  This work proposes the OmniLens++ framework, which resolves two challenges that hinder the generalization ability of existing pipelines.<n>Experiments on diverse aberrations of real-world lenses and synthetic LensLib show that OmniLens++ exhibits state-of-the-art generalization capacity in blind aberration correction.
  arXiv  Detail & Related papers  (2025-11-21T10:41:54Z)
• Neuro-inspired automated lens design [49.79054309060706]
  OptiNeuro is a novel automated lens design framework that generates diverse initial structures and then progressively eliminates lenses.<n>By fully automating the design of complex aspheric lenses, OptiNeuro demonstrates quasi-human-level performance, identifying multiple viable candidates with minimal human intervention.
  arXiv  Detail & Related papers  (2025-10-11T03:14:56Z)
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization [18.41925837760181]
  We introduce a model-free reinforcement learning approach utilizing Proximal Policy Optimization for the in situ training of diffractive optical processors.<n>We experimentally validated our method across a range of in situ learning tasks, including targeted energy focusing through a random diffuser, holographic image generation, aberration correction, and optical image classification.
  arXiv  Detail & Related papers  (2025-07-08T01:39:36Z)
• GOBench: Benchmarking Geometric Optics Generation and Understanding of MLLMs [66.55945133516776]
  We introduce GOBench, the first benchmark to evaluate MLLMs' ability across two tasks: Generating Optically Authentic Imagery and Understanding Underlying Optical Phenomena.<n>We use MLLMs to construct GOBench-Gen-1k dataset. We then organize subjective experiments to assess the generated imagery based on Optical Authenticity, Aesthetic Quality, and Instruction Fidelity.<n>For the understanding task, we apply crafted evaluation instructions to test optical understanding ability of eleven prominent MLLMs. The experimental results demonstrate that current models face significant challenges in both optical generation and understanding.
  arXiv  Detail & Related papers  (2025-06-01T12:46:14Z)
• Tolerance-Aware Deep Optics [15.445359232123133]
  Deep optics has emerged as a promising approach by co-designing optical elements with deep learning algorithms.<n>We present the first end-to-end tolerance-aware optimization framework that incorporates multiple tolerance types into the deep optics design pipeline.<n>Our method combines physics-informed modelling with data-driven training to enhance optical design by accounting for and compensating for structural deviations in manufacturing and assembly.
  arXiv  Detail & Related papers  (2025-02-07T07:42:25Z)
• Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems [15.976326291076377]
  We present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems.<n>QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability.
  arXiv  Detail & Related papers  (2024-04-30T01:59:25Z)
• Neural Lithography: Close the Design-to-Manufacturing Gap in Computational Optics with a 'Real2Sim' Learned Photolithography Simulator [2.033983045970252]
  We introduce neural lithography to address the 'design-to-manufacturing' gap in computational optics. We propose a fully differentiable design framework that integrates a pre-trained photolithography simulator into the model-based optical design loop.
  arXiv  Detail & Related papers  (2023-09-29T15:50:26Z)
• Curriculum Learning for ab initio Deep Learned Refractive Optics [17.52983714236245]
  DeepLens is able to learn optical designs of compound ab initio from randomlytuning surfaces without human intervention. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens.
  arXiv  Detail & Related papers  (2023-02-02T13:22:18Z)
• Photonic Differential Privacy with Direct Feedback Alignment [66.61196212740359]
  We show how to leverage the intrinsic noise of optical random projections to build a differentially private DFA mechanism. We conduct experiments demonstrating the ability of our learning procedure to achieve solid end-task performance.
  arXiv  Detail & Related papers  (2021-06-07T14:18:01Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.