Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization

• URL: http://arxiv.org/abs/2507.05583v1
• Date: Tue, 08 Jul 2025 01:39:36 GMT
• Title: Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Authors: Yuhang Li, Shiqi Chen, Tingyu Gong, Aydogan Ozcan,
• Abstract summary: We introduce a model-free reinforcement learning approach utilizing Proximal Policy Optimization for the in situ training of diffractive optical processors.<n>We experimentally validated our method across a range of in situ learning tasks, including targeted energy focusing through a random diffuser, holographic image generation, aberration correction, and optical image classification.
• Score: 18.41925837760181
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical computing holds promise for high-speed, energy-efficient information processing, with diffractive optical networks emerging as a flexible platform for implementing task-specific transformations. A challenge, however, is the effective optimization and alignment of the diffractive layers, which is hindered by the difficulty of accurately modeling physical systems with their inherent hardware imperfections, noise, and misalignments. While existing in situ optimization methods offer the advantage of direct training on the physical system without explicit system modeling, they are often limited by slow convergence and unstable performance due to inefficient use of limited measurement data. Here, we introduce a model-free reinforcement learning approach utilizing Proximal Policy Optimization (PPO) for the in situ training of diffractive optical processors. PPO efficiently reuses in situ measurement data and constrains policy updates to ensure more stable and faster convergence. We experimentally validated our method across a range of in situ learning tasks, including targeted energy focusing through a random diffuser, holographic image generation, aberration correction, and optical image classification, demonstrating in each task better convergence and performance. Our strategy operates directly on the physical system and naturally accounts for unknown real-world imperfections, eliminating the need for prior system knowledge or modeling. By enabling faster and more accurate training under realistic experimental constraints, this in situ reinforcement learning approach could offer a scalable framework for various optical and physical systems governed by complex, feedback-driven dynamics.

Related papers

• Cost-effective Reduced-Order Modeling via Bayesian Active Learning [12.256032958843065]
  We propose BayPOD-AL, an active learning framework based on an uncertainty-aware Bayesian proper decomposition (POD) approach.<n> Experimental results on predicting the temperature evolution over a rod demonstrate BayPOD-AL's effectiveness in suggesting the informative data.<n>We demonstrate BayPOD-AL's generalizability and efficiency by evaluating its performance on a dataset of higher temporal resolution than the training dataset.
  arXiv  Detail & Related papers  (2025-06-27T21:23:37Z)
• LAPSO: A Unified Optimization View for Learning-Augmented Power System Operations [3.754570687412345]
  This paper proposes a holistic framework of Learning-Augmented Power System Operations (LAPSO)<n>LAPSO is centered on the operation stage and aims to break the boundary between temporally siloed power system tasks.<n>A dedicated Python package-lapso is introduced to automatically augment existing power system optimization models with learnable components.
  arXiv  Detail & Related papers  (2025-05-08T13:00:24Z)
• Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information [9.527960631238173]
  We introduce a precise optical simulation model, and every operation in pipeline is differentiable.<n>To efficiently address various degradation, we design a joint optimization procedure that leverages field information.
  arXiv  Detail & Related papers  (2024-12-19T07:49:40Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• High-performance real-world optical computing trained by in situ gradient-based model-free optimization [2.2407602847819827]
  We propose a gradient-based model-free optimization (G-MFO) method based on a Monte Carlo gradient estimation algorithm. G-MFO treats an optical computing system as a black box and back-propagates the loss directly to the optical computing weights' probability distributions. Our experiments on diffractive optical computing systems show that G-MFO outperforms hybrid training on the MNIST and FMNIST datasets.
  arXiv  Detail & Related papers  (2023-07-22T01:56:58Z)
• Forward-Forward Training of an Optical Neural Network [6.311461340782698]
  We present an experiment utilizing multimode nonlinear wave propagation in an optical fiber demonstrating the feasibility of the FFA approach using an optical system. The results show that incorporating optical transforms in multilayer NN architectures trained with the FFA, can lead to performance improvements.
  arXiv  Detail & Related papers  (2023-05-30T16:15:57Z)
• Toward Fast, Flexible, and Robust Low-Light Image Enhancement [87.27326390675155]
  We develop a new Self-Calibrated Illumination (SCI) learning framework for fast, flexible, and robust brightening images in real-world low-light scenarios. Considering the computational burden of the cascaded pattern, we construct the self-calibrated module which realizes the convergence between results of each stage. We make comprehensive explorations to SCI's inherent properties including operation-insensitive adaptability and model-irrelevant generality.
  arXiv  Detail & Related papers  (2022-04-21T14:40:32Z)
• Low-light Image Enhancement by Retinex Based Algorithm Unrolling and Adjustment [50.13230641857892]
  We propose a new deep learning framework for the low-light image enhancement (LIE) problem. The proposed framework contains a decomposition network inspired by algorithm unrolling, and adjustment networks considering both global brightness and local brightness sensitivity. Experiments on a series of typical LIE datasets demonstrated the effectiveness of the proposed method, both quantitatively and visually, as compared with existing methods.
  arXiv  Detail & Related papers  (2022-02-12T03:59:38Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Progressive residual learning for single image dehazing [57.651704852274825]
  A progressive residual learning strategy has been proposed to combine the physical model-free dehazing process with reformulated scattering model-based dehazing operations. The proposed method performs favorably against the state-of-the-art methods on public dehazing benchmarks with better model interpretability and adaptivity for complex data.
  arXiv  Detail & Related papers  (2021-03-14T16:54:44Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.