An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms

• URL: http://arxiv.org/abs/2203.12114v2
• Date: Sun, 1 Oct 2023 15:54:32 GMT
• Title: An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms
• Authors: Abulikemu Abuduweili and Changliu Liu
• Abstract summary: Deep reinforcement learning has the potential to address various scientific problems. In this paper, we implement an optics simulation environment for learning based controllers. Results demonstrate the superiority of the environment over traditional simulation environments.
• Score: 7.6418236982756955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep reinforcement learning has the potential to address various scientific problems. In this paper, we implement an optics simulation environment for reinforcement learning based controllers. The environment captures the essence of nonconvexity, nonlinearity, and time-dependent noise inherent in optical systems, offering a more realistic setting. Subsequently, we provide the benchmark results of several reinforcement learning algorithms on the proposed simulation environment. The experimental findings demonstrate the superiority of off-policy reinforcement learning approaches over traditional control algorithms in navigating the intricacies of complex optical control environments. The code of the paper is available at https://github.com/Walleclipse/Reinforcement-Learning-Pulse-Stacking.

Related papers

• Model-based deep reinforcement learning for accelerated learning from flow simulations [0.0]
  We demonstrate the benefits of model-based reinforcement learning for flow control applications. Specifically, we optimize the policy by alternating between trajectories sampled from flow simulations and trajectories sampled from an ensemble of environment models. The model-based learning reduces the overall training time by up to $85%$ for the fluidic pinball test case.
  arXiv  Detail & Related papers  (2024-02-26T13:01:45Z)
• Meta-Learning Strategies through Value Maximization in Neural Networks [7.285835869818669]
  We present a learning effort framework capable of efficiently optimizing control signals on a fully normative objective. We apply this framework to investigate the effect of approximations in common meta-learning algorithms. Across settings, we find that control effort is most beneficial when applied to easier aspects of a task early in learning.
  arXiv  Detail & Related papers  (2023-10-30T18:29:26Z)
• Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization [63.75188254377202]
  Deep reinforcement learning algorithms can perform poorly in real-world tasks due to discrepancy between source and target environments. We propose a novel model-free actor-critic algorithm to learn robust policies without modeling the disturbance in advance. Experiments in several robot control tasks demonstrate that SCPO learns robust policies against the disturbance in transition dynamics.
  arXiv  Detail & Related papers  (2021-12-20T13:13:05Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• Composable Learning with Sparse Kernel Representations [110.19179439773578]
  We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment.
  arXiv  Detail & Related papers  (2021-03-26T13:58:23Z)
• Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarking [52.44199258132215]
  We present an open-source library of differentiable physics and robotics environments. The library features several popular environments, including classical control settings from OpenAI Gym. We give several use-cases of new scientific results obtained using the library.
  arXiv  Detail & Related papers  (2021-02-19T15:06:47Z)
• Reinforcement Learning for Control of Valves [0.0]
  This paper is a study of reinforcement learning (RL) as an optimal-control strategy for control of nonlinear valves. It is evaluated against the PID (proportional-integral-derivative) strategy, using a unified framework.
  arXiv  Detail & Related papers  (2020-12-29T09:01:47Z)
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity [67.24635298387624]
  We show that an off-policy RL algorithm can reason about and tackle lifelong non-stationarity. Our method leverages latent variable models to learn a representation of the environment from current and past experiences. We also introduce several simulation environments that exhibit lifelong non-stationarity, and empirically find that our approach substantially outperforms approaches that do not reason about environment shift.
  arXiv  Detail & Related papers  (2020-06-18T17:34:50Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.