Self-optimizing adaptive optics control with Reinforcement Learning for high-contrast imaging

• URL: http://arxiv.org/abs/2108.11332v1
• Date: Tue, 24 Aug 2021 10:02:55 GMT
• Title: Self-optimizing adaptive optics control with Reinforcement Learning for high-contrast imaging
• Authors: Rico Landman, Sebastiaan Y. Haffert, Vikram M. Radhakrishnan, Christoph U. Keller
• Abstract summary: We describe how model-free Reinforcement Learning can be used to optimize a Recurrent Neural Network controller for closed-loop predictive control. We show in simulations that our algorithm can also be applied to the control of a high-order deformable mirror.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current and future high-contrast imaging instruments require extreme adaptive optics (XAO) systems to reach contrasts necessary to directly image exoplanets. Telescope vibrations and the temporal error induced by the latency of the control loop limit the performance of these systems. One way to reduce these effects is to use predictive control. We describe how model-free Reinforcement Learning can be used to optimize a Recurrent Neural Network controller for closed-loop predictive control. First, we verify our proposed approach for tip-tilt control in simulations and a lab setup. The results show that this algorithm can effectively learn to mitigate vibrations and reduce the residuals for power-law input turbulence as compared to an optimal gain integrator. We also show that the controller can learn to minimize random vibrations without requiring online updating of the control law. Next, we show in simulations that our algorithm can also be applied to the control of a high-order deformable mirror. We demonstrate that our controller can provide two orders of magnitude improvement in contrast at small separations under stationary turbulence. Furthermore, we show more than an order of magnitude improvement in contrast for different wind velocities and directions without requiring online updating of the control law.

Related papers

• Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning [10.886819238167286]
  This study employs a deep reinforcement learning framework to train agents for exposure control. A lightweight image simulator is developed to facilitate the training process. Different levels of reward functions are crafted to enhance the VO systems.
  arXiv  Detail & Related papers  (2024-08-30T04:37:52Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• Structured Deep Neural Network-Based Backstepping Trajectory Tracking Control for Lagrangian Systems [9.61674297336072]
  The proposed controller can ensure closed-loop stability for any compatible neural network parameters. We show that in the presence of model approximation errors and external disturbances, the closed-loop stability and tracking control performance can still be guaranteed.
  arXiv  Detail & Related papers  (2024-03-01T09:09:37Z)
• Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference [73.63024765499719]
  Mobile networks can be compromised by interference from neighboring cells or jammers. We propose Spectrum Breathing, which cascades-gradient pruning and spread spectrum to suppress interference without bandwidth expansion. We show a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth.
  arXiv  Detail & Related papers  (2023-05-10T07:05:43Z)
• Improving the Performance of Robust Control through Event-Triggered Learning [74.57758188038375]
  We propose an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem. We demonstrate improved performance over a robust controller baseline in a numerical example.
  arXiv  Detail & Related papers  (2022-07-28T17:36:37Z)
• Regret-optimal Estimation and Control [52.28457815067461]
  We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form. We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
  arXiv  Detail & Related papers  (2021-06-22T23:14:21Z)
• Two-step reinforcement learning for model-free redesign of nonlinear optimal regulator [1.5624421399300306]
  Reinforcement learning (RL) is one of the promising approaches that enable model-free redesign of optimal controllers for nonlinear dynamical systems. We propose a model-free two-step design approach that improves the transient learning performance of RL in an optimal regulator redesign problem for unknown nonlinear systems.
  arXiv  Detail & Related papers  (2021-03-05T17:12:33Z)
• Meta-Learning-Based Robust Adaptive Flight Control Under Uncertain Wind Conditions [13.00214468719929]
  Realtime model learning is challenging for complex dynamical systems, such as drones flying in variable wind conditions. We propose an online composite adaptation method that treats outputs from a deep neural network as a set of basis functions. We validate our approach by flying a drone in an open air wind tunnel under varying wind conditions and along challenging trajectories.
  arXiv  Detail & Related papers  (2021-03-02T18:43:59Z)
• Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control [2.524528674141466]
  We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. Experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels.
  arXiv  Detail & Related papers  (2020-08-10T20:22:15Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.