TempoRL: laser pulse temporal shape optimization with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2304.12187v1
• Date: Thu, 20 Apr 2023 22:15:27 GMT
• Title: TempoRL: laser pulse temporal shape optimization with Deep Reinforcement Learning
• Authors: Francesco Capuano and Davorin Peceli and Gabriele Tiboni and Raffaello Camoriano and Bed\v{r}ich Rus
• Abstract summary: High Power Laser's (HPL) optimal performance is essential for the success of a wide variety of experimental tasks related to light-matter interactions. Traditionally, HPL parameters are optimised in an automated fashion relying on black-box numerical methods. Model-free Deep Reinforcement Learning (DRL) offers a promising alternative framework for optimising HPL performance.
• Score: 0.577478614918139
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: High Power Laser's (HPL) optimal performance is essential for the success of a wide variety of experimental tasks related to light-matter interactions. Traditionally, HPL parameters are optimised in an automated fashion relying on black-box numerical methods. However, these can be demanding in terms of computational resources and usually disregard transient and complex dynamics. Model-free Deep Reinforcement Learning (DRL) offers a promising alternative framework for optimising HPL performance since it allows to tune the control parameters as a function of system states subject to nonlinear temporal dynamics without requiring an explicit dynamics model of those. Furthermore, DRL aims to find an optimal control policy rather than a static parameter configuration, particularly suitable for dynamic processes involving sequential decision-making. This is particularly relevant as laser systems are typically characterised by dynamic rather than static traits. Hence the need for a strategy to choose the control applied based on the current context instead of one single optimal control configuration. This paper investigates the potential of DRL in improving the efficiency and safety of HPL control systems. We apply this technique to optimise the temporal profile of laser pulses in the L1 pump laser hosted at the ELI Beamlines facility. We show how to adapt DRL to the setting of spectral phase control by solely tuning dispersion coefficients of the spectral phase and reaching pulses similar to transform limited with full-width at half-maximum (FWHM) of ca1.6 ps.

Related papers

• Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System [0.7499722271664147]
  This study conducts a comparative analysis of Model Predictive Control (MPC) and Proximal Policy Optimization (PPO), a Deep Reinforcement Learning (DRL) algorithm, applied to a Quanser Aero 2 system. PPO excels in rise-time and adaptability, making it a promising approach for applications requiring rapid response and adaptability.
  arXiv  Detail & Related papers  (2024-08-28T08:35:34Z)
• When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL [37.58940726230092]
  Reinforcement learning (RL) excels in optimizing policies for discrete-time Markov decision processes (MDP) We formalize an RL framework, Time-adaptive Control & Sensing (TaCoS), that tackles this challenge. We demonstrate that state-of-the-art RL algorithms trained on TaCoS drastically reduce the interaction amount over their discrete-time counterpart.
  arXiv  Detail & Related papers  (2024-06-03T09:57:18Z)
• Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves [69.9104427437916]
  Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
  arXiv  Detail & Related papers  (2024-04-17T02:04:10Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e [3.860979702631594]
  We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab) Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale.
  arXiv  Detail & Related papers  (2023-12-28T21:35:20Z)
• AutoRL Hyperparameter Landscapes [69.15927869840918]
  Reinforcement Learning (RL) has shown to be capable of producing impressive results, but its use is limited by the impact of its hyperparameters on performance. We propose an approach to build and analyze these hyperparameter landscapes not just for one point in time but at multiple points in time throughout training. This supports the theory that hyperparameters should be dynamically adjusted during training and shows the potential for more insights on AutoRL problems that can be gained through landscape analyses.
  arXiv  Detail & Related papers  (2023-04-05T12:14:41Z)
• A Framework for History-Aware Hyperparameter Optimisation in Reinforcement Learning [8.659973888018781]
  A Reinforcement Learning (RL) system depends on a set of initial conditions that affect the system's performance. We propose a framework based on integrating complex event processing and temporal models, to alleviate these trade-offs. We tested the proposed approach in a 5G mobile communications case study that uses DQN, a variant of RL, for its decision-making.
  arXiv  Detail & Related papers  (2023-03-09T11:30:40Z)
• Online hyperparameter optimization by real-time recurrent learning [57.01871583756586]
  Our framework takes advantage of the analogy between hyperparameter optimization and parameter learning in neural networks (RNNs) It adapts a well-studied family of online learning algorithms for RNNs to tune hyperparameters and network parameters simultaneously. This procedure yields systematically better generalization performance compared to standard methods, at a fraction of wallclock time.
  arXiv  Detail & Related papers  (2021-02-15T19:36:18Z)
• Sample-Efficient Automated Deep Reinforcement Learning [33.53903358611521]
  We propose a population-based automated RL framework to meta-optimize arbitrary off-policy RL algorithms. By sharing the collected experience across the population, we substantially increase the sample efficiency of the meta-optimization. We demonstrate the capabilities of our sample-efficient AutoRL approach in a case study with the popular TD3 algorithm in the MuJoCo benchmark suite.
  arXiv  Detail & Related papers  (2020-09-03T10:04:06Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.