Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines

• URL: http://arxiv.org/abs/2411.12183v1
• Date: Tue, 19 Nov 2024 02:50:11 GMT
• Title: Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines
• Authors: Siyu Wang, Shengran Dai, Jianhui Jiang, Shuang Wu, Yufei Peng, Junbin Zhang,
• Abstract summary: Synchrotron radiation sources play a crucial role in fields such as materials science, biology, and chemistry. The alignment of beamlines is a complex and time-consuming process, primarily carried out manually by engineers. This paper addresses the alignment of beamlines by modeling it as a Markov Decision Process (MDP) and training an intelligent agent using RL.
• Score: 8.893851834398179
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synchrotron radiation sources play a crucial role in fields such as materials science, biology, and chemistry. The beamline, a key subsystem of the synchrotron, modulates and directs the radiation to the sample for analysis. However, the alignment of beamlines is a complex and time-consuming process, primarily carried out manually by experienced engineers. Even minor misalignments in optical components can significantly affect the beam's properties, leading to suboptimal experimental outcomes. Current automated methods, such as bayesian optimization (BO) and reinforcement learning (RL), although these methods enhance performance, limitations remain. The relationship between the current and target beam properties, crucial for determining the adjustment, is not fully considered. Additionally, the physical characteristics of optical elements are overlooked, such as the need to adjust specific devices to control the output beam's spot size or position. This paper addresses the alignment of beamlines by modeling it as a Markov Decision Process (MDP) and training an intelligent agent using RL. The agent calculates adjustment values based on the current and target beam states, executes actions, and iterates until optimal parameters are achieved. A policy network with action attention is designed to improve decision-making by considering both state differences and the impact of optical components. Experiments on two simulated beamlines demonstrate that our algorithm outperforms existing methods, with ablation studies highlighting the effectiveness of the action attention-based policy network.

Related papers

• Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration [49.03824084306578]
  We propose to incorporate a physical inductive bias into the neural network calibration architecture to enhance the robustness and the trustworthiness of the AI target application. We pave the way for a trustworthy uncertainty representation and for a holistic verification strategy of the perception chain.
  arXiv  Detail & Related papers  (2024-12-18T10:36:46Z)
• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Dynamic Exclusion of Low-Fidelity Data in Bayesian Optimization for Autonomous Beamline Alignment [0.0]
  This study is an investigation of methods to identify untrustworthy readings of beam quality and discourage the optimization model from seeking out points likely to yield low-fidelity beams. The approaches explored include dynamic pruning using loss analysis of size and position models and a lengthscale-based genetic algorithm to determine which points to include in the model for optimal fit.
  arXiv  Detail & Related papers  (2024-08-13T00:20:39Z)
• InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance [56.184255657175335]
  We develop textbfInferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-20T10:41:03Z)
• Laboratory Experiments of Model-based Reinforcement Learning for Adaptive Optics Control [0.565395466029518]
  We implement and adapt an RL method called Policy Optimization for AO (PO4AO) to the GHOST test bench at ESO headquarters. We study the predictive and self-calibrating aspects of the method. New implementation on GHOST running PyTorch introduces only around 700 microseconds in addition to hardware, pipeline, and Python interface latency.
  arXiv  Detail & Related papers  (2023-12-30T14:11:43Z)
• Machine Learning For Beamline Steering [0.0]
  The LINAC To Undulator section of the beamline is difficult to aim. Each use of the accelerator requires re-calibration of the magnets in this section. We investigate the use of deep neural networks to assist in this task.
  arXiv  Detail & Related papers  (2023-11-13T18:00:06Z)
• TempoRL: laser pulse temporal shape optimization with Deep Reinforcement Learning [0.577478614918139]
  High Power Laser's (HPL) optimal performance is essential for the success of a wide variety of experimental tasks related to light-matter interactions. Traditionally, HPL parameters are optimised in an automated fashion relying on black-box numerical methods. Model-free Deep Reinforcement Learning (DRL) offers a promising alternative framework for optimising HPL performance.
  arXiv  Detail & Related papers  (2023-04-20T22:15:27Z)
• Spectral Decomposition Representation for Reinforcement Learning [100.0424588013549]
  We propose an alternative spectral method, Spectral Decomposition Representation (SPEDER), that extracts a state-action abstraction from the dynamics without inducing spurious dependence on the data collection policy. A theoretical analysis establishes the sample efficiency of the proposed algorithm in both the online and offline settings. An experimental investigation demonstrates superior performance over current state-of-the-art algorithms across several benchmarks.
  arXiv  Detail & Related papers  (2022-08-19T19:01:30Z)
• Aligning an optical interferometer with beam divergence control and continuous action space [64.71260357476602]
  We implement vision-based alignment of an optical Mach-Zehnder interferometer with a confocal telescope in one arm. In an experimental evaluation, the agent significantly outperforms an existing solution and a human expert.
  arXiv  Detail & Related papers  (2021-07-09T14:23:01Z)
• Interferobot: aligning an optical interferometer by a reinforcement learning agent [118.43526477102573]
  We train an RL agent to align a Mach-Zehnder interferometer, based on images of fringes acquired by a monocular camera. The agent is trained in a simulated environment, without any hand-coded features or a priori information about the physics. Thanks to a set of domain randomizations simulating uncertainties in physical measurements, the agent successfully aligns this interferometer without any fine tuning.
  arXiv  Detail & Related papers  (2020-06-03T13:10:54Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)
• A Reinforcement Learning based approach for Multi-target Detection in Massive MIMO radar [12.982044791524494]
  This paper considers the problem of multi-target detection for massive multiple input multiple output (MMIMO) cognitive radar (CR) We propose a reinforcement learning (RL) based algorithm for cognitive multi-target detection in the presence of unknown disturbance statistics. Numerical simulations are performed to assess the performance of the proposed RL-based algorithm in both stationary and dynamic environments.
  arXiv  Detail & Related papers  (2020-05-10T16:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.