Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves

• URL: http://arxiv.org/abs/2404.10991v1
• Date: Wed, 17 Apr 2024 02:04:10 GMT
• Title: Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves
• Authors: Soumyendu Sarkar, Vineet Gundecha, Sahand Ghorbanpour, Alexander Shmakov, Ashwin Ramesh Babu, Avisek Naug, Alexandre Pichard, Mathieu Cocho,
• Abstract summary: Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
• Score: 69.9104427437916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The industrial multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices in challenging circumstances need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. The Multi-Agent Reinforcement Learning (MARL) controller trained with the Proximal Policy Optimization (PPO) algorithm can handle these complexities. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics and find that they are key to better performance. We investigated the performance of a fully connected neural network (FCN), LSTM, and Transformer model variants with varying depths and gated residual connections. Our results show that the transformer model of moderate depth with gated residual connections around the multi-head attention, multi-layer perceptron, and the transformer block (STrXL) proposed in this paper is optimal and boosts energy efficiency by an average of 22.1% for these complex spread waves over the existing spring damper (SD) controller. Furthermore, unlike the default SD controller, the transformer controller almost eliminated the mechanical stress from the rotational yaw motion for angled waves. Demo: https://tinyurl.com/yueda3jh

Related papers

• Investigating Recurrent Transformers with Dynamic Halt [64.862738244735]
  We study the inductive biases of two major approaches to augmenting Transformers with a recurrent mechanism. We propose and investigate novel ways to extend and combine the methods.
  arXiv  Detail & Related papers  (2024-02-01T19:47:31Z)
• Exploring Frequency-Inspired Optimization in Transformer for Efficient Single Image Super-Resolution [32.29219284419944]
  Cross-refinement adaptive feature modulation transformer (CRAFT) We introduce a frequency-guided post-training quantization (PTQ) method aimed at enhancing CRAFT's efficiency. Our experimental findings showcase CRAFT's superiority over current state-of-the-art methods, both in full-precision and quantization scenarios.
  arXiv  Detail & Related papers  (2023-08-09T15:38:36Z)
• Private Federated Learning with Dynamic Power Control via Non-Coherent Over-the-Air Computation [12.56727008993937]
  scheme based on dynamic power control is proposed. We show that the whole scheme can mitigate the impact of the time synchronization error, channel fading and noise.
  arXiv  Detail & Related papers  (2023-08-05T13:46:50Z)
• Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters [94.84709449845352]
  Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller.
  arXiv  Detail & Related papers  (2022-09-13T00:20:31Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Extensible circuit-QED architecture via amplitude- and frequency-variable microwaves [52.77024349608834]
  We introduce a circuit-QED architecture combining fixed-frequency qubits and microwave-driven couplers. Drive parameters appear as tunable knobs enabling selective two-qubit coupling and coherent-error suppression.
  arXiv  Detail & Related papers  (2022-04-17T22:49:56Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Learning OFDM Waveforms with PAPR and ACLR Constraints [15.423422040627331]
  We propose a learning-based method to design OFDM-based waveforms that satisfy selected constraints while maximizing an achievable information rate. We show that the end-to-end system is able to satisfy target PAPR and ACLR constraints and allows significant throughput gains.
  arXiv  Detail & Related papers  (2021-10-21T08:58:59Z)
• End-to-End Learning of OFDM Waveforms with PAPR and ACLR Constraints [15.423422040627331]
  We propose to use a neural network (NN) at the transmitter to learn a high-dimensional modulation scheme allowing to control the PAPR and adjacent channel leakage ratio (ACLR) The two NNs operate on top of OFDM, and are jointly optimized in and end-to-end manner using a training algorithm that enforces constraints on the PAPR and ACLR.
  arXiv  Detail & Related papers  (2021-06-30T13:09:30Z)
• High-bandwidth nonlinear control for soft actuators with recursive network models [1.4174475093445231]
  We present a high-bandwidth, lightweight, and nonlinear output tracking technique for soft actuators using Newton-Raphson. This technique allows for reduced model sizes and increased control loop frequencies when compared with conventional RNN models.
  arXiv  Detail & Related papers  (2021-01-04T18:12:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.