Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows

• URL: http://arxiv.org/abs/2506.10153v1
• Date: Wed, 11 Jun 2025 20:14:28 GMT
• Title: Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows
• Authors: Zhecheng Liu, Jeff D. Eldredge,
• Abstract summary: We propose a transformer-based reinforcement learning framework to learn an effective control strategy for regulating aerodynamic lift in gust sequences via pitch control.<n>We show that the learned strategy outperforms the best proportional control, with the performance gap widening as the number of gusts increases.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: A linear flow control strategy designed for weak disturbances may not remain effective in sequences of strong disturbances due to nonlinear interactions, but it is sensible to leverage it for developing a better strategy. In the present study, we propose a transformer-based reinforcement learning (RL) framework to learn an effective control strategy for regulating aerodynamic lift in gust sequences via pitch control. The transformer addresses the challenge of partial observability from limited surface pressure sensors. We demonstrate that the training can be accelerated with two techniques -- pretraining with an expert policy (here, linear control) and task-level transfer learning (here, extending a policy trained on isolated gusts to multiple gusts). We show that the learned strategy outperforms the best proportional control, with the performance gap widening as the number of gusts increases. The control strategy learned in an environment with a small number of successive gusts is shown to effectively generalize to an environment with an arbitrarily long sequence of gusts. We investigate the pivot configuration and show that quarter-chord pitching control can achieve superior lift regulation with substantially less control effort compared to mid-chord pitching control. Through a decomposition of the lift, we attribute this advantage to the dominant added-mass contribution accessible via quarter-chord pitching. The success on multiple configurations shows the generalizability of the proposed transformer-based RL framework, which offers a promising approach to solve more computationally demanding flow control problems when combined with the proposed acceleration techniques.

Related papers

• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows [109.99020160824553]
  We propose an efficient deep reinforcement learning framework for modeling and control of turbulent flows.<n>It is model-based RL for predictive control (PC), where both the policy and the observer models for turbulence control are learned jointly.<n>We find that PINO-PC achieves a drag reduction of 39.0% under a bulk-velocity Reynolds number of 15,000, outperforming previous fluid control methods by more than 32%.
  arXiv  Detail & Related papers  (2025-10-03T00:18:26Z)
• Improving Action Smoothness for a Cascaded Online Learning Flight Control System [7.871518182413388]
  We introduce an online temporal smoothness technique and a low-pass filter to reduce the amplitude and frequency of the control actions.<n> Simulation results demonstrate the improvements achieved by the two proposed techniques.
  arXiv  Detail & Related papers  (2025-07-06T11:19:34Z)
• Transformer-Based Fault-Tolerant Control for Fixed-Wing UAVs Using Knowledge Distillation and In-Context Adaptation [3.1498833540989413]
  This study presents a transformer-based approach for fault-tolerant control in fixed-wing Unmanned Aerial Vehicles (UAVs)<n>Our method directly maps outer-loop reference values into control commands using the in-student learning and attention mechanisms of transformers.<n> Experimental results demonstrate that our transformer-based controller outperforms industry-standard FCS and state-of-the-art reinforcement learning (RL) methods.
  arXiv  Detail & Related papers  (2024-11-05T10:24:45Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• A Reinforcement Learning Approach for Robust Supervisory Control of UAVs Under Disturbances [1.8799681615947088]
  We present an approach to supervisory reinforcement learning control for unmanned aerial vehicles (UAVs) We formulate a supervisory control architecture that interleaves with extant embedded control and demonstrates robustness to environmental disturbances in the form of adverse wind conditions.
  arXiv  Detail & Related papers  (2023-05-21T19:00:06Z)
• Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference [73.63024765499719]
  Mobile networks can be compromised by interference from neighboring cells or jammers. We propose Spectrum Breathing, which cascades-gradient pruning and spread spectrum to suppress interference without bandwidth expansion. We show a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth.
  arXiv  Detail & Related papers  (2023-05-10T07:05:43Z)
• Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blower [30.032847855193864]
  blowing controller must continually adapt to unexpected changes from its actions. We introduce a multi-frequency version of the spatial action maps framework. This allows for efficient learning of vision-based policies that effectively combine high-level planning and low-level closed-loop control.
  arXiv  Detail & Related papers  (2022-04-05T17:55:58Z)
• Optimizing Airborne Wind Energy with Reinforcement Learning [0.0]
  Reinforcement Learning is a technique that learns to associate observations with profitable actions without requiring prior knowledge of the system. We show that in a simulated environment Reinforcement Learning finds an efficient way to control a kite so that it can tow a vehicle for long distances.
  arXiv  Detail & Related papers  (2022-03-27T10:28:16Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)
• Responsive Safety in Reinforcement Learning by PID Lagrangian Methods [74.49173841304474]
  Lagrangian methods exhibit oscillations and overshoot which, when applied to safe reinforcement learning, leads to constraint-violating behavior. We propose a novel Lagrange multiplier update method that utilizes derivatives of the constraint function. We apply our PID Lagrangian methods in deep RL, setting a new state of the art in Safety Gym, a safe RL benchmark.
  arXiv  Detail & Related papers  (2020-07-08T08:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.