Offline Reinforcement Learning for Rotation Profile Control in Tokamaks

• URL: http://arxiv.org/abs/2605.05857v1
• Date: Thu, 07 May 2026 08:26:59 GMT
• Title: Offline Reinforcement Learning for Rotation Profile Control in Tokamaks
• Abstract summary: Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved.<n>One such challenge is controlling the plasma rotation profile, which strongly influences stability, confinement, and transport.<n>Learning-based control methods, such as reinforcement learning (RL), provide a potential solution to this challenging problem.
• Score: 9.641882418424712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved. One such challenge is controlling the plasma rotation profile, which strongly influences stability, confinement, and transport. While the average rotation can be controlled, controlling the full profile is challenging due to high dimensionality, response to multiple actuators and dependence on plasma condition. Learning-based control methods, such as reinforcement learning (RL), provide a potential solution to this challenging problem with ability to model complex interactions leading to effective multi-input multi-output control. However, learning such policies is challenging due to the lack of accurate simulators that can model the rotation profile dynamics. In this work, we investigate the use of offline RL and offline model-based RL algorithms for rotation profile control, training them solely on historical data from the DIII-D tokamak. Our final method uses probabilistic models of plasma dynamics to generate rollouts for RL training. We deploy this policy on the DIII-D Tokamak and observe promising real-world results. We conclude by highlighting key challenges and insights from training and deploying an RL policy on a complex physical device while using only limited past data.

Related papers

• Offline Reinforcement Learning for Plasma Control in Nuclear Fusion: Codebase and Benchmark [25.41253331826036]
  We introduce RL4F, an Offline Reinforcement Learning Benchmark for Plasma Control in Nuclear Fusion.<n> RL4F provides closed-loop evaluation environments and baseline comparisons across four full-profile tracking tasks.<n>We evaluate a broad set of imitation learning and offline RL baselines under a unified protocol.
  arXiv  Detail & Related papers  (2026-05-19T14:09:29Z)
• Plasma Shape Control via Zero-shot Generative Reinforcement Learning [17.3934551430283]
  We develop a novel framework for developing a versatile, zero-shot control policy from a large-scale offline dataset of PID-controlled discharges.<n>The resulting foundation policy can be deployed for diverse trajectory tracking tasks in a zero-shot manner without any task-specific fine-tuning.
  arXiv  Detail & Related papers  (2025-10-20T13:34:51Z)
• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows [109.99020160824553]
  We propose an efficient deep reinforcement learning framework for modeling and control of turbulent flows.<n>It is model-based RL for predictive control (PC), where both the policy and the observer models for turbulence control are learned jointly.<n>We find that PINO-PC achieves a drag reduction of 39.0% under a bulk-velocity Reynolds number of 15,000, outperforming previous fluid control methods by more than 32%.
  arXiv  Detail & Related papers  (2025-10-03T00:18:26Z)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning [93.00629872970364]
  Reinforcement learning (RL) has become the dominant paradigm for improving the performance of language models on complex reasoning tasks.<n>We introduce SPARKLE, a fine-grained analytic framework to dissect the effects of RL across three key dimensions.<n>We study whether difficult problems -- those yielding no RL signals and mixed-quality reasoning traces -- can still be effectively used for training.
  arXiv  Detail & Related papers  (2025-06-05T07:53:59Z)
• CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishing [5.467140383171385]
  This work presents an experimental demonstration of the hybrid RL algorithm CHEQ for robotic polishing with variable impedance.<n>On hardware, CHEQ achieves effective polishing behaviour, requiring only eight hours of training and incurring just five failures.<n>Results highlight the potential of adaptive hybrid RL for real-world, contact-rich tasks trained directly on hardware.
  arXiv  Detail & Related papers  (2025-01-14T10:13:41Z)
• Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling [35.2859997591196]
  offline reinforcement learning holds promise for scaling data-driven decision-making.<n>However, real-world data collected from sensors or humans often contains noise and errors.<n>Our study reveals that prior research falls short under data corruption when the dataset is limited.
  arXiv  Detail & Related papers  (2024-07-05T06:34:32Z)
• SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning [5.036739921794781]
  SINDy-RL is a framework for combining SINDy and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy.<n>We demonstrate the effectiveness of our approaches on benchmark control environments and flow control problems.
  arXiv  Detail & Related papers  (2024-03-14T05:17:39Z)
• Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving [63.155562267383864]
  Deep reinforcement learning (DRL) has shown remarkable success in complex autonomous driving scenarios. DRL models inevitably bring high memory consumption and computation, which hinders their wide deployment in resource-limited autonomous driving devices. We introduce a novel dynamic structured pruning approach that gradually removes a DRL model's unimportant neurons during the training stage.
  arXiv  Detail & Related papers  (2024-02-07T09:00:30Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning [3.1635451288803638]
  We present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball. By adequately designing reward functions and encoding historical observations, the DRL-based control was shown to make reasonable and valid control decisions. One of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process.
  arXiv  Detail & Related papers  (2023-04-23T03:39:50Z)
• Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator [56.11574814802912]
  State-of-the art robotic applications utilize series elastic actuators (SEAs) with closed-loop force control to achieve complex tasks such as walking, lifting, and manipulation. Model-free PID control methods are more prone to instability due to nonlinearities in the SEA. Deep reinforcement learning has proved to be an effective model-free method for continuous control tasks.
  arXiv  Detail & Related papers  (2023-04-11T00:51:47Z)
• On Transforming Reinforcement Learning by Transformer: The Development Trajectory [97.79247023389445]
  Transformer, originally devised for natural language processing, has also attested significant success in computer vision. We group existing developments in two categories: architecture enhancement and trajectory optimization. We examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving.
  arXiv  Detail & Related papers  (2022-12-29T03:15:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.