Gym-TORAX: Open-source software for integrating RL with plasma control simulators

• URL: http://arxiv.org/abs/2510.11283v1
• Date: Mon, 13 Oct 2025 11:16:25 GMT
• Title: Gym-TORAX: Open-source software for integrating RL with plasma control simulators
• Authors: Antoine Mouchamps, Arthur Malherbe, Adrien Bolland, Damien Ernst,
• Abstract summary: Gym-TORAX is a Python package enabling the implementation of Reinforcement Learning environments.<n>Gym-TORAX creates a Gymnasium environment that wraps TORAX for simulating the plasma dynamics.<n>In its current version, one environment is readily available, based on a ramp-up scenario of the International Thermonuclear Experimental Reactor (ITER)
• Score: 1.226598527858578
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents Gym-TORAX, a Python package enabling the implementation of Reinforcement Learning (RL) environments for simulating plasma dynamics and control in tokamaks. Users define succinctly a set of control actions and observations, and a control objective from which Gym-TORAX creates a Gymnasium environment that wraps TORAX for simulating the plasma dynamics. The objective is formulated through rewards depending on the simulated state of the plasma and control action to optimize specific characteristics of the plasma, such as performance and stability. The resulting environment instance is then compatible with a wide range of RL algorithms and libraries and will facilitate RL research in plasma control. In its current version, one environment is readily available, based on a ramp-up scenario of the International Thermonuclear Experimental Reactor (ITER).

Related papers

• TokaMind: A Multi-Modal Transformer Foundation Model for Tokamak Plasma Dynamics [56.073642366268764]
  TokaMind is an open-source foundation model framework for fusion plasma modeling.<n>It is trained on heterogeneous tokamak diagnostics from the publicly available MAST dataset.<n>We evaluate TokaMind on the recently introduced MAST benchmark TokaMark.
  arXiv  Detail & Related papers  (2026-02-16T12:26:07Z)
• Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control [61.155940786140455]
  Reinforcement learning (RL) has shown promising results in active flow control (AFC)<n>Current AFC benchmarks rely on external computational fluid dynamics (CFD) solvers, are not fully differentiable, and provide limited 3D and multi-agent support.<n>We introduce FluidGym, the first standalone, fully differentiable benchmark suite for RL in AFC.
  arXiv  Detail & Related papers  (2026-01-21T14:13:44Z)
• Simulating Environments with Reasoning Models for Agent Training [55.98861707136674]
  Building bespoke environments for training is heavy, brittle, and limits progress.<n>We propose two frameworks: Simia-SFT and Simia-RL.<n>Simia-SFT and Simia-RL enable scalable agent training without environment engineering.
  arXiv  Detail & Related papers  (2025-11-03T18:29:57Z)
• Plasma Shape Control via Zero-shot Generative Reinforcement Learning [17.3934551430283]
  We develop a novel framework for developing a versatile, zero-shot control policy from a large-scale offline dataset of PID-controlled discharges.<n>The resulting foundation policy can be deployed for diverse trajectory tracking tasks in a zero-shot manner without any task-specific fine-tuning.
  arXiv  Detail & Related papers  (2025-10-20T13:34:51Z)
• SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning [40.71019623757305]
  This paper presents Sinergym, an open-source Python-based virtual testbed for large-scale building simulation, data collection, continuous control, and experiment monitoring.
  arXiv  Detail & Related papers  (2024-12-11T11:09:13Z)
• SwarmRL: Building the Future of Smart Active Systems [1.8087133416885264]
  This work introduces SwarmRL, a Python package designed to study intelligent active particles. SwarmRL provides an easy-to-use interface for developing models to control microscopic colloids.
  arXiv  Detail & Related papers  (2024-04-25T07:57:11Z)
• Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations [5.394255369988441]
  This paper presents a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap.
  arXiv  Detail & Related papers  (2023-11-21T00:59:27Z)
• Plasma Surrogate Modelling using Fourier Neural Operators [57.52074029826172]
  Predicting plasma evolution within a Tokamak reactor is crucial to realizing the goal of sustainable fusion. We demonstrate accurate predictions of evolution plasma using deep learning-based surrogate modelling tools, viz., Neural Operators (FNO) We show that FNO has a speedup of six orders of magnitude over traditional solvers in predicting the plasma dynamics simulated from magnetohydrodynamic models. FNOs can also predict plasma evolution on real-world experimental data observed by the cameras positioned within the MAST Tokamak.
  arXiv  Detail & Related papers  (2023-11-10T10:05:00Z)
• Closing the loop: Autonomous experiments enabled by machine-learning-based online data analysis in synchrotron beamline environments [80.49514665620008]
  Machine learning can be used to enhance research involving large or rapidly generated datasets. In this study, we describe the incorporation of ML into a closed-loop workflow for X-ray reflectometry (XRR) We present solutions that provide an elementary data analysis in real time during the experiment without introducing the additional software dependencies in the beamline control software environment.
  arXiv  Detail & Related papers  (2023-06-20T21:21:19Z)
• Gym-preCICE: Reinforcement Learning Environments for Active Flow Control [0.0]
  Gym-preCICE is a Python adapter fully compliant with Gymnasium (formerly known as OpenAI Gym) API. Gym-preCICE takes advantage of preCICE, an open-source coupling library for partitioned multi-physics simulations. The framework results in a seamless integration of realistic physics-based simulation toolboxes with RL algorithms.
  arXiv  Detail & Related papers  (2023-05-03T10:54:56Z)
• SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering [49.78647219715034]
  We propose a sensing-aware model-based reinforcement learning system called SAM-RL. With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process. We apply our framework to real world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation.
  arXiv  Detail & Related papers  (2022-10-27T05:30:43Z)
• Transmit Power Control for Indoor Small Cells: A Method Based on Federated Reinforcement Learning [2.392377380146]
  This paper proposes a distributed cell power-control scheme based on Federated Reinforcement Learning (FRL) Models in different indoor environments are aggregated to the global model during the training process, and then the central server broadcasts the updated model back to each client. The results of the generalisation test show that using the FRL model as the base model improves the convergence speed of the model in the new environment.
  arXiv  Detail & Related papers  (2022-08-31T14:46:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.