Model-Based Reinforcement Learning for Control under Time-Varying Dynamics

• URL: http://arxiv.org/abs/2604.02260v1
• Date: Thu, 02 Apr 2026 16:52:59 GMT
• Title: Model-Based Reinforcement Learning for Control under Time-Varying Dynamics
• Abstract summary: We study reinforcement learning for control under time-varying dynamics.<n>We propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms.
• Score: 40.96573079787976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcement learning setting in which an agent repeatedly learns and controls a dynamical system whose transition dynamics evolve across episodes. We analyze the problem using Gaussian process dynamics models under frequentist variation-budget assumptions. Our analysis shows that persistent non-stationarity requires explicitly limiting the influence of outdated data to maintain calibrated uncertainty and meaningful dynamic regret guarantees. Motivated by these insights, we propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms and demonstrate improved performance on continuous control benchmarks with non-stationary dynamics.

Related papers

• Attention-based Experience Replay Framework for Continual Learning of Agnostic Time Series Forecasting Models [0.9668407688201359]
  Continual learning aims to develop models capable of adapting incrementally while maintaining a balance between stability and plasticity under computational constraints.<n>We introduce a novel framework for continual time series forecasting, designed to extend existing static forecasting models.<n>Results show that our approach effectively increases or maintains predictive performance over time while reducing retraining costs and data requirements.
  arXiv  Detail & Related papers  (2026-06-12T11:44:05Z)
• Continuous Data Assimilation with Learned Surrogate Dynamics [1.461685903055768]
  In many applications, the state dynamics are unknown or prohibitively expensive to simulate at the required resolution, leading to model error.<n>Motivated by this challenge and the increasing adoption of machine learning surrogates in data assimilation, this paper develops a unified finite-dimensional analysis of nudging algorithms that employ learned surrogate models of the dynamics.
  arXiv  Detail & Related papers  (2026-05-30T02:15:51Z)
• Dyna-Style Reinforcement Learning Modeling and Control of Non-linear Dynamics [0.0]
  We propose a Dyna-Style Reinforcement Learning control framework that integrates Sparse Identification of Dynamics (SINDy) with Twin Delayed Deep Deterministic Policy Gradient (TD3)<n>SINDy is used to identify a data-driven model of the system, capturing its key dynamics without requiring an explicit physical model.<n>To demonstrate the effectiveness of this framework, we apply it to a bi-rotor system as a case study, evaluating its performance in stabilization and trajectory tracking.
  arXiv  Detail & Related papers  (2025-12-24T09:56:28Z)
• Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making [48.998030470623384]
  offline decision-making requires reliable behaviors from fixed datasets without further interaction.<n>We propose a compositional model-based diffusion framework consisting of: (i) a planner that generates diverse, task-aligned trajectories; (ii) a dynamics model that enforces consistency with the underlying system dynamics; and (iii) a ranker module that selects behaviors aligned with the task objectives.
  arXiv  Detail & Related papers  (2025-12-09T06:26:02Z)
• Data-Driven Control with Inherent Lyapunov Stability [3.695480271934742]
  We propose Control with Inherent Lyapunov Stability (CoILS) as a method for jointly learning parametric representations of a nonlinear dynamics model and a stabilizing controller from data. In addition to the stabilizability of the learned dynamics guaranteed by our novel construction, we show that the learned controller stabilizes the true dynamics under certain assumptions on the fidelity of the learned dynamics.
  arXiv  Detail & Related papers  (2023-03-06T14:21:42Z)
• Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control [46.81433026280051]
  We present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions.
  arXiv  Detail & Related papers  (2022-10-23T00:45:05Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.