Reservoir Predictive Path Integral Control for Unknown Nonlinear Dynamics

• URL: http://arxiv.org/abs/2509.03839v1
• Date: Thu, 04 Sep 2025 03:05:17 GMT
• Title: Reservoir Predictive Path Integral Control for Unknown Nonlinear Dynamics
• Authors: Daisuke Inoue, Tadayoshi Matsumori, Gouhei Tanaka, Yuji Ito,
• Abstract summary: This paper integrates echo-state networks (ESNs) and model predictive path integral (MPPI) control to meet challenges of fast online identification and control of unknown dynamics.<n>The proposed reservoir predictive path integral (RPPI) enables fast learning of nonlinear dynamics with ESN and exploits the learned nonlinearities directly in parallelized MPPI control computation.
• Score: 2.152586952462536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural networks capable of approximating complex nonlinearities have found extensive application in data-driven control of nonlinear dynamical systems. However, fast online identification and control of unknown dynamics remain central challenges. This paper integrates echo-state networks (ESNs) -- reservoir computing models implemented with recurrent neural networks -- and model predictive path integral (MPPI) control -- sampling-based variants of model predictive control -- to meet these challenges. The proposed reservoir predictive path integral (RPPI) enables fast learning of nonlinear dynamics with ESN and exploits the learned nonlinearities directly in parallelized MPPI control computation without linearization approximations. The framework is further extended to uncertainty-aware RPPI (URPPI), which leverages ESN uncertainty to balance exploration and exploitation: exploratory inputs dominate during early learning, while exploitative inputs prevail as model confidence grows. Experiments on controlling the Duffing oscillator and four-tank systems demonstrate that URPPI improves control performance, reducing control costs by up to 60% compared to traditional quadratic programming-based model predictive control methods.

Related papers

• Zero-Shot Function Encoder-Based Differentiable Predictive Control [18.369429085503548]
  We introduce a differentiable framework for zero-shot adaptive control over parametric families of nonlinear dynamical systems.<n>Our approach integrates a function encoder-based neural ODE (FE-NODE) for modeling system dynamics with a differentiable predictive control (DPC) for offline self-supervised learning of explicit control policies.
  arXiv  Detail & Related papers  (2025-11-07T23:02:55Z)
• Model-based controller assisted domain randomization in deep reinforcement learning: application to nonlinear powertrain control [0.0]
  This study proposes a new robust control approach using the framework of deep reinforcement learning (DRL)<n>The problem setup is modeled via the latent Markov decision process (LMDP), a set of vanilla MDPs, for a controlled system subject to uncertainties and nonlinearities.<n>Compared to traditional DRL-based controls, the proposed controller design is smarter in that we can achieve a high level of generalization ability.
  arXiv  Detail & Related papers  (2025-04-28T12:09:07Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study [56.283944756315066]
  We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
  arXiv  Detail & Related papers  (2024-01-09T11:54:54Z)
• End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control [45.84205238554709]
  We present a method for reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC.
  arXiv  Detail & Related papers  (2023-08-03T10:21:53Z)
• Unifying Model-Based and Neural Network Feedforward: Physics-Guided Neural Networks with Linear Autoregressive Dynamics [0.0]
  This paper develops a feedforward control framework to compensate unknown nonlinear dynamics. The feedforward controller is parametrized as a parallel combination of a physics-based model and a neural network.
  arXiv  Detail & Related papers  (2022-09-26T08:01:28Z)
• Physics-informed Neural Networks-based Model Predictive Control for Multi-link Manipulators [0.0]
  We discuss nonlinear model predictive control (NMPC) for multi-body dynamics via physics-informed machine learning methods. We present the idea of enhancing PINNs by adding control actions and initial conditions as additional network inputs. We present our results using our PINN-based MPC to solve a tracking problem for a complex mechanical system.
  arXiv  Detail & Related papers  (2021-09-22T15:31:24Z)
• Stochastic Deep Model Reference Adaptive Control [9.594432031144715]
  We present a Deep Neural Network-based Model Reference Adaptive Control. Deep Model Reference Adaptive Control uses a Lyapunov-based method to adapt the output-layer weights of the DNN model in real-time. A data-driven supervised learning algorithm is used to update the inner-layers parameters.
  arXiv  Detail & Related papers  (2021-08-04T14:05:09Z)
• Approximate Robust NMPC using Reinforcement Learning [0.0]
  We present a Reinforcement Learning-based Robust Model Predictive Control (RL-RNMPC) for controlling nonlinear systems in the presence of disturbances and uncertainties.
  arXiv  Detail & Related papers  (2021-04-06T18:34:58Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: from Time-Driven to Event-Driven [80.94390916562179]
  Time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives. It is desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise. We show how the event-driven dHDP algorithm works in comparison to the original time-driven dHDP.
  arXiv  Detail & Related papers  (2020-06-16T05:51:25Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.