Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning

• URL: http://arxiv.org/abs/2603.23245v1
• Date: Tue, 24 Mar 2026 14:12:41 GMT
• Title: Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning
• Abstract summary: We investigate neural ordinary and differential equations (neural ODEs and SDEs) to model dynamics in fully and partially observed environments.<n>We show that neural SDEs more effectively capture the inherentity of transition dynamics.<n>This work demonstrates the applicability of action-conditional latent SDEs for RL planning in environments with transitions.
• Score: 1.9129489840935257
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate neural ordinary and stochastic differential equations (neural ODEs and SDEs) to model stochastic dynamics in fully and partially observed environments within a model-based reinforcement learning (RL) framework. Through a sequence of simulations, we show that neural SDEs more effectively capture the inherent stochasticity of transition dynamics, enabling high-performing policies with improved sample efficiency in challenging scenarios. We leverage neural ODEs and SDEs for efficient policy adaptation to changes in environment dynamics via inverse models, requiring only limited interactions with the new environment. To address partial observability, we introduce a latent SDE model that combines an ODE with a GAN-trained stochastic component in latent space. Policies derived from this model provide a strong baseline, outperforming or matching general model-based and model-free approaches across stochastic continuous-control benchmarks. This work demonstrates the applicability of action-conditional latent SDEs for RL planning in environments with stochastic transitions. Our code is available at: https://github.com/ChaoHan-UoS/NeuralRL

Related papers

• Expanding the Chaos: Neural Operator for Stochastic (Partial) Differential Equations [65.80144621950981]
  We build on Wiener chaos expansions (WCE) to design neural operator (NO) architectures for SPDEs and SDEs.<n>We show that WCE-based neural operators provide a practical and scalable way to learn SDE/SPDE solution operators.
  arXiv  Detail & Related papers  (2026-01-03T00:59:25Z)
• HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training [3.4515388499147654]
  We introduce HGAN-SDEs, a novel GAN-based framework that leverages Neural Hermite functions to construct a structured and efficient discriminator.<n>HGAN-SDEs achieve superior sample quality and learning efficiency compared to existing generative models for SDEs.
  arXiv  Detail & Related papers  (2025-12-23T11:25:22Z)
• A joint optimization approach to identifying sparse dynamics using least squares kernel collocation [70.13783231186183]
  We develop an all-at-once modeling framework for learning systems of ordinary differential equations (ODE) from scarce, partial, and noisy observations of the states.<n>The proposed methodology amounts to a combination of sparse recovery strategies for the ODE over a function library combined with techniques from reproducing kernel Hilbert space (RKHS) theory for estimating the state and discretizing the ODE.
  arXiv  Detail & Related papers  (2025-11-23T18:04:15Z)
• Neural SDEs as a Unified Approach to Continuous-Domain Sequence Modeling [3.8980564330208662]
  We propose a novel and intuitive approach to continuous sequence modeling.<n>Our method interprets time-series data as textitdiscrete samples from an underlying continuous dynamical system.<n>We derive a maximum principled objective and a textitsimulation-free scheme for efficient training of our Neural SDE model.
  arXiv  Detail & Related papers  (2025-01-31T03:47:22Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics.<n>We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• Learning Space-Time Continuous Neural PDEs from Partially Observed States [13.01244901400942]
  We introduce a grid-independent model learning partial differential equations (PDEs) from noisy and partial observations on irregular grids. We propose a space-time continuous latent neural PDE model with an efficient probabilistic framework and a novel design encoder for improved data efficiency and grid independence.
  arXiv  Detail & Related papers  (2023-07-09T06:53:59Z)
• Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs [70.51212431290611]
  Partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. We propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-04-13T08:53:41Z)
• Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization [60.73540999409032]
  We show that expressive autoregressive dynamics models generate different dimensions of the next state and reward sequentially conditioned on previous dimensions. We also show that autoregressive dynamics models are useful for offline policy optimization by serving as a way to enrich the replay buffer.
  arXiv  Detail & Related papers  (2021-04-28T16:48:44Z)
• Robust pricing and hedging via neural SDEs [0.0]
  We develop and analyse novel algorithms needed for efficient use of neural SDEs. We find robust bounds for prices of derivatives and the corresponding hedging strategies while incorporating relevant market data. Neural SDEs allow consistent calibration under both the risk-neutral and the real-world measures.
  arXiv  Detail & Related papers  (2020-07-08T14:33:17Z)
• Stochasticity in Neural ODEs: An Empirical Study [68.8204255655161]
  Regularization of neural networks (e.g. dropout) is a widespread technique in deep learning that allows for better generalization. We show that data augmentation during the training improves the performance of both deterministic and versions of the same model. However, the improvements obtained by the data augmentation completely eliminate the empirical regularization gains, making the performance of neural ODE and neural SDE negligible.
  arXiv  Detail & Related papers  (2020-02-22T22:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.