Towards Foundation Inference Models that Learn ODEs In-Context

• URL: http://arxiv.org/abs/2510.12650v1
• Date: Tue, 14 Oct 2025 15:44:54 GMT
• Title: Towards Foundation Inference Models that Learn ODEs In-Context
• Authors: Maximilian Mauel, Manuel Hinz, Patrick Seifner, David Berghaus, Ramses J. Sanchez,
• Abstract summary: We introduce FIM-ODE (Foundation Inference Model for ODEs), a pretrained neural model designed to estimate ODEs from sparse and noisy observations.<n>Trained on synthetic data, the model utilizes a flexible neural operator for robust ODE inference, even from corrupted data.<n>We empirically verify that FIM-ODE provides accurate estimates, on par with a neural state-of-the-art method, and qualitatively compare the structure of their estimated vector fields.
• Score: 3.4253416336476246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ordinary differential equations (ODEs) describe dynamical systems evolving deterministically in continuous time. Accurate data-driven modeling of systems as ODEs, a central problem across the natural sciences, remains challenging, especially if the data is sparse or noisy. We introduce FIM-ODE (Foundation Inference Model for ODEs), a pretrained neural model designed to estimate ODEs zero-shot (i.e., in context) from sparse and noisy observations. Trained on synthetic data, the model utilizes a flexible neural operator for robust ODE inference, even from corrupted data. We empirically verify that FIM-ODE provides accurate estimates, on par with a neural state-of-the-art method, and qualitatively compare the structure of their estimated vector fields.

Related papers

• CODE: A global approach to ODE dynamics learning [1.1499574149885023]
  In data-driven settings, one learns the ODE's right-hand side (RHS)<n>In this work we introduce ChaosODE (CODE), a Polynomial Chaos ODE Expansion.<n>We evaluate the performance of CODE in several experiments on the Lotka-Volterra system.
  arXiv  Detail & Related papers  (2025-11-19T17:04:24Z)
• A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series [7.01848433242846]
  We propose the use of the weak formulation as a complementary approach to the classical strong formulation of data-driven time-series forecasting models.<n>We show that our proposed training strategy, which we coined as the Weak-Penalty NODE (WP-NODE), achieves state-of-the-art forecasting accuracy and exceptional robustness across benchmark chaotic dynamical systems.
  arXiv  Detail & Related papers  (2025-11-10T01:40:35Z)
• No Equations Needed: Learning System Dynamics Without Relying on Closed-Form ODEs [56.78271181959529]
  This paper proposes a conceptual shift to modeling low-dimensional dynamical systems by departing from the traditional two-step modeling process.<n>Instead of first discovering a closed-form equation and then analyzing it, our approach, direct semantic modeling, predicts the semantic representation of the dynamical system.<n>Our approach not only simplifies the modeling pipeline but also enhances the transparency and flexibility of the resulting models.
  arXiv  Detail & Related papers  (2025-01-30T18:36:48Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Zero-shot Imputation with Foundation Inference Models for Dynamical Systems [5.549794481031468]
  We offer a fresh perspective on the classical problem of imputing missing time series data, whose underlying dynamics are assumed to be determined by ODEs.<n>We propose a novel supervised learning framework for zero-shot time series imputation, through parametric functions satisfying some (hidden) ODEs.<n>We empirically demonstrate that one and the same (pretrained) recognition model can perform zero-shot imputation across 63 distinct time series with missing values.
  arXiv  Detail & Related papers  (2024-02-12T11:48:54Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Analysis of Numerical Integration in RNN-Based Residuals for Fault Diagnosis of Dynamic Systems [0.6999740786886536]
  The paper includes a case study of a heavy-duty truck's after-treatment system to highlight the potential of these techniques for improving fault diagnosis performance. Data-driven modeling and machine learning are widely used to model the behavior of dynamic systems.
  arXiv  Detail & Related papers  (2023-05-08T12:48:18Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• Neural ODEs with Irregular and Noisy Data [8.349349605334316]
  We discuss a methodology to learn differential equation(s) using noisy and irregular sampled measurements. In our methodology, the main innovation can be seen in the integration of deep neural networks with the neural ordinary differential equations (ODEs) approach. The proposed framework to learn a model describing the vector field is highly effective under noisy measurements.
  arXiv  Detail & Related papers  (2022-05-19T11:24:41Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Stochasticity in Neural ODEs: An Empirical Study [68.8204255655161]
  Regularization of neural networks (e.g. dropout) is a widespread technique in deep learning that allows for better generalization. We show that data augmentation during the training improves the performance of both deterministic and versions of the same model. However, the improvements obtained by the data augmentation completely eliminate the empirical regularization gains, making the performance of neural ODE and neural SDE negligible.
  arXiv  Detail & Related papers  (2020-02-22T22:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.