Hierarchical Stochastic Differential Equation Models for Latent Manifold Learning in Neural Time Series

• URL: http://arxiv.org/abs/2507.21531v1
• Date: Tue, 29 Jul 2025 06:51:58 GMT
• Title: Hierarchical Stochastic Differential Equation Models for Latent Manifold Learning in Neural Time Series
• Authors: Pedram Rajaei, Maryam Ostadsharif Memar, Navid Ziaei, Behzad Nazari, Ali Yousefi,
• Abstract summary: We propose a novel hierarchical differential equation (SDE) model that balances computational efficiency and interpretability.<n>We derive training and inference procedures and show that the computational cost of inference scales linearly with the length of the observation data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The manifold hypothesis suggests that high-dimensional neural time series lie on a low-dimensional manifold shaped by simpler underlying dynamics. To uncover this structure, latent dynamical variable models such as state-space models, recurrent neural networks, neural ordinary differential equations, and Gaussian Process Latent Variable Models are widely used. We propose a novel hierarchical stochastic differential equation (SDE) model that balances computational efficiency and interpretability, addressing key limitations of existing methods. Our model assumes the trajectory of a manifold can be reconstructed from a sparse set of samples from the manifold trajectory. The latent space is modeled using Brownian bridge SDEs, with points - specified in both time and value - sampled from a multivariate marked point process. These Brownian bridges define the drift of a second set of SDEs, which are then mapped to the observed data. This yields a continuous, differentiable latent process capable of modeling arbitrarily complex time series as the number of manifold points increases. We derive training and inference procedures and show that the computational cost of inference scales linearly with the length of the observation data. We then validate our model on both synthetic data and neural recordings to demonstrate that it accurately recovers the underlying manifold structure and scales effectively with data dimensionality.

Related papers

• Generative Learning for Slow Manifolds and Bifurcation Diagrams [0.35587965024910395]
  Conditional score-based generative models (cSGMs) have demonstrated capabilities in generating plausible data from target distributions conditioned on some given label.<n>We present a framework for using cSGMs to quickly initialize on a low-dimensional (reduced-order) slow manifold of a multi-time-scale system.<n>This conditional sampling can help uncover the geometry of the reduced slow-manifold and/or approximately fill in'' missing segments of steady states in a bifurcation diagram.
  arXiv  Detail & Related papers  (2025-04-29T02:38:44Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences.<n>We train the model using maximum likelihood estimation with Markov chain Monte Carlo.<n> Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Learning Space-Time Continuous Neural PDEs from Partially Observed States [13.01244901400942]
  We introduce a grid-independent model learning partial differential equations (PDEs) from noisy and partial observations on irregular grids. We propose a space-time continuous latent neural PDE model with an efficient probabilistic framework and a novel design encoder for improved data efficiency and grid independence.
  arXiv  Detail & Related papers  (2023-07-09T06:53:59Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Optimizing differential equations to fit data and predict outcomes [0.0]
  Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations.
  arXiv  Detail & Related papers  (2022-04-16T16:08:08Z)
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations [0.0]
  We present a data-driven reduced order modeling method that capitalizes on the chaotic dynamics of partial differential equations. We find that dimension reduction improves performance relative to predictions in the ambient space. With the low-dimensional model, we find excellent short- and long-time statistical recreation of the true dynamics for widely spaced data.
  arXiv  Detail & Related papers  (2021-08-31T20:00:33Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Identifying Latent Stochastic Differential Equations [29.103393300261587]
  We present a method for learning latent differential equations (SDEs) from high-dimensional time series data. The proposed method learns the mapping from ambient to latent space, and the underlying SDE coefficients, through a self-supervised learning approach. We validate the method through several simulated video processing tasks, where the underlying SDE is known, and through real world datasets.
  arXiv  Detail & Related papers  (2020-07-12T19:46:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.