Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets

• URL: http://arxiv.org/abs/2510.04138v1
• Date: Sun, 05 Oct 2025 10:36:14 GMT
• Title: Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets
• Authors: Muhao Guo, Haoran Li, Yang Weng,
• Abstract summary: We propose a novel approach to explore the underlying manifold to restrict the ODE process.<n>Specifically, we employ a structure-preserved encoder to process data and find the underlying graph to approximate the manifold.<n>Our results demonstrate superior performance, underscoring the effectiveness of our approach in addressing the challenges of high-dimensional datasets.
• Score: 8.436711484752365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural ordinary differential equations (NODE) have garnered significant attention for their design of continuous-depth neural networks and the ability to learn data/feature dynamics. However, for high-dimensional systems, estimating dynamics requires extensive calculations and suffers from high truncation errors for the ODE solvers. To address the issue, one intuitive approach is to consider the non-trivial topological space of the data distribution, i.e., a low-dimensional manifold. Existing methods often rely on knowledge of the manifold for projection or implicit transformation, restricting the ODE solutions on the manifold. Nevertheless, such knowledge is usually unknown in realistic scenarios. Therefore, we propose a novel approach to explore the underlying manifold to restrict the ODE process. Specifically, we employ a structure-preserved encoder to process data and find the underlying graph to approximate the manifold. Moreover, we propose novel methods to combine the NODE learning with the manifold, resulting in significant gains in computational speed and accuracy. Our experimental evaluations encompass multiple datasets, where we compare the accuracy, number of function evaluations (NFEs), and convergence speed of our model against existing baselines. Our results demonstrate superior performance, underscoring the effectiveness of our approach in addressing the challenges of high-dimensional datasets.

Related papers

• Multi-Dimensional Visual Data Recovery: Scale-Aware Tensor Modeling and Accelerated Randomized Computation [51.65236537605077]
  We propose a new type of network compression optimization technique, fully randomized tensor network compression (FCTN)<n>FCTN has significant advantages in correlation characterization and transpositional in algebra, and has notable achievements in multi-dimensional data processing and analysis.<n>We derive efficient algorithms with guarantees to solve the formulated models.
  arXiv  Detail & Related papers  (2026-02-13T14:56:37Z)
• A joint optimization approach to identifying sparse dynamics using least squares kernel collocation [70.13783231186183]
  We develop an all-at-once modeling framework for learning systems of ordinary differential equations (ODE) from scarce, partial, and noisy observations of the states.<n>The proposed methodology amounts to a combination of sparse recovery strategies for the ODE over a function library combined with techniques from reproducing kernel Hilbert space (RKHS) theory for estimating the state and discretizing the ODE.
  arXiv  Detail & Related papers  (2025-11-23T18:04:15Z)
• Towards Foundation Inference Models that Learn ODEs In-Context [3.4253416336476246]
  We introduce FIM-ODE (Foundation Inference Model for ODEs), a pretrained neural model designed to estimate ODEs from sparse and noisy observations.<n>Trained on synthetic data, the model utilizes a flexible neural operator for robust ODE inference, even from corrupted data.<n>We empirically verify that FIM-ODE provides accurate estimates, on par with a neural state-of-the-art method, and qualitatively compare the structure of their estimated vector fields.
  arXiv  Detail & Related papers  (2025-10-14T15:44:54Z)
• Learning functions through Diffusion Maps [0.6768558752130311]
  We build on the Diffusion Maps framework under the manifold hypothesis.<n>We introduce a dimensionality reduction strategy based on the low-rank structure of the distance matrix.<n>Online updating mechanism enables efficient incorporation of new data.
  arXiv  Detail & Related papers  (2025-09-03T22:57:33Z)
• Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics [5.380276949049726]
  We develop and analyze an efficient high-dimensional Partial Differential Equations solver based on Deep Learning (DL) We show, both theoretically and numerically, that it can compete with a novel stable and accurate compressive spectral collocation method.
  arXiv  Detail & Related papers  (2024-06-03T17:16:11Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Score-based Diffusion Models in Function Space [137.70916238028306]
  Diffusion models have recently emerged as a powerful framework for generative modeling.<n>This work introduces a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space.<n>We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patterns [77.34726150561087]
  We introduce four complicated missing patterns, including missing and three fiber-like missing cases according to the mode-drivenn fibers. Despite nonity of the objective function in our model, we derive the optimal solutions by integrating alternating data-mputation method of multipliers.
  arXiv  Detail & Related papers  (2022-05-19T08:37:56Z)
• Distributional Gradient Matching for Learning Uncertain Neural Dynamics Models [38.17499046781131]
  We propose a novel approach towards estimating uncertain neural ODEs, avoiding the numerical integration bottleneck. Our algorithm - distributional gradient matching (DGM) - jointly trains a smoother and a dynamics model and matches their gradients via minimizing a Wasserstein loss. Our experiments show that, compared to traditional approximate inference methods based on numerical integration, our approach is faster to train, faster at predicting previously unseen trajectories, and in the context of neural ODEs, significantly more accurate.
  arXiv  Detail & Related papers  (2021-06-22T08:40:51Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.