Deep Efficient Continuous Manifold Learning for Time Series Modeling
- URL: http://arxiv.org/abs/2112.03379v2
- Date: Fri, 6 Oct 2023 00:44:19 GMT
- Title: Deep Efficient Continuous Manifold Learning for Time Series Modeling
- Authors: Seungwoo Jeong, Wonjun Ko, Ahmad Wisnu Mulyadi, Heung-Il Suk
- Abstract summary: A symmetric positive definite matrix is being studied in computer vision, signal processing, and medical image analysis.
In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifold and a Cholesky space.
For dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network.
- Score: 11.876985348588477
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Modeling non-Euclidean data is drawing extensive attention along with the
unprecedented successes of deep neural networks in diverse fields.
Particularly, a symmetric positive definite matrix is being actively studied in
computer vision, signal processing, and medical image analysis, due to its
ability to learn beneficial statistical representations. However, owing to its
rigid constraints, it remains challenging to optimization problems and
inefficient computational costs, especially, when incorporating it with a deep
learning framework. In this paper, we propose a framework to exploit a
diffeomorphism mapping between Riemannian manifolds and a Cholesky space, by
which it becomes feasible not only to efficiently solve optimization problems
but also to greatly reduce computation costs. Further, for dynamic modeling of
time-series data, we devise a continuous manifold learning method by
systematically integrating a manifold ordinary differential equation and a
gated recurrent neural network. It is worth noting that due to the nice
parameterization of matrices in a Cholesky space, training our proposed network
equipped with Riemannian geometric metrics is straightforward. We demonstrate
through experiments over regular and irregular time-series datasets that our
proposed model can be efficiently and reliably trained and outperforms existing
manifold methods and state-of-the-art methods in various time-series tasks.
Related papers
- Learning Controlled Stochastic Differential Equations [61.82896036131116]
This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion.
We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity.
Our method is available as an open-source Python library.
arXiv Detail & Related papers (2024-11-04T11:09:58Z) - Invertible Solution of Neural Differential Equations for Analysis of
Irregularly-Sampled Time Series [4.14360329494344]
We propose an invertible solution of Neural Differential Equations (NDE)-based method to handle the complexities of irregular and incomplete time series data.
Our method suggests the variation of Neural Controlled Differential Equations (Neural CDEs) with Neural Flow, which ensures invertibility while maintaining a lower computational burden.
At the core of our approach is an enhanced dual latent states architecture, carefully designed for high precision across various time series tasks.
arXiv Detail & Related papers (2024-01-10T07:51:02Z) - Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization.
We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
arXiv Detail & Related papers (2023-10-09T14:30:06Z) - Deep Learning-based surrogate models for parametrized PDEs: handling
geometric variability through graph neural networks [0.0]
This work explores the potential usage of graph neural networks (GNNs) for the simulation of time-dependent PDEs.
We propose a systematic strategy to build surrogate models based on a data-driven time-stepping scheme.
We show that GNNs can provide a valid alternative to traditional surrogate models in terms of computational efficiency and generalization to new scenarios.
arXiv Detail & Related papers (2023-08-03T08:14:28Z) - MMGP: a Mesh Morphing Gaussian Process-based machine learning method for
regression of physical problems under non-parameterized geometrical
variability [0.30693357740321775]
We propose a machine learning method that do not rely on graph neural networks.
The proposed methodology can easily deal with large meshes without the need for explicit shape parameterization.
arXiv Detail & Related papers (2023-05-22T09:50:15Z) - A Causality-Based Learning Approach for Discovering the Underlying
Dynamics of Complex Systems from Partial Observations with Stochastic
Parameterization [1.2882319878552302]
This paper develops a new iterative learning algorithm for complex turbulent systems with partial observations.
It alternates between identifying model structures, recovering unobserved variables, and estimating parameters.
Numerical experiments show that the new algorithm succeeds in identifying the model structure and providing suitable parameterizations for many complex nonlinear systems.
arXiv Detail & Related papers (2022-08-19T00:35:03Z) - Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
We consider the impact of the training set and its structure on the quality of the long-term prediction.
We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
arXiv Detail & Related papers (2021-12-15T20:09:20Z) - Learning Time-Varying Graphs from Online Data [39.21234914444073]
This work proposes an algorithmic framework to learn time-varying graphs from online data.
It renders it model-independent, i.e., it can be theoretically analyzed in its abstract formulation.
We specialize the framework to three well-known graph learning models, namely, the Gaussian graphical model (GGM), the structural equation model (SEM), and the smoothness-based model (SBM)
arXiv Detail & Related papers (2021-10-21T09:46:44Z) - Optimization on manifolds: A symplectic approach [127.54402681305629]
We propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems.
Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.
arXiv Detail & Related papers (2021-07-23T13:43:34Z) - Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems.
Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC.
We provide a practical parametrization of the core optimization problem.
arXiv Detail & Related papers (2021-07-08T18:01:02Z) - Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
There is an increasing need for active learning algorithms that are compatible with deep neural networks.
This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
arXiv Detail & Related papers (2021-06-17T17:26:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.