Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs

• URL: http://arxiv.org/abs/2305.08849v1
• Date: Mon, 15 May 2023 17:59:02 GMT
• Title: Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs
• Authors: Karthik Elamvazhuthi, Xuechen Zhang, Samet Oymak, Fabio Pasqualetti
• Abstract summary: We study a class of neural ordinary differential equations that leave a given manifold invariant. We show that any map that can be represented as the flow of a manifold-constrained dynamical system can be approximated using the flow of manifold-constrained neural ODE.
• Score: 29.87898857250788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In numerous robotics and mechanical engineering applications, among others, data is often constrained on smooth manifolds due to the presence of rotational degrees of freedom. Common datadriven and learning-based methods such as neural ordinary differential equations (ODEs), however, typically fail to satisfy these manifold constraints and perform poorly for these applications. To address this shortcoming, in this paper we study a class of neural ordinary differential equations that, by design, leave a given manifold invariant, and characterize their properties by leveraging the controllability properties of control affine systems. In particular, using a result due to Agrachev and Caponigro on approximating diffeomorphisms with flows of feedback control systems, we show that any map that can be represented as the flow of a manifold-constrained dynamical system can also be approximated using the flow of manifold-constrained neural ODE, whenever a certain controllability condition is satisfied. Additionally, we show that this universal approximation property holds when the neural ODE has limited width in each layer, thus leveraging the depth of network instead for approximation. We verify our theoretical findings using numerical experiments on PyTorch for the manifolds S2 and the 3-dimensional orthogonal group SO(3), which are model manifolds for mechanical systems such as spacecrafts and satellites. We also compare the performance of the manifold invariant neural ODE with classical neural ODEs that ignore the manifold invariant properties and show the superiority of our approach in terms of accuracy and sample complexity.

Related papers

• SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases [66.61789780666727]
  We show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. We also offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states. Our model yields a significant improvement over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-08-25T07:15:58Z)
• Sample-efficient Model-based Reinforcement Learning for Quantum Control [0.2999888908665658]
  We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization. We show an order of magnitude advantage in the sample complexity of our method over standard model-free RL. Our algorithm is well suited for controlling partially characterised one and two qubit systems.
  arXiv  Detail & Related papers  (2023-04-19T15:05:19Z)
• Constraining Gaussian Processes to Systems of Linear Ordinary Differential Equations [5.33024001730262]
  LODE-GPs follow a system of linear homogeneous ODEs with constant coefficients. We show the effectiveness of LODE-GPs in a number of experiments.
  arXiv  Detail & Related papers  (2022-08-26T09:16:53Z)
• Neural Laplace: Learning diverse classes of differential equations in the Laplace domain [86.52703093858631]
  We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
  arXiv  Detail & Related papers  (2022-06-10T02:14:59Z)
• Equivariant Graph Mechanics Networks with Constraints [83.38709956935095]
  We propose Graph Mechanics Network (GMN) which is efficient, equivariant and constraint-aware. GMN represents, by generalized coordinates, the forward kinematics information (positions and velocities) of a structural object. Extensive experiments support the advantages of GMN compared to the state-of-the-art GNNs in terms of prediction accuracy, constraint satisfaction and data efficiency.
  arXiv  Detail & Related papers  (2022-03-12T14:22:14Z)
• Decimation technique for open quantum systems: a case study with driven-dissipative bosonic chains [62.997667081978825]
  Unavoidable coupling of quantum systems to external degrees of freedom leads to dissipative (non-unitary) dynamics. We introduce a method to deal with these systems based on the calculation of (dissipative) lattice Green's function. We illustrate the power of this method with several examples of driven-dissipative bosonic chains of increasing complexity.
  arXiv  Detail & Related papers  (2022-02-15T19:00:09Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Neural Stochastic Partial Differential Equations [1.2183405753834562]
  We introduce the Neural SPDE model providing an extension to two important classes of physics-inspired neural architectures. On the one hand, it extends all the popular neural -- ordinary, controlled, rough -- differential equation models in that it is capable of processing incoming information. On the other hand, it extends Neural Operators -- recent generalizations of neural networks modelling mappings between functional spaces -- in that it can be used to learn complex SPDE solution operators.
  arXiv  Detail & Related papers  (2021-10-19T20:35:37Z)
• Go with the Flow: Adaptive Control for Neural ODEs [10.265713480189484]
  We describe a new module called neurally controlled ODE (N-CODE) designed to improve the expressivity of NODEs. N-CODE modules are dynamic variables governed by a trainable map from initial or current activation state. A single module is sufficient for learning a distribution on non-autonomous flows that adaptively drive neural representations.
  arXiv  Detail & Related papers  (2020-06-16T22:21:15Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Neural Ordinary Differential Equations on Manifolds [0.342658286826597]
  Recently normalizing flows in Euclidean space based on Neural ODEs show great promise, yet suffer the same limitations. We show how vector fields provide a general framework for parameterizing a flexible class of invertible mapping on these spaces.
  arXiv  Detail & Related papers  (2020-06-11T17:56:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.