Control, Optimal Transport and Neural Differential Equations in Supervised Learning

• URL: http://arxiv.org/abs/2503.15105v3
• Date: Mon, 19 May 2025 10:04:15 GMT
• Title: Control, Optimal Transport and Neural Differential Equations in Supervised Learning
• Authors: Minh-Nhat Phung, Minh-Binh Tran,
• Abstract summary: We study the fundamental computational problem of approximating optimal transport equations using neural differential equations (Neural ODEs)<n>We develop a novel framework for approximating unbalanced optimal transport (UOT) in the continuum using Neural ODEs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the fundamental computational problem of approximating optimal transport (OT) equations using neural differential equations (Neural ODEs). More specifically, we develop a novel framework for approximating unbalanced optimal transport (UOT) in the continuum using Neural ODEs. By generalizing a discrete UOT problem with Pearson divergence, we constructively design vector fields for Neural ODEs that converge to the true UOT dynamics, thereby advancing the mathematical foundations of computational transport and machine learning. To this end, we design a numerical scheme inspired by the Sinkhorn algorithm to solve the corresponding minimization problem and rigorously prove its convergence, providing explicit error estimates. From the obtained numerical solutions, we derive vector fields defining the transport dynamics and construct the corresponding transport equation. Finally, from the numerically obtained transport equation, we construct a neural differential equation whose flow converges to the true transport dynamics in an appropriate limiting regime.

Related papers

• Proximal optimal transport divergences [6.6875717609310765]
  We introduce proximal optimal transport divergence, a novel discrepancy measure that interpolates between information divergences and optimal transport distances via an infimal convolution formulation.<n>We explore its mathematical properties, including smoothness, boundedness, and computational tractability, and establish connections to primal-dual formulation and adversarial learning.<n>Our framework generalizes existing approaches while offering new insights and computational tools for generative modeling, distributional optimization, and gradient-based learning in probability spaces.
  arXiv  Detail & Related papers  (2025-05-17T17:48:11Z)
• Training Neural ODEs Using Fully Discretized Simultaneous Optimization [2.290491821371513]
  Training Neural Ordinary Differential Equations (Neural ODEs) requires solving differential equations at each epoch, leading to high computational costs.<n>In particular, we employ a collocation-based, fully discretized formulation and use IPOPT-a solver for large-scale nonlinear optimization.<n>Our results show significant potential for (collocation-based) simultaneous Neural ODE training pipelines.
  arXiv  Detail & Related papers  (2025-02-21T18:10:26Z)
• Convex Physics Informed Neural Networks for the Monge-Ampère Optimal Transport Problem [49.1574468325115]
  Optimal transportation of raw material from suppliers to customers is an issue arising in logistics.<n>A physics informed neuralnetwork method is advocated here for the solution of the corresponding generalized Monge-Ampere equation.<n>A particular focus is set on the enforcement of transport boundary conditions in the loss function.
  arXiv  Detail & Related papers  (2025-01-17T12:51:25Z)
• A Mathematical Analysis of Neural Operator Behaviors [0.0]
  This paper presents a rigorous framework for analyzing the behaviors of neural operators. We focus on their stability, convergence, clustering dynamics, universality, and generalization error. We aim to offer clear and unified guidance in a single setting for the future design of neural operator-based methods.
  arXiv  Detail & Related papers  (2024-10-28T19:38:53Z)
• Optimal Transportation by Orthogonal Coupling Dynamics [0.0]
  We propose a novel framework to address the Monge-Kantorovich problem based on a projection type gradient descent scheme. The micro-dynamics is built on the notion of the conditional expectation, where the connection with the opinion dynamics is explored. We demonstrate that the devised dynamics recovers random maps with favourable computational performance.
  arXiv  Detail & Related papers  (2024-10-10T15:53:48Z)
• Solving Poisson Equations using Neural Walk-on-Spheres [80.1675792181381]
  We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. We demonstrate the superiority of NWoS in accuracy, speed, and computational costs.
  arXiv  Detail & Related papers  (2024-06-05T17:59:22Z)
• A minimax optimal control approach for robust neural ODEs [44.99833362998488]
  We address the adversarial training of neural ODEs from a robust control perspective. We derive first order optimality conditions in the form of Pontryagin's Maximum Principle.
  arXiv  Detail & Related papers  (2023-10-26T17:07:43Z)
• A Computational Framework for Solving Wasserstein Lagrangian Flows [48.87656245464521]
  In general, the optimal density path is unknown, and solving these variational problems can be computationally challenging. We propose a novel deep learning based framework approaching all of these problems from a unified perspective. We showcase the versatility of the proposed framework by outperforming previous approaches for the single-cell trajectory inference.
  arXiv  Detail & Related papers  (2023-10-16T17:59:54Z)
• Physics-constrained neural differential equations for learning multi-ionic transport [0.0]
  We develop the first physics-informed deep learning model to learn ion transport behaviour across polyamide nanopores. We use neural differential equations in conjunction with classical closure models as inductive biases directly into the neural framework.
  arXiv  Detail & Related papers  (2023-03-07T17:18:52Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Neural Conservation Laws: A Divergence-Free Perspective [36.668126758052814]
  We propose building divergence-free neural networks through the concept of differential forms. We prove these models are universal and so can be used to represent any divergence-free vector field.
  arXiv  Detail & Related papers  (2022-10-04T17:01:53Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Online Learning to Transport via the Minimal Selection Principle [2.3857747529378917]
  We study the Online Learning Transport (OLT) problem where the decision variable is a convex, an-dimensional object. We derive a novel method called the minimal selection or exploration (SoMLT) algorithm to solve OLT problems using mean-field and discretization techniques.
  arXiv  Detail & Related papers  (2022-02-09T21:25:58Z)
• Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics [73.77459272878025]
  We propose to enhance the supervised signal in learning dynamics by pre-training a neural differential operator (NDO) NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives. We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library.
  arXiv  Detail & Related papers  (2021-06-08T08:04:47Z)
• Fourier Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  We formulate a new neural operator by parameterizing the integral kernel directly in Fourier space. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. It is up to three orders of magnitude faster compared to traditional PDE solvers.
  arXiv  Detail & Related papers  (2020-10-18T00:34:21Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)
• Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics [50.83356836818667]
  We introduce a new theoretical framework to analyze deep learning optimization with connection to its generalization error. Existing frameworks such as mean field theory and neural tangent kernel theory for neural network optimization analysis typically require taking limit of infinite width of the network to show its global convergence.
  arXiv  Detail & Related papers  (2020-07-11T18:19:50Z)
• A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models [93.24030378630175]
  We propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs) We derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities.
  arXiv  Detail & Related papers  (2019-10-31T02:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.