Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees

• URL: http://arxiv.org/abs/2402.08090v4
• Date: Wed, 08 Jan 2025 03:08:11 GMT
• Title: Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees
• Abstract summary: We present Extended Linearized Contracting Dynamics (ELCD), the first neural network-based dynamical system with global contractivity guarantees in arbitrary metrics.<n>In its most basic form, ELCD is guaranteed to be (i) globally exponentially stable, (ii) equilibrium contracting, and (iii) globally contracting with respect to some metric.<n>We demonstrate the performance of ELCD on the high dimensional LASA, multi-link pendulum, and Rosenbrock datasets.
• Score: 43.71741802581781
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Global stability and robustness guarantees in learned dynamical systems are essential to ensure well-behavedness of the systems in the face of uncertainty. We present Extended Linearized Contracting Dynamics (ELCD), the first neural network-based dynamical system with global contractivity guarantees in arbitrary metrics. The key feature of ELCD is a parametrization of the extended linearization of the nonlinear vector field. In its most basic form, ELCD is guaranteed to be (i) globally exponentially stable, (ii) equilibrium contracting, and (iii) globally contracting with respect to some metric. To allow for contraction with respect to more general metrics in the data space, we train diffeomorphisms between the data space and a latent space and enforce contractivity in the latent space, which ensures global contractivity in the data space. We demonstrate the performance of ELCD on the high dimensional LASA, multi-link pendulum, and Rosenbrock datasets.

Related papers

• Effective Covariance Dynamics in Solvable High-Dimensional GANs [2.639628765667673]
  We study a solvable high-dimensional model of generative adversarial network (GAN) training in which a linear generator learns a low-dimensional subspace from data.<n>We prove that the microscopic training process converges, in the high-dimensional limit, to deterministic ordinary differential equations governed by this effective covariance.
  arXiv  Detail & Related papers  (2026-06-25T16:33:54Z)
• When Does LeJEPA Learn a World Model? [48.74247491501631]
  A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization.<n>We prove that LeJEPA linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability.<n>We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control.
  arXiv  Detail & Related papers  (2026-05-25T22:56:26Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• scMRDR: A scalable and flexible framework for unpaired single-cell multi-omics data integration [53.683726781791385]
  We introduce a scalable and flexible generative framework called single-cell Multi-omics Regularized Disentangled Representations (scMRDR) for unpaired multi-omics integration.<n>Our method achieves excellent performance on benchmark datasets in terms of batch correction, modality alignment, and biological signal preservation.
  arXiv  Detail & Related papers  (2025-10-28T21:28:39Z)
• Stable Robot Motions on Manifolds: Learning Lyapunov-Constrained Neural Manifold ODEs [6.454329937275794]
  We propose a general framework for learning stable dynamical systems on Riemannian manifold.<n>Our method guarantees stability by projecting the neural vector field evolving on the manifold so that it strictly satisfies the Lyapunov stability criterion.<n>We demonstrate the performance, scalability, and practical applicability of our approach through extensive simulations and by learning robot motions in a real-world experiment.
  arXiv  Detail & Related papers  (2025-10-07T09:16:48Z)
• Regularity and Stability Properties of Selective SSMs with Discontinuous Gating [18.718025325906762]
  In this paper, we investigate the stability and regularity properties of continuous-time selective SSMs.<n>We establish that intrinsic energy dissipation guarantees exponential forgetting of past states.<n>Our findings offer a rigorous framework for understanding and designing stable and reliable deep selective SSMs.
  arXiv  Detail & Related papers  (2025-05-16T18:08:40Z)
• Generative System Dynamics in Recurrent Neural Networks [56.958984970518564]
  We investigate the continuous time dynamics of Recurrent Neural Networks (RNNs) We show that skew-symmetric weight matrices are fundamental to enable stable limit cycles in both linear and nonlinear configurations. Numerical simulations showcase how nonlinear activation functions not only maintain limit cycles, but also enhance the numerical stability of the system integration process.
  arXiv  Detail & Related papers  (2025-04-16T10:39:43Z)
• World-Consistent Data Generation for Vision-and-Language Navigation [52.08816337783936]
  Vision-and-Language Navigation (VLN) is a challenging task that requires an agent to navigate through photorealistic environments following natural-language instructions. One main obstacle existing in VLN is data scarcity, leading to poor generalization performance over unseen environments. We propose the world-consistent data generation (WCGEN), an efficacious data-augmentation framework satisfying both diversity and world-consistency.
  arXiv  Detail & Related papers  (2024-12-09T11:40:54Z)
• Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem [22.648448759446907]
  We show that a low-rank factorization serves as a building block in many learning tasks. We offer new insight into the shape of the trajectories associated with local search parts of the dynamics.
  arXiv  Detail & Related papers  (2024-11-24T20:05:10Z)
• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Inferring stability properties of chaotic systems on autoencoders' latent spaces [4.266376725904727]
  In chaotic systems and turbulence, convolutional autoencoders and echo state networks (CAE-ESN) successfully forecast the dynamics. We show that the CAE-ESN model infers the invariant stability properties and the geometry of the space vectors in the low-dimensional manifold. This work opens up new opportunities for inferring the stability of high-dimensional chaotic systems in latent spaces.
  arXiv  Detail & Related papers  (2024-10-23T16:25:36Z)
• Stability analysis of chaotic systems in latent spaces [4.266376725904727]
  We show that a latent-space approach can infer the solution of a chaotic partial differential equation. It can also predict the stability properties of the physical system.
  arXiv  Detail & Related papers  (2024-10-01T08:09:14Z)
• Unified theoretical guarantees for stability, consistency, and convergence in neural PDE solvers from non-IID data to physics-informed networks [0.0]
  We establish a unified theoretical framework addressing the stability, consistency, and convergence of neural networks under realistic training conditions.<n>For standard supervised learning with dependent data, we derive uniform stability bounds for gradient-based methods.<n>In federated learning with heterogeneous data, we quantify model inconsistency via curvature-aware aggregation and information-theoretic divergence.
  arXiv  Detail & Related papers  (2024-09-08T08:48:42Z)
• Data-Driven Stochastic Optimal Control in Reproducing Kernel Hilbert Spaces [4.722936792899906]
  This paper proposes a fully data-driven approach for optimal control of nonlinear control-affine systems represented by a diffusion.<n>The focus is on the scenario where both the nonlinear dynamics and stage cost functions are unknown, while only a control penalty function and constraints are provided.<n> Numerical results demonstrate its ability to address diverse nonlinear control tasks, including the depth regulation of an autonomous underwater vehicle.
  arXiv  Detail & Related papers  (2024-07-23T11:53:03Z)
• Ensuring Both Positivity and Stability Using Sector-Bounded Nonlinearity for Systems with Neural Network Controllers [0.0]
  We present a stability theorem that demonstrates the global exponential stability of linear systems under fully connected FFNN control. Our approach effectively addresses the challenge of ensuring stability in highly nonlinear systems. We showcase the practical applicability of our methodology through its implementation in a linear system managed by a FFNN trained on output feedback controller data.
  arXiv  Detail & Related papers  (2024-06-18T16:05:57Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction [49.66486092259376]
  The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. We provide a framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and gradient approximation.
  arXiv  Detail & Related papers  (2023-06-12T16:28:11Z)
• Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring [104.19414150171472]
  Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients. We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches. Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
  arXiv  Detail & Related papers  (2022-06-14T13:12:12Z)
• Towards Robust Unsupervised Disentanglement of Sequential Data -- A Case Study Using Music Audio [17.214062755082065]
  Disentangled sequential autoencoders (DSAEs) represent a class of probabilistic graphical models. We show that the vanilla DSAE suffers from being sensitive to the choice of model architecture and capacity of the dynamic latent variables. We propose TS-DSAE, a two-stage training framework that first learns sequence-level prior distributions.
  arXiv  Detail & Related papers  (2022-05-12T04:11:25Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.