FxTS-Net: Fixed-Time Stable Learning Framework for Neural ODEs

• URL: http://arxiv.org/abs/2411.09118v1
• Date: Thu, 14 Nov 2024 01:37:24 GMT
• Title: FxTS-Net: Fixed-Time Stable Learning Framework for Neural ODEs
• Authors: Chaoyang Luo, Yan Zou, Wanying Li, Nanjing Huang,
• Abstract summary: We propose a new method for training Neural ODEs using fixed-time stability (FxTS) Lyapunov conditions. Our framework, called FxTS-Net, is based on the novel FxTS loss (FxTS-Loss) designed on Lyapunov functions. We find that FxTS-Net provides better prediction performance and better robustness under input perturbation.
• Score: 0.48123217909844934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural Ordinary Differential Equations (Neural ODEs), as a novel category of modeling big data methods, cleverly link traditional neural networks and dynamical systems. However, it is challenging to ensure the dynamics system reaches a correctly predicted state within a user-defined fixed time. To address this problem, we propose a new method for training Neural ODEs using fixed-time stability (FxTS) Lyapunov conditions. Our framework, called FxTS-Net, is based on the novel FxTS loss (FxTS-Loss) designed on Lyapunov functions, which aims to encourage convergence to accurate predictions in a user-defined fixed time. We also provide an innovative approach for constructing Lyapunov functions to meet various tasks and network architecture requirements, achieved by leveraging supervised information during training. By developing a more precise time upper bound estimation for bounded non-vanishingly perturbed systems, we demonstrate that minimizing FxTS-Loss not only guarantees FxTS behavior of the dynamics but also input perturbation robustness. For optimising FxTS-Loss, we also propose a learning algorithm, in which the simulated perturbation sampling method can capture sample points in critical regions to approximate FxTS-Loss. Experimentally, we find that FxTS-Net provides better prediction performance and better robustness under input perturbation.

Related papers

• MesaNet: Sequence Modeling by Locally Optimal Test-Time Training [67.45211108321203]
  We introduce a numerically stable, chunkwise parallelizable version of the recently proposed Mesa layer.<n>We show that optimal test-time training enables reaching lower language modeling perplexity and higher downstream benchmark performance than previous RNNs.
  arXiv  Detail & Related papers  (2025-06-05T16:50:23Z)
• LaPON: A Lagrange's-mean-value-theorem-inspired operator network for solving PDEs and its application on NSE [8.014720523981385]
  We propose LaPON, an operator network inspired by the Lagrange's mean value theorem.<n>It embeds prior knowledge directly into the neural network architecture instead of the loss function.<n>LaPON provides a scalable and reliable solution for high-fidelity fluid dynamics simulation.
  arXiv  Detail & Related papers  (2025-05-18T10:45:17Z)
• Introducing Interval Neural Networks for Uncertainty-Aware System Identification [2.8391355909797644]
  This paper introduces a framework for constructing and learning Interval Neural Networks (INNs) to perform uncertainty quantification (UQ) in System Identification (SysID) tasks. INNs are derived by transforming the learnable parameters (LPs) of pre-trained neural networks into interval-valued LPs without relying on probabilistic assumptions. We extend Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equations (Neural ODEs) into Interval LSTM (ILSTM) and Interval NODE (INODE) architectures, providing the mathematical foundations for their application in SysID.
  arXiv  Detail & Related papers  (2025-04-26T08:16:46Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems [20.006163951844357]
  We propose a simulation-free framework for training neural ordinary differential equations (NODEs) We employ the Fourier analysis to estimate temporal and potential high-order spatial gradients from noisy observational data. Our approach outperforms state-of-the-art methods in terms of training time, dynamics prediction, and robustness.
  arXiv  Detail & Related papers  (2024-05-19T13:15:23Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Optimal Transport-inspired Deep Learning Framework for Slow-Decaying Problems: Exploiting Sinkhorn Loss and Wasserstein Kernel [0.0]
  Reduced order models (ROMs) are widely used in scientific computing to tackle high-dimensional systems. We propose a novel ROM framework that integrates optimal transport theory and neural network-based methods. Our framework outperforms traditional ROM methods in terms of accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2023-08-26T10:24:43Z)
• Spintronics for image recognition: performance benchmarking via ultrafast data-driven simulations [4.2412715094420665]
  We present a demonstration of image classification using an echo-state network (ESN) relying on a single simulated spintronic nanostructure. We employ an ultrafast data-driven simulation framework called the data-driven Thiele equation approach to simulate the STVO dynamics. We showcase the versatility of our solution by successfully applying it to solve classification challenges with the MNIST, EMNIST-letters and Fashion MNIST datasets.
  arXiv  Detail & Related papers  (2023-08-10T18:09:44Z)
• Learning Signal Temporal Logic through Neural Network for Interpretable Classification [13.829082181692872]
  We propose an explainable neural-symbolic framework for the classification of time-series behaviors. We demonstrate the computational efficiency, compactness, and interpretability of the proposed method through driving scenarios and naval surveillance case studies.
  arXiv  Detail & Related papers  (2022-10-04T21:11:54Z)
• Learning Fast and Slow for Online Time Series Forecasting [76.50127663309604]
  Fast and Slow learning Networks (FSNet) is a holistic framework for online time-series forecasting. FSNet balances fast adaptation to recent changes and retrieving similar old knowledge. Our code will be made publicly available.
  arXiv  Detail & Related papers  (2022-02-23T18:23:07Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.