Manifold meta-learning for reduced-complexity neural system identification

• URL: http://arxiv.org/abs/2504.11811v1
• Date: Wed, 16 Apr 2025 06:49:56 GMT
• Title: Manifold meta-learning for reduced-complexity neural system identification
• Authors: Marco Forgione, Ankush Chakrabarty, Dario Piga, Matteo Rufolo, Alberto Bemporad,
• Abstract summary: We propose a meta-learning framework that discovers a low-dimensional manifold.<n>This manifold is learned from a meta-dataset of input-output sequences generated by a class of related dynamical systems.<n>Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold.
• Score: 1.0276024900942875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources at training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from a meta-dataset of input-output sequences generated by a class of related dynamical systems, enabling efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and reducing the number of first-order updates required in inference, which could be expensive for large models. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.

Related papers

• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• SINDyG: Sparse Identification of Nonlinear Dynamical Systems from Graph-Structured Data [0.27624021966289597]
  We develop a new method called Sparse Identification of Dynamical Systems from Graph-structured data (SINDyG)<n>SINDyG incorporates the network structure into sparse regression to identify model parameters that explain the underlying network dynamics.<n>Our experiments validate the improved accuracy and simplicity of discovered network dynamics.
  arXiv  Detail & Related papers  (2024-09-02T17:51:37Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Physics guided neural networks for modelling of non-linear dynamics [0.0]
  This work demonstrates that injection of partially known information at an intermediate layer in a deep neural network can improve model accuracy, reduce model uncertainty, and yield improved convergence during the training. The value of these physics-guided neural networks has been demonstrated by learning the dynamics of a wide variety of nonlinear dynamical systems represented by five well-known equations in nonlinear systems theory.
  arXiv  Detail & Related papers  (2022-05-13T19:06:36Z)
• Deep transfer learning for system identification using long short-term memory neural networks [0.0]
  This paper proposes using two types of deep transfer learning, namely parameter fine-tuning and freezing, to reduce the data and computation requirements for system identification. Results show that compared with direct learning, our method accelerates learning by 10% to 50%, which also saves data and computing resources.
  arXiv  Detail & Related papers  (2022-04-06T23:39:06Z)
• Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets [0.0]
  We propose a bi-fidelity modeling approach for complex physical systems. We model the discrepancy between the true system's response and low-fidelity response in the presence of a small training dataset. We apply the approach to model systems that have parametric uncertainty and are partially unknown.
  arXiv  Detail & Related papers  (2022-04-03T05:30:57Z)
• Supervised DKRC with Images for Offline System Identification [77.34726150561087]
  Modern dynamical systems are becoming increasingly non-linear and complex. There is a need for a framework to model these systems in a compact and comprehensive representation for prediction and control. Our approach learns these basis functions using a supervised learning approach.
  arXiv  Detail & Related papers  (2021-09-06T04:39:06Z)
• A novel Deep Neural Network architecture for non-linear system identification [78.69776924618505]
  We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. Inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function) This architecture allows for automatic complexity selection based solely on available data.
  arXiv  Detail & Related papers  (2021-06-06T10:06:07Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.