How deep is your network? Deep vs. shallow learning of transfer operators

• URL: http://arxiv.org/abs/2509.19930v1
• Date: Wed, 24 Sep 2025 09:38:42 GMT
• Title: How deep is your network? Deep vs. shallow learning of transfer operators
• Authors: Mohammad Tabish, Benedict Leimkuhler, Stefan Klus,
• Abstract summary: We propose a randomized neural network approach called RaNNDy for learning transfer operators and their spectral decompositions from data.<n>The main advantage is that without a noticeable reduction in accuracy, this approach significantly reduces the training time and resources.<n>We present results for different dynamical operators, including Koopman and Perron-Frobenius operators, which have important applications in analyzing the behavior of complex dynamical systems.
• Score: 0.4473327661758546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a randomized neural network approach called RaNNDy for learning transfer operators and their spectral decompositions from data. The weights of the hidden layers of the neural network are randomly selected and only the output layer is trained. The main advantage is that without a noticeable reduction in accuracy, this approach significantly reduces the training time and resources while avoiding common problems associated with deep learning such as sensitivity to hyperparameters and slow convergence. Additionally, the proposed framework allows us to compute a closed-form solution for the output layer which directly represents the eigenfunctions of the operator. Moreover, it is possible to estimate uncertainties associated with the computed spectral properties via ensemble learning. We present results for different dynamical operators, including Koopman and Perron-Frobenius operators, which have important applications in analyzing the behavior of complex dynamical systems, and the Schr\"odinger operator. The numerical examples, which highlight the strengths but also weaknesses of the proposed framework, include several stochastic dynamical systems, protein folding processes, and the quantum harmonic oscillator.

Related papers

• Data-Driven Deep MIMO Detection:Network Architectures and Generalization Analysis [50.20709408241935]
  This paper proposes inspecting the fully data-driven DeepSIC detection within a Network-of-MLPs architecture.<n>Within such an architecture, DeepSIC can be upgraded as a graph-based message-passing process using Graph Neural Networks (GNNs)<n>GNNSIC achieves excellent expressivity comparable to DeepSIC with substantially fewer trainable parameters.
  arXiv  Detail & Related papers  (2026-02-13T04:38:51Z)
• An Evolutionary Multi-objective Optimization for Replica-Exchange-based Physics-informed Operator Learning Network [7.1950116347185995]
  We propose an evolutionary Multi-objective Optimization for Replica-based Physics-informed Operator learning Network.<n>Our framework consistently outperforms the general operator learning methods in accuracy, noise, and the ability to quantify uncertainty.
  arXiv  Detail & Related papers  (2025-08-31T02:17:59Z)
• Stochastic Fractional Neural Operators: A Symmetrized Approach to Modeling Turbulence in Complex Fluid Dynamics [0.0]
  We introduce a new class of neural network operators designed to handle problems where memory effects and randomness play a central role.<n>Our approach provides theoretical guarantees for the approximation quality and suggests that these neural operators can serve as effective tools in the analysis and simulation of complex systems.
  arXiv  Detail & Related papers  (2025-05-12T13:11:08Z)
• DeepONet Augmented by Randomized Neural Networks for Efficient Operator Learning in PDEs [5.84093922354671]
  We propose RaNN-DeepONets, a hybrid architecture designed to balance accuracy and efficiency.<n>RaNN-DeepONets achieves comparable accuracy while reducing computational costs by orders of magnitude.<n>These results highlight the potential of RaNN-DeepONets as an efficient alternative for operator learning in PDE-based systems.
  arXiv  Detail & Related papers  (2025-03-01T03:05:29Z)
• Efficient Training of Deep Neural Operator Networks via Randomized Sampling [0.0]
  We introduce a random sampling technique to be adopted the training of DeepONet.<n>We demonstrate substantial reductions in training time while achieving comparable or lower overall test errors relative to the traditional training approach.<n>Our results indicate that incorporating randomization in the trunk network inputs during training enhances the efficiency and robustness of DeepONet.
  arXiv  Detail & Related papers  (2024-09-20T07:18:31Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations [44.89798007370551]
  This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions.
  arXiv  Detail & Related papers  (2022-03-02T19:09:52Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Deep Neural Networks and PIDE discretizations [2.4063592468412276]
  We propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN) We propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and fractional inverse Laplacian operators. We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving.
  arXiv  Detail & Related papers  (2021-08-05T08:03:01Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• Self-Organized Operational Neural Networks with Generative Neurons [87.32169414230822]
  ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators. We propose Self-organized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection.
  arXiv  Detail & Related papers  (2020-04-24T14:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.