ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance

• URL: http://arxiv.org/abs/2509.09611v1
• Date: Thu, 11 Sep 2025 16:52:54 GMT
• Title: ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance
• Authors: Haolan Zheng, Yanlai Chen, Jiequn Han, Yue Yu,
• Abstract summary: We propose a novel data-lean operator learning algorithm, the Reduced Basis Neural Operator (ReBaNO) to solve a group of PDEs with multiple distinct inputs.<n>Inspired by the Reduced Basis Method and the recently introduced Generative Pre-Trained Physics-Informed Neural Networks, ReBaNO relies on a mathematically rigorous greedy algorithm to build its network structure offline adaptively from the ground up.
• Score: 12.855964713673055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel data-lean operator learning algorithm, the Reduced Basis Neural Operator (ReBaNO), to solve a group of PDEs with multiple distinct inputs. Inspired by the Reduced Basis Method and the recently introduced Generative Pre-Trained Physics-Informed Neural Networks, ReBaNO relies on a mathematically rigorous greedy algorithm to build its network structure offline adaptively from the ground up. Knowledge distillation via task-specific activation function allows ReBaNO to have a compact architecture requiring minimal computational cost online while embedding physics. In comparison to state-of-the-art operator learning algorithms such as PCA-Net, DeepONet, FNO, and CNO, numerical results demonstrate that ReBaNO significantly outperforms them in terms of eliminating/shrinking the generalization gap for both in- and out-of-distribution tests and being the only operator learning algorithm achieving strict discretization invariance.

Related papers

• Quadratic Unconstrained Binary Optimisation for Training and Regularisation of Binary Neural Networks [0.0]
  Training binary neural networks (BNNs) is computationally challenging because of its discrete characteristics.<n>Recent work proposing a framework for training BNNs based on unconstrained binary optimisation (QUBO)<n>We extend existing QUBO models for training BNNs to accommodate arbitrary network topologies and propose two novel methods for regularisation.
  arXiv  Detail & Related papers  (2026-01-01T19:21:03Z)
• BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training [21.908847701590428]
  Binary Neural Networks (BNNs) offer substantial reductions in computational complexity, memory footprint, and energy consumption.<n>However, training BNNs via gradient-based optimization remains challenging due to the discrete nature of their variables.<n>This paper introduces Binary Error Propagation (BEP), the first learning algorithm to establish a principled, discrete analog of the backpropagation chain rule.
  arXiv  Detail & Related papers  (2025-12-03T19:03:55Z)
• Accelerating PDE Solvers with Equation-Recast Neural Operator Preconditioning [9.178290601589365]
  Minimal-Data Parametric Neural Operator Preconditioning (MD-PNOP) is a new paradigm for accelerating parametric PDE solvers.<n>It recasts the residual from parameter deviation as additional source term, where trained neural operators can be used to refine the solution in an offline fashion.<n>It consistently achieves 50% reduction in computational time while maintaining full order fidelity for fixed-source, single-group eigenvalue, and multigroup coupled eigenvalue problems.
  arXiv  Detail & Related papers  (2025-09-01T12:14:58Z)
• Unsupervised Learning in Echo State Networks for Input Reconstruction [0.0]
  Echo state networks (ESNs) are a class of recurrent neural networks in which only the readout layer is trainable.<n>In this study, we focus on input reconstruction (IR), where the readout layer is trained to reconstruct the input time series fed into the ESN.
  arXiv  Detail & Related papers  (2025-01-20T11:16:44Z)
• Efficient Training of Deep Neural Operator Networks via Randomized Sampling [0.0]
  We introduce a random sampling technique to be adopted the training of DeepONet.<n>We demonstrate substantial reductions in training time while achieving comparable or lower overall test errors relative to the traditional training approach.<n>Our results indicate that incorporating randomization in the trunk network inputs during training enhances the efficiency and robustness of DeepONet.
  arXiv  Detail & Related papers  (2024-09-20T07:18:31Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Analytically Tractable Inference in Deep Neural Networks [0.0]
  Tractable Approximate Inference (TAGI) algorithm was shown to be a viable and scalable alternative to backpropagation for shallow fully-connected neural networks. We are demonstrating how TAGI matches or exceeds the performance of backpropagation, for training classic deep neural network architectures.
  arXiv  Detail & Related papers  (2021-03-09T14:51:34Z)
• Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the Brain [62.997667081978825]
  Activation Relaxation (AR) is motivated by constructing the backpropagation gradient as the equilibrium point of a dynamical system. Our algorithm converges rapidly and robustly to the correct backpropagation gradients, requires only a single type of computational unit, and can operate on arbitrary computation graphs.
  arXiv  Detail & Related papers  (2020-09-11T11:56:34Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)
• Self-Organized Operational Neural Networks with Generative Neurons [87.32169414230822]
  ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators. We propose Self-organized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection.
  arXiv  Detail & Related papers  (2020-04-24T14:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.