Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelines

• URL: http://arxiv.org/abs/2602.19265v1
• Date: Sun, 22 Feb 2026 16:29:18 GMT
• Title: Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelines
• Authors: Siavash Khodakarami, Vivek Oommen, Nazanin Ahmadi Daryakenari, Maxim Beekenkamp, George Em Karniadakis,
• Abstract summary: spectral bias is when low-frequency components of a solution are learned faster than high-frequency modes.<n>We show that spectral bias is not simply representational but fundamentally dynamical.<n>For neural operators, we show that spectral bias is dependent on the neural operator architecture.
• Score: 3.758814046658822
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Solving partial differential equations (PDEs) by neural networks as well as Kolmogorov-Arnold Networks (KANs), including physics-informed neural networks (PINNs), physics-informed KANs (PIKANs), and neural operators, are known to exhibit spectral bias, whereby low-frequency components of the solution are learned significantly faster than high-frequency modes. While spectral bias is often treated as an intrinsic representational limitation of neural architectures, its interaction with optimization dynamics and physics-based loss formulations remains poorly understood. In this work, we provide a systematic investigation of spectral bias in physics-informed and operator learning frameworks, with emphasis on the coupled roles of network architecture, activation functions, loss design, and optimization strategy. We quantify spectral bias through frequency-resolved error metrics, Barron-norm diagnostics, and higher-order statistical moments, enabling a unified analysis across elliptic, hyperbolic, and dispersive PDEs. Through diverse benchmark problems, including the Korteweg-de Vries, wave and steady-state diffusion-reaction equations, turbulent flow reconstruction, and earthquake dynamics, we demonstrate that spectral bias is not simply representational but fundamentally dynamical. In particular, second-order optimization methods substantially alter the spectral learning order, enabling earlier and more accurate recovery of high-frequency modes for all PDE types. For neural operators, we further show that spectral bias is dependent on the neural operator architecture and can also be effectively mitigated through spectral-aware loss formulations without increasing the inference cost.

Related papers

• NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods [49.84495044725856]
  NeuraLSP is a novel neural preconditioner combined with a novel loss metric.<n>Our method exhibits both theoretical guarantees and empirical robustness to rank inflation, up to a 53% speedup.
  arXiv  Detail & Related papers  (2026-01-28T02:15:16Z)
• Graph Neural Regularizers for PDE Inverse Problems [62.49743146797144]
  We present a framework for solving a broad class of ill-posed inverse problems governed by partial differential equations (PDEs)<n>The forward problem is numerically solved using the finite element method (FEM)<n>We employ physics-inspired graph neural networks as learned regularizers, providing a robust, interpretable, and generalizable alternative to standard approaches.
  arXiv  Detail & Related papers  (2025-10-23T21:43:25Z)
• Neuro-Spectral Architectures for Causal Physics-Informed Networks [4.105582334433184]
  NeuSA is a novel class of PINNs inspired by classical spectral methods.<n>NeuSA learns a projection of the underlying PDE onto a spectral basis, leading to a finite-dimensional representation of the dynamics.<n>Code and pretrained models will be released.
  arXiv  Detail & Related papers  (2025-09-05T09:43:45Z)
• Fractional Spike Differential Equations Neural Network with Efficient Adjoint Parameters Training [63.3991315762955]
  Spiking Neural Networks (SNNs) draw inspiration from biological neurons to create realistic models for brain-like computation.<n>Most existing SNNs assume a single time constant for neuronal membrane voltage dynamics, modeled by first-order ordinary differential equations (ODEs) with Markovian characteristics.<n>We propose the Fractional SPIKE Differential Equation neural network (fspikeDE), which captures long-term dependencies in membrane voltage and spike trains through fractional-order dynamics.
  arXiv  Detail & Related papers  (2025-07-22T18:20:56Z)
• LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators [20.77877474840923]
  High-frequency information is a critical challenge in machine learning.<n>Deep neural nets exhibit the so-called spectral bias toward learning low-frequency components.<n>We propose a novel frequency-sensitive loss term based on radially binned spectral errors.
  arXiv  Detail & Related papers  (2025-04-05T19:35:04Z)
• Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems [1.6874375111244329]
  We introduce a new approach named high-frequency scaling (HFS) to mitigate spectral bias in convolutional-based neural operators.<n>We demonstrate a higher prediction accuracy by mitigating spectral bias in single and two-phase flow problems.
  arXiv  Detail & Related papers  (2025-03-17T20:08:47Z)
• Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations [0.0]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving Partial Differential Equations (PDEs)<n>These limitations impact their accuracy for problems involving rapid oscillations, sharp gradients, and complex boundary behaviors.<n>We investigate learnable activation functions as a solution to these challenges.
  arXiv  Detail & Related papers  (2024-11-22T18:25:13Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• The Spectral Bias of Polynomial Neural Networks [63.27903166253743]
  Polynomial neural networks (PNNs) have been shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Previous studies have revealed that neural networks demonstrate a $textitspectral bias$ towards low-frequency functions, which yields faster learning of low-frequency components during training. Inspired by such studies, we conduct a spectral analysis of the Tangent Kernel (NTK) of PNNs. We find that the $Pi$-Net family, i.e., a recently proposed parametrization of PNNs, speeds up the
  arXiv  Detail & Related papers  (2022-02-27T23:12:43Z)
• Neural Dynamic Mode Decomposition for End-to-End Modeling of Nonlinear Dynamics [49.41640137945938]
  We propose a neural dynamic mode decomposition for estimating a lift function based on neural networks. With our proposed method, the forecast error is backpropagated through the neural networks and the spectral decomposition. Our experiments demonstrate the effectiveness of our proposed method in terms of eigenvalue estimation and forecast performance.
  arXiv  Detail & Related papers  (2020-12-11T08:34:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.