Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention

• URL: http://arxiv.org/abs/2510.04304v1
• Date: Sun, 05 Oct 2025 17:52:52 GMT
• Title: Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention
• Authors: Harshil Vejendla,
• Abstract summary: Wave-PDE Nets is a neural architecture whose elementary operation is a differentiable simulation of the second-order wave equation.<n>A symplectic spectral solver based on FFTs realises this propagation in O(nlog n) time.<n>On language and vision benchmarks, Wave-PDE Nets match or exceed Transformer performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Wave-PDE Nets, a neural architecture whose elementary operation is a differentiable simulation of the second-order wave equation. Each layer propagates its hidden state as a continuous field through a medium with trainable spatial velocity c(x) and damping {\gamma}(x). A symplectic spectral solver based on FFTs realises this propagation in O(nlog n) time. This oscillatory, global mechanism provides a powerful alternative to attention and first-order state-space models. We prove that a single Wave-PDE layer is a universal approximator. On language and vision benchmarks, Wave-PDE Nets match or exceed Transformer performance while demonstrating superior practical efficiency, reducing wall-clock time by up to 30% and peak memory by 25%. Ablation studies confirm the critical role of symplectic integration and a spectral Laplacian for stability and performance. Visualizations of the learned physical parameters reveal that the model learns intuitive strategies for information propagation. These results position Wave-PDE Nets as a computationally efficient and robust architecture with a strong physical inductive bias.

Related papers

• WaveFormer: Frequency-Time Decoupled Vision Modeling with Wave Equation [24.13944601660532]
  Vision modeling has advanced rapidly with Transformers, whose attention mechanisms capture visual dependencies but lack a principled account of how semantic information propagates spatially.<n>We revisit this problem from a wave-based perspective, treating feature maps as spatial signals whose evolution over an internal propagation time is governed by an underdamped wave equation.<n>We propose a family of WaveFormer models as drop-in replacements for standard ViTs and CNNs, achieving competitive accuracy across image classification, object detection, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-01-13T14:47:22Z)
• The Adaptive Vekua Cascade: A Differentiable Spectral-Analytic Solver for Physics-Informed Representation [0.0]
  Adaptive-based neural networks have emerged as a powerful tool for representing continuous physical fields.<n>They face two fundamental pathologies: spectral bias and the curse of dimensionality.<n>We propose a hybrid architecture that bridges analytic analytic decouples manifold learning from function approximation.
  arXiv  Detail & Related papers  (2025-12-12T18:41:35Z)
• Physics-informed waveform inversion using pretrained wavefield neural operators [9.048550821334116]
  Full waveform inversion (FWI) is crucial for reconstructing high-resolution subsurface models.<n>Recent attempts to accelerate FWI using learned wavefield neural operators have shown promise in efficiency and differentiability.<n>We introduce a novel physics-informed FWI framework to enhance the inversion in accuracy while maintaining the efficiency of neural operator-based FWI.
  arXiv  Detail & Related papers  (2025-09-10T19:57:18Z)
• Wave-Based Semantic Memory with Resonance-Based Retrieval: A Phase-Aware Alternative to Vector Embedding Stores [51.56484100374058]
  We propose a novel framework that models knowledge as wave patterns $psi(x) = A(x) eiphi(x)$ and retrieves it through resonance-based interference.<n>This approach preserves both amplitude and phase information, enabling more expressive and robust semantic similarity.
  arXiv  Detail & Related papers  (2025-08-21T10:13:24Z)
• Fractional Spike Differential Equations Neural Network with Efficient Adjoint Parameters Training [63.3991315762955]
  Spiking Neural Networks (SNNs) draw inspiration from biological neurons to create realistic models for brain-like computation.<n>Most existing SNNs assume a single time constant for neuronal membrane voltage dynamics, modeled by first-order ordinary differential equations (ODEs) with Markovian characteristics.<n>We propose the Fractional SPIKE Differential Equation neural network (fspikeDE), which captures long-term dependencies in membrane voltage and spike trains through fractional-order dynamics.
  arXiv  Detail & Related papers  (2025-07-22T18:20:56Z)
• An effective physics-informed neural operator framework for predicting wavefields [10.94738894332709]
  We introduce a physics-informed convolutional neural operator (PICNO) to solve the Helmholtz equation efficiently.<n> PICNO takes the background wavefield corresponding to a homogeneous medium and the velocity model as input function space, generating the scattered wavefield as the output function space.<n>It allows for high-resolution reasonably accurate predictions even with limited training samples.
  arXiv  Detail & Related papers  (2025-07-22T10:22:30Z)
• Graph Wave Networks [17.80926325018177]
  We develop a graph wave equation to leverage the wave propagation on graphs.<n>In details, we demonstrate that the graph wave equation can be connected to traditional spectral GNNs.<n>Experiments show that GWNs achieve SOTA and efficient performance on benchmark datasets.
  arXiv  Detail & Related papers  (2025-05-26T14:20:41Z)
• KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter [49.85369344101118]
  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with an efficient and high-quality motion reconstruction framework based on Kalman filtering. Our key idea is to model the dynamic radiance field as a dynamic system whose temporally varying states are estimated based on two sources of knowledge: observations and predictions. Our KFD-NeRF demonstrates similar or even superior performance within comparable computational time and state-of-the-art view synthesis performance with thorough training.
  arXiv  Detail & Related papers  (2024-07-18T05:48:24Z)
• SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases [66.61789780666727]
  We show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. We also offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states. Our model yields a significant improvement over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-08-25T07:15:58Z)
• Machine learning for phase-resolved reconstruction of nonlinear ocean wave surface elevations from sparse remote sensing data [37.69303106863453]
  We propose a novel approach for phase-resolved wave surface reconstruction using neural networks. Our approach utilizes synthetic yet highly realistic training data on uniform one-dimensional grids.
  arXiv  Detail & Related papers  (2023-05-18T12:30:26Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Fast Sampling of Diffusion Models via Operator Learning [74.37531458470086]
  We use neural operators, an efficient method to solve the probability flow differential equations, to accelerate the sampling process of diffusion models. Compared to other fast sampling methods that have a sequential nature, we are the first to propose a parallel decoding method. We show our method achieves state-of-the-art FID of 3.78 for CIFAR-10 and 7.83 for ImageNet-64 in the one-model-evaluation setting.
  arXiv  Detail & Related papers  (2022-11-24T07:30:27Z)
• NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer [45.47667026025716]
  We propose a novel, robust and accelerated iteration that relies on two key elements. The convergence and stability of the obtained method, referred to as NAG-GS, are first studied extensively. We show that NAG-arity is competitive with state-the-art methods such as momentum SGD with weight decay and AdamW for the training of machine learning models.
  arXiv  Detail & Related papers  (2022-09-29T16:54:53Z)
• Wave simulation in non-smooth media by PINN with quadratic neural network and PML condition [2.7651063843287718]
  The recently proposed physics-informed neural network (PINN) has achieved successful applications in solving a wide range of partial differential equations (PDEs) In this paper, we solve the acoustic and visco-acoustic scattered-field wave equation in the frequency domain with PINN instead of the wave equation to remove source perturbation. We show that PML and quadratic neurons improve the results as well as attenuation and discuss the reason for this improvement.
  arXiv  Detail & Related papers  (2022-08-16T13:29:01Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.