STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem

• URL: http://arxiv.org/abs/2510.23986v1
• Date: Tue, 28 Oct 2025 01:43:54 GMT
• Title: STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem
• Authors: Hong Wang, Jiang Yixuan, Jie Wang, Xinyi Li, Jian Luo, Huanshuo Dong,
• Abstract summary: Operator eigenvalue problems play a critical role in various scientific fields and engineering applications.<n>Recent deep learning methods provide an efficient approach to address this challenge by iteratively updating neural networks.<n>We propose the Spectral Transformation Network (STNet), which consistently outperforms existing learning-based methods.
• Score: 10.27238431947351
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Operator eigenvalue problems play a critical role in various scientific fields and engineering applications, yet numerical methods are hindered by the curse of dimensionality. Recent deep learning methods provide an efficient approach to address this challenge by iteratively updating neural networks. These methods' performance relies heavily on the spectral distribution of the given operator: larger gaps between the operator's eigenvalues will improve precision, thus tailored spectral transformations that leverage the spectral distribution can enhance their performance. Based on this observation, we propose the Spectral Transformation Network (STNet). During each iteration, STNet uses approximate eigenvalues and eigenfunctions to perform spectral transformations on the original operator, turning it into an equivalent but easier problem. Specifically, we employ deflation projection to exclude the subspace corresponding to already solved eigenfunctions, thereby reducing the search space and avoiding converging to existing eigenfunctions. Additionally, our filter transform magnifies eigenvalues in the desired region and suppresses those outside, further improving performance. Extensive experiments demonstrate that STNet consistently outperforms existing learning-based methods, achieving state-of-the-art performance in accuracy.

Related papers

• SpectraKAN: Conditioning Spectral Operators [21.190440188964452]
  We introduce SpectraKAN, a neural operator that conditions the spectral operator on the input itself, turning spectral into an input-conditioned integral operator.<n>This is achieved by extracting a compact global representation from static-temporal history and using it to modulate a multi-scale trunk via single-query cross-attention.<n>Across diverse PDE benchmarks, SpectraKAN achieves state-of-the-art performance, reducing RMSE by up to 49% over strong baselines, with particularly large gains on challenging-temporal prediction tasks.
  arXiv  Detail & Related papers  (2026-02-05T01:30:25Z)
• An Evolutionary Multi-objective Optimization for Replica-Exchange-based Physics-informed Operator Learning Network [7.1950116347185995]
  We propose an evolutionary Multi-objective Optimization for Replica-based Physics-informed Operator learning Network.<n>Our framework consistently outperforms the general operator learning methods in accuracy, noise, and the ability to quantify uncertainty.
  arXiv  Detail & Related papers  (2025-08-31T02:17:59Z)
• Variance-Reducing Couplings for Random Features [57.73648780299374]
  Random features (RFs) are a popular technique to scale up kernel methods in machine learning. We find couplings to improve RFs defined on both Euclidean and discrete input spaces. We reach surprising conclusions about the benefits and limitations of variance reduction as a paradigm.
  arXiv  Detail & Related papers  (2024-05-26T12:25:09Z)
• Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction [55.57072563835959]
  We propose an eigenvalue correction strategy that can free filters from the constraints of repeated eigenvalue inputs.<n>Concretely, the proposed eigenvalue correction strategy enhances the uniform distribution of eigenvalues, and improves the fitting capacity and expressive power of filters.
  arXiv  Detail & Related papers  (2024-01-28T08:12:00Z)
• Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains [13.56018270837999]
  We propose a simple method to extend neural operators to arbitrary domains. An efficient implementation* of such direct spectral evaluations is coupled with existing neural operator models. We demonstrate that the proposed method allows us to extend neural operators to arbitrary point distributions with significant gains in training speed over baselines.
  arXiv  Detail & Related papers  (2023-05-31T09:01:20Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• Deep Neural Networks with Efficient Guaranteed Invariances [77.99182201815763]
  We address the problem of improving the performance and in particular the sample complexity of deep neural networks. Group-equivariant convolutions are a popular approach to obtain equivariant representations. We propose a multi-stream architecture, where each stream is invariant to a different transformation.
  arXiv  Detail & Related papers  (2023-03-02T20:44:45Z)
• Convexifying Transformers: Improving optimization and understanding of transformer networks [56.69983975369641]
  We study the training problem of attention/transformer networks and introduce a novel convex analytic approach. We first introduce a convex alternative to the self-attention mechanism and reformulate the regularized training problem of transformer networks. As a byproduct of our convex analysis, we reveal an implicit regularization mechanism, which promotes sparsity across tokens.
  arXiv  Detail & Related papers  (2022-11-20T18:17:47Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders [1.2234742322758418]
  characterization of the spectral behavior of generative models based on neural networks remains an open issue. Recent research has focused heavily on generative adversarial networks and the high-frequency discrepancies between real and generated images. We propose a simple 2D Fourier transform-based spectral regularization loss for the Variational Autoencoders (VAEs)
  arXiv  Detail & Related papers  (2022-01-19T11:49:57Z)
• Reconstructing spectral functions via automatic differentiation [30.015034534260664]
  Reconstructing spectral functions from Euclidean Green's functions is an important inverse problem in many-body physics. We propose an automatic differentiation(AD) framework as a generic tool for the spectral reconstruction from propagator observable.
  arXiv  Detail & Related papers  (2021-11-29T18:09:49Z)
• Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior [54.629850694790036]
  spectral-normalized identity priors (SNIP) is a structured pruning approach that penalizes an entire residual module in a Transformer model toward an identity mapping. We conduct experiments with BERT on 5 GLUE benchmark tasks to demonstrate that SNIP achieves effective pruning results while maintaining comparable performance.
  arXiv  Detail & Related papers  (2020-10-05T05:40:56Z)
• Machine learning in spectral domain [4.724825031148412]
  tuning the eigenvalues correspond in fact to performing a global training of the neural network. spectral learning bound to the eigenvalues could be also employed for pre-training of deep neural networks.
  arXiv  Detail & Related papers  (2020-05-29T07:55:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.