Spectral Gating Networks

• URL: http://arxiv.org/abs/2602.07679v1
• Date: Sat, 07 Feb 2026 20:00:49 GMT
• Title: Spectral Gating Networks
• Authors: Jusheng Zhang, Yijia Fan, Kaitong Cai, Jing Yang, Yongsen Zheng, Kwok-Yan Lam, Liang Lin, Keze Wang,
• Abstract summary: We introduce Spectral Gating Networks (SGN) to introduce frequency-rich expressivity in feed-forward networks.<n>SGN augments a standard activation pathway with a compact spectral pathway and learnable gates that allow the model to start from a stable base behavior.<n>It consistently improves accuracy-efficiency trade-offs under comparable computational budgets.
• Score: 65.9496901693099
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gating mechanisms are ubiquitous, yet a complementary question in feed-forward networks remains under-explored: how to introduce frequency-rich expressivity without sacrificing stability and scalability? This tension is exposed by spline-based Kolmogorov-Arnold Network (KAN) parameterizations, where grid refinement can induce parameter growth and brittle optimization in high dimensions. To propose a stability-preserving way to inject spectral capacity into existing MLP/FFN layers under fixed parameter and training budgets, we introduce Spectral Gating Networks (SGN), a drop-in spectral reparameterization. SGN augments a standard activation pathway with a compact spectral pathway and learnable gates that allow the model to start from a stable base behavior and progressively allocate capacity to spectral features during training. The spectral pathway is instantiated with trainable Random Fourier Features (learned frequencies and phases), replacing grid-based splines and removing resolution dependence. A hybrid GELU-Fourier formulation further improves optimization robustness while enhancing high-frequency fidelity. Across vision, NLP, audio, and PDE benchmarks, SGN consistently improves accuracy-efficiency trade-offs under comparable computational budgets, achieving 93.15% accuracy on CIFAR-10 and up to 11.7x faster inference than spline-based KAN variants. Code and trained models will be released.

Related papers

• A Unified Matrix-Spectral Framework for Stability and Interpretability in Deep Learning [0.0]
  We develop a unified framework for analyzing stability and interpretability in deep neural networks.<n>We introduce a Global Matrix Stability Index that aggregates spectral information from Jacobians, parameter gradients, Neural Tangent Kernel operators, and loss Hessians into a single stability scale.
  arXiv  Detail & Related papers  (2026-02-01T10:18:37Z)
• Adaptive Benign Overfitting (ABO): Overparameterized RLS for Online Learning in Non-stationary Time-series [0.0]
  ABO is highly accurate (comparable to baseline kernel methods) while achieving speed improvements of between 20 and 40 percent.<n>Results provide a unified view linking adaptive filtering, kernel approximation, and benign overfitting within a stable online learning framework.
  arXiv  Detail & Related papers  (2026-01-29T15:58:01Z)
• Fluid Antenna System-assisted Physical Layer Secret Key Generation [64.92952968689636]
  This paper investigates physical-layer generation (PLKG) in multiant base station systems by leveraging a fluid antenna system (FAS) to dynamically radio environments.<n>We propose an assisted PLKG model that integrates transmit beamforming and port selection under independent and spatially correlated environments.<n>It is shown that the sliding window-based port selection method introduced in this paper achieves higher KGR with fewer chains through dynamic port selection.
  arXiv  Detail & Related papers  (2025-09-19T03:01:29Z)
• Natural Spectral Fusion: p-Exponent Cyclic Scheduling and Early Decision-Boundary Alignment in First-Order Optimization [11.323131201168572]
  We propose Natural Spectral Fusion (NSF): reframing training as controllable spectral coverage and information fusion.<n>NSF has two core principles: treating the balances as a spectral controller that dynamically low- and high-frequency information.<n>We show that cyclic scheduling consistently reduces test error and demonstrates distinct convergence behavior.
  arXiv  Detail & Related papers  (2025-09-05T00:00:00Z)
• SpectrumFM: Redefining Spectrum Cognition via Foundation Modeling [65.65474629224558]
  We propose a spectrum foundation model, termed SpectrumFM, which provides a new paradigm for spectrum cognition.<n>An innovative spectrum encoder that exploits the convolutional neural networks is proposed to effectively capture both fine-grained local signal structures and high-level global dependencies in the spectrum data.<n>Two novel self-supervised learning tasks, namely masked reconstruction and next-slot signal prediction, are developed for pre-training SpectrumFM, enabling the model to learn rich and transferable representations.
  arXiv  Detail & Related papers  (2025-08-02T14:40:50Z)
• SpectrumFM: A Foundation Model for Intelligent Spectrum Management [99.08036558911242]
  Existing intelligent spectrum management methods, typically based on small-scale models, suffer from notable limitations in recognition accuracy, convergence speed, and generalization.<n>This paper proposes a novel spectrum foundation model, termed SpectrumFM, establishing a new paradigm for spectrum management.<n>Experiments demonstrate that SpectrumFM achieves superior performance in terms of accuracy, robustness, adaptability, few-shot learning efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2025-05-02T04:06:39Z)
• Sparse Hyperparametric Itakura-Saito NMF via Bi-Level Optimization [1.5379084885764847]
  We propose a new algorithm called SHINBO, which introduces a bi-level optimization framework to automatically and adaptively tune the row-dependent penalty hyper parameters.<n> Experimental results showed SHINBO ensures precise spectral decomposition and demonstrates superior performance in both synthetic and real-world applications.
  arXiv  Detail & Related papers  (2025-02-24T13:05:01Z)
• Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space [54.13671100638092]
  Holistic Physics Mixer (HPM) is a framework for integrating spectral and physical information in a unified space.<n>We show that HPM consistently outperforms state-of-the-art methods in both accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2024-10-15T08:19:39Z)
• Spectral Adapter: Fine-Tuning in Spectral Space [45.72323731094864]
  We study the enhancement of current PEFT methods by incorporating the spectral information of pretrained weight matrices into the fine-tuning procedure. We show through extensive experiments that the proposed fine-tuning model enables better parameter efficiency and tuning performance as well as benefits multi-adapter fusion.
  arXiv  Detail & Related papers  (2024-05-22T19:36:55Z)
• Reconfigurable Intelligent Surface (RIS)-Assisted Entanglement Distribution in FSO Quantum Networks [62.87033427172205]
  Quantum networks (QNs) relying on free-space optical (FSO) quantum channels can support quantum applications in environments where establishing an optical fiber infrastructure is challenging and costly. A reconfigurable intelligent surface (RIS)-assisted FSO-based QN is proposed as a cost-efficient framework providing a virtual line-of-sight between users for entanglement distribution.
  arXiv  Detail & Related papers  (2024-01-19T17:16:40Z)
• Hybrid Spectral Denoising Transformer with Guided Attention [34.34075175179669]
  We present a Hybrid Spectral Denoising Transformer (HSDT) for hyperspectral image denoising. Our HSDT significantly outperforms the existing state-of-the-art methods while maintaining low computational overhead.
  arXiv  Detail & Related papers  (2023-03-16T02:24:31Z)
• Spectral Tensor Train Parameterization of Deep Learning Layers [136.4761580842396]
  We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. We show the effects of neural network compression in the classification setting and both compression and improved stability training in the generative adversarial training setting.
  arXiv  Detail & Related papers  (2021-03-07T00:15:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.