Accelerating Regularized Attention Kernel Regression for Spectrum Cartography

• URL: http://arxiv.org/abs/2604.25138v1
• Date: Tue, 28 Apr 2026 02:27:40 GMT
• Title: Accelerating Regularized Attention Kernel Regression for Spectrum Cartography
• Abstract summary: This paper proposes a Learning-based Attention Kernel Regression (LAKER) algorithm for accelerating regularized attention kernel regression in spectrum cartography.<n>The key idea is to learn a data-dependent preconditioner that captures the inverse spectral structure of the attention kernel system.<n>Experiments demonstrate that LAKER significantly reduces condition numbers by up to three orders of magnitude, accelerates convergence by over twenty-fold compared to baselines, and maintains high reconstruction accuracy.
• Score: 1.9651461643699661
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spectrum cartography reconstructs spatial radio fields from sparse and heterogeneous wireless measurements, underpinning many sensing and optimization tasks in wireless networks. Attention mechanisms have recently enabled adaptive measurement aggregation via attention kernel-based formulations. However, the resulting exponential kernels exhibit severe spectral imbalance, inducing large condition numbers that render standard iterative solvers ineffective for regularized attention kernel regression. This paper proposes a Learning-based Attention Kernel Regression (LAKER) algorithm for accelerating regularized attention kernel regression in spectrum cartography. The key idea is to learn a data-dependent preconditioner that captures the inverse spectral structure of the attention kernel system, directly reducing the condition number bottleneck. The preconditioner is obtained by solving a regularized maximum-likelihood estimation problem via a shrinkage-regularized convex--concave procedure, and is integrated with a preconditioned conjugate gradient solver for efficient optimization, whose solution is used for radio map reconstruction. Extensive experiments demonstrate that LAKER significantly reduces condition numbers by up to three orders of magnitude, accelerates convergence by over twenty-fold compared to baselines, and maintains high reconstruction accuracy, establishing learning-based preconditioning as an effective approach for attention kernel regression in spectrum cartography.

Related papers

• Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion [82.7007012514703]
  We show that under a denoising objective, linear attention is suboptimal and can only learn an average spectral denoising filter over the training distribution.<n>We introduce Spectral Attention, which directly utilizes the input graph spectrum and provably outperforms linear attention by a margin governed by the spectral diversity of the distribution.<n>For block models, GCA provably matches the idealized Spectral Attention mechanism.
  arXiv  Detail & Related papers  (2026-07-07T17:52:17Z)
• Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation [18.745286331950133]
  Iterative Refinement Neural Operator (IRNO) augments pre-trained operators with a learned refinement module iteratively applied via fixed-point iteration.<n>Across physical systems, IRNO consistently lowers error, with up to 56.05% improvement on turbulent flow.<n>On Active Matter, spectral analysis reveals that, relative to base operator, the normalized error ratios decrease to 27.72-36.10% in low, 5.07-6.68% in mid, and 1.48-2.04% in high-frequencies.
  arXiv  Detail & Related papers  (2026-05-21T19:41:51Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Exact Linear Attention [0.0]
  Exact Linear Attention (ELA) achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions.<n>ELA achieves up to 6x faster decoding speed and 75% reduction in KV cache memory usage compared to full attention.<n>We extend the linear attention principle to vision models, yielding YOLO-LAT, which attains up to 4.3x GPU inference speedup and 7.9x parameter reduction with competitive detection accuracy.
  arXiv  Detail & Related papers  (2026-05-13T08:06:48Z)
• STARK denoises spatial transcriptomics images via adaptive regularization [3.8065244182878613]
  We present an approach to denoising transcriptomics images that is particularly effective for uncovering cell identities in the numerical regime of ultra-low sequencing depths.<n>The method -- Spatial Transcriptomics via Adaptive Regularization and Kernel iterations (STARK) -- shows consistent improvement over the competing methods.
  arXiv  Detail & Related papers  (2025-12-10T06:19:13Z)
• Truncated Kernel Stochastic Gradient Descent with General Losses and Spherical Radial Basis Functions [3.308257914042897]
  We propose a novel kernel gradient unifies descent (SGD) algorithm for large-scale supervised learning with general losses.<n>Compared to traditional kernel SGD, our algorithm improves efficiency and scalability through an innovative regularization strategy.<n>Our framework accommodates a broad class of classical loss functions, including least-squares, Huber, and logistic losses.
  arXiv  Detail & Related papers  (2025-10-05T15:04:03Z)
• Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise [60.92029979853314]
  We investigate the roles of gradient normalization and clipping in ensuring the convergence of Gradient Descent (SGD) under heavy-tailed noise. Our work provides the first theoretical evidence demonstrating the benefits of gradient normalization in SGD under heavy-tailed noise. We introduce an accelerated SGD variant incorporating gradient normalization and clipping, further enhancing convergence rates under heavy-tailed noise.
  arXiv  Detail & Related papers  (2024-10-21T22:40:42Z)
• Variance-Reducing Couplings for Random Features [57.73648780299374]
  Random features (RFs) are a popular technique to scale up kernel methods in machine learning. We find couplings to improve RFs defined on both Euclidean and discrete input spaces. We reach surprising conclusions about the benefits and limitations of variance reduction as a paradigm.
  arXiv  Detail & Related papers  (2024-05-26T12:25:09Z)
• Optimal Algorithms for the Inhomogeneous Spiked Wigner Model [89.1371983413931]
  We derive an approximate message-passing algorithm (AMP) for the inhomogeneous problem. We identify in particular the existence of a statistical-to-computational gap where known algorithms require a signal-to-noise ratio bigger than the information-theoretic threshold to perform better than random.
  arXiv  Detail & Related papers  (2023-02-13T19:57:17Z)
• Overcoming the Spectral Bias of Neural Value Approximation [17.546011419043644]
  Value approximation using deep neural networks is often the primary module that provides learning signals to the rest of the algorithm. Recent works in neural kernel regression suggest the presence of a spectral bias, where fitting high-frequency components of the value function requires exponentially more gradient update steps than the low-frequency ones. We re-examine off-policy reinforcement learning through the lens of kernel regression and propose to overcome such bias via a composite neural kernel.
  arXiv  Detail & Related papers  (2022-06-09T17:59:57Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• On Learning Rates and Schr\"odinger Operators [105.32118775014015]
  We present a general theoretical analysis of the effect of the learning rate. We find that the learning rate tends to zero for a broad non- neural class functions.
  arXiv  Detail & Related papers  (2020-04-15T09:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.