The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

• URL: http://arxiv.org/abs/2603.17433v1
• Date: Wed, 18 Mar 2026 07:18:41 GMT
• Title: The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
• Abstract summary: We introduce the textbfPhasor Transformer block, a phase-native alternative representing sequence states on the unit-circle manifold $S1$.<n> Stacking these blocks defines the textbfLarge Phasor Model (LPM), which learns stable global dynamics and achieves competitive forecasting behavior.<n>Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase with deterministic global coupling.
• Score: 0.09229852843814058
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the \textbf{Phasor Transformer} block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the \textbf{Large Phasor Model (LPM)}. We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks. Operating with a highly compact parameter budget, LPM learns stable global dynamics and achieves competitive forecasting behavior compared to conventional self-attention baselines. Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase computation with deterministic global coupling, offering a practical path toward scalable temporal modeling in oscillatory domains.

Related papers

• Beyond Similarity: Temporal Operator Attention for Time Series Analysis [5.084091872064936]
  A persistent paradox in time-series forecasting is that structurally simple and linear models often outperform high-capacity Transformers.<n>We propose $textbf Operator Attention (TOA)$, a framework that augments attention with explicit learnable sequence-space operators.<n>To make dense $N times operators practical, we introduce a high-variance dropout mechanism that stabilizes training and prevents trivial memorization.
  arXiv  Detail & Related papers  (2026-05-11T22:13:53Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting [28.587343014443576]
  vecTrans is a lightweight module that utilizes a learnable vector to model multivariate correlations.<n>WFMLoss is an effective plug-and-play objective, consistently improving existing forecasters.
  arXiv  Detail & Related papers  (2026-01-20T09:23:10Z)
• Spectral-Window Hybrid (SWH) [0.0]
  Scaling sequence modeling to extreme contexts requires balancing computational efficiency with representational expressivity.<n>We propose the textbfSpectral-Window Hybrid (SWH), an architecture that decouples sequence modeling into two textitparallel streams.<n>We demonstrate that SWH matches the perplexity of standard Transformers on short contexts while enabling efficient linear scaling to extended sequences.
  arXiv  Detail & Related papers  (2026-01-04T00:31:36Z)
• When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning [53.17227599983122]
  Functional tensor decomposition can analyze multi-dimensional data with real-valued indices.<n>We propose a rank-revealing functional low-rank tensor completion (RR-F) method.<n>We establish the universal approximation property of the model for continuous multi-dimensional signals.
  arXiv  Detail & Related papers  (2025-12-25T03:15:52Z)
• Less Is More: Generating Time Series with LLaMA-Style Autoregression in Simple Factorized Latent Spaces [29.349850914912846]
  We propose FAR-TS, a simple yet effective framework that combines disentangled factorization with an autoregressive Transformer over a discrete, quantized latent space to generate time series.<n>A LLaMA-style autoregressive Transformer then models these token sequences, enabling fast and controllable generation of sequences with arbitrary length.
  arXiv  Detail & Related papers  (2025-11-07T04:15:38Z)
• FFT-Accelerated Auxiliary Variable MCMC for Fermionic Lattice Models: A Determinant-Free Approach with $O(N\log N)$ Complexity [52.3171766248012]
  We introduce a Markov Chain Monte Carlo (MCMC) algorithm that dramatically accelerates the simulation of quantum many-body systems.<n>We validate our algorithm on benchmark quantum physics problems, accurately reproducing known theoretical results.<n>Our work provides a powerful tool for large-scale probabilistic inference and opens avenues for physics-inspired generative models.
  arXiv  Detail & Related papers  (2025-10-13T07:57:21Z)
• Is Attention Required for Transformer Inference? Explore Function-preserving Attention Replacement [13.38679135071682]
  We propose a Function-preserving Attention Replacement framework that replaces all attention blocks in pretrained transformers with learnable sequence-to-sequence modules.<n>We validate FAR on the DeiT vision transformer family and demonstrate that it matches the accuracy of the original models on ImageNet and multiple downstream tasks with reduced parameters and latency.
  arXiv  Detail & Related papers  (2025-05-24T02:23:46Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a causal Transformer for unified time series forecasting.<n>Based on large-scale pre-training, Timer-XL achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• SCHEME: Scalable Channel Mixer for Vision Transformers [52.605868919281086]
  Vision Transformers have achieved impressive performance in many computation tasks.<n>We show that the dense connections can be replaced with a sparse block diagonal structure that supports larger expansion ratios.<n>We also propose the use of a lightweight, parameter-free, channel covariance attention mechanism as a parallel branch during training.
  arXiv  Detail & Related papers  (2023-12-01T08:22:34Z)
• Convolutional State Space Models for Long-Range Spatiotemporal Modeling [65.0993000439043]
  ConvS5 is an efficient variant for long-rangetemporal modeling. It significantly outperforms Transformers and ConvNISTTM on a long horizon Moving-Lab experiment while training 3X faster than ConvLSTM and generating samples 400X faster than Transformers.
  arXiv  Detail & Related papers  (2023-10-30T16:11:06Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.