Selective Synchronization Attention

• URL: http://arxiv.org/abs/2602.14445v1
• Date: Mon, 16 Feb 2026 03:58:12 GMT
• Title: Selective Synchronization Attention
• Abstract summary: Transformer architecture has become the foundation of modern deep learning, but its core self-attention mechanism lacks grounding in biological neural computation.<n>We propose Selective Synchronization Attention (SSA), a novel attention mechanism that replaces the standard dotproduct self-attention with a closed-form operator.<n>SSA provides three key advantages: (i) natural sparsity from the phase-locking threshold, whereby tokens with frequencies automatically receive zero attention weight without explicit masking; (ii) positional-semantic encoding through the natural frequency spectrum, eliminating the need for separate positional encodings; and (iii) a single-pass, closed-
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Transformer architecture has become the foundation of modern deep learning, yet its core self-attention mechanism suffers from quadratic computational complexity and lacks grounding in biological neural computation. We propose Selective Synchronization Attention (SSA), a novel attention mechanism that replaces the standard dot-product self-attention with a closed-form operator derived from the steady-state solution of the Kuramoto model of coupled oscillators. In SSA, each token is represented as an oscillator characterized by a learnable natural frequency and phase; the synchronization strength between token pairs, determined by a frequency-dependent coupling and phase-locking condition, serves as the attention weight. This formulation provides three key advantages: (i) natural sparsity arising from the phase-locking threshold, whereby tokens with incompatible frequencies automatically receive zero attention weight without explicit masking; (ii) unified positional-semantic encoding through the natural frequency spectrum, eliminating the need for separate positional encodings; and (iii) a single-pass, closed-form computation that avoids iterative ODE integration, with all components (coupling, order parameter, synchronization) derived from the oscillatory framework. We instantiate SSA within the Oscillatory Synchronization Network (OSN), a drop-in replacement for the Transformer block. Analysis of the synchronization matrices reveals non-uniform, head-diverse coupling patterns even at initialization, demonstrating a stronger architectural inductive bias than the approximately uniform attention produced by randomly initialized Transformers.

Related papers

• Second harmonic phase locking and synchronization blockade in quadratically coupled driven quantum van der Pol oscillators [0.0]
  We investigate the dynamics of a quadratically coupled system under the influence of an external drive applied to the second oscillator.<n>Our analysis reveals a synchronization blockade, characterized by the complete suppression of conventional 1:1 phase-locking with the drive.<n>A classical mean-field analysis of the corresponding equations of motion reproduces this asymmetric phase-locking geometry.
  arXiv  Detail & Related papers  (2026-08-29T10:36:42Z)
• Agnostic Dynamical Decoupling for Single-Qubit Gates [0.0]
  We introduce a method for designing smooth single-qubit control pulses that implement a desired gate.<n>Unlike dynamically corrected gate constructions that require prior knowledge of the noise model, the present approach is agnostic to the detailed form of the target-bath interaction.
  arXiv  Detail & Related papers  (2026-03-13T18:00:09Z)
• Hebbian-Oscillatory Co-Learning [0.0]
  Hebbian-Oscillatory Co-Learning (HOC-L) is a unified two-timescale dynamical framework for joint structural plasticity and phase synchronization.<n>We introduce HOC-L in the hyperbolic sparse geometry of Resonant Sparse Geometry Networks (RSGN), which employs Poincaré ball embeddings with Hebbian-driven dynamic sparsity.<n>We prove convergence of the joint system to a stable equilibrium via a composite Lyapunov function and derive explicit timescale separation bounds.
  arXiv  Detail & Related papers  (2026-02-21T06:25:51Z)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking [49.1352577985191]
  We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task.<n>Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics.
  arXiv  Detail & Related papers  (2026-02-18T20:25:13Z)
• SKANet: A Cognitive Dual-Stream Framework with Adaptive Modality Fusion for Robust Compound GNSS Interference Classification [47.20483076887704]
  Global Navigation Satellite Systems (GNSS) face growing threats from sophisticated jamming interference.<n>We propose a cognitive deep learning framework built upon a dual-stream architecture that integrates Time-Frequency Images (TFIs) and Power Spectral Density (PSD)<n>We show that SKANet achieves an overall accuracy of 96.99%, exhibiting superior robustness for compound jamming classification.
  arXiv  Detail & Related papers  (2026-01-19T07:42:45Z)
• STORE: Semantic Tokenization, Orthogonal Rotation and Efficient Attention for Scaling Up Ranking Models [11.965535230928372]
  Store is a unified and scalable token-based ranking framework built upon three core innovations.<n>Our framework consistently improves prediction accuracy(online CTR by 2.71%, AUC by 1.195%) and training effeciency (1.84 throughput)
  arXiv  Detail & Related papers  (2025-11-24T06:20:02Z)
• Stochastic Clock Attention for Aligning Continuous and Ordered Sequences [1.2418532541734193]
  We formulate an attention mechanism for continuous and ordered sequences that functions as an alignment model.<n>In a Transformer text-to-speech testbed, this construction produces more stable alignments and improved robustness to global time-scaling.
  arXiv  Detail & Related papers  (2025-09-18T07:18:34Z)
• Quantum synchronization in one-dimensional topological systems [3.5253782180203146]
  Synchronization in quantum systems can be induced through applying dissipation on the central sites.<n>Two types of synchronization, stemming from the topological edge states, are characterized by the off-diagonal or diagonal correlations between the boundary sites.<n>Remarkably, the synchronous oscillations maintain steady amplitude and frequency in the thermodynamic limit.
  arXiv  Detail & Related papers  (2024-09-19T09:03:09Z)
• Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery [62.43562856605473]
  We argue for the computational advantages of a recurrent architecture with complex-valued weights. We propose a fully convolutional autoencoder, SynCx, that performs iterative constraint satisfaction.
  arXiv  Detail & Related papers  (2024-05-27T15:47:03Z)
• Learning Iterative Robust Transformation Synchronization [71.73273007900717]
  We propose to use graph neural networks (GNNs) to learn transformation synchronization. In this work, we avoid handcrafting robust loss functions, and propose to use graph neural networks (GNNs) to learn transformation synchronization.
  arXiv  Detail & Related papers  (2021-11-01T07:03:14Z)
• Reservoir Computing using High Order Synchronization of Coupled Oscillators [0.0]
  We propose a concept for reservoir computing on oscillators using the high-order synchronization effect. The proposed computing concept can be implemented on oscillators of diverse nature.
  arXiv  Detail & Related papers  (2020-04-08T16:47:25Z)
• Switching dynamics of single and coupled VO2-based oscillators as elements of neural networks [55.41644538483948]
  We report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in neural networks. For the resistive coupling, it is shown that synchronization takes place at a certain value of the coupling resistance, though it is unstable and a synchronization failure occurs periodically. For the capacitive coupling, two synchronization modes, with weak and strong coupling, are found. The transition between these modes is accompanied by chaotic oscillations.
  arXiv  Detail & Related papers  (2020-01-07T02:16:04Z)
• Thermal coupling and effect of subharmonic synchronization in a system of two VO2 based oscillators [55.41644538483948]
  We explore a prototype of an oscillatory neural network (ONN) based on vanadium dioxide switching devices. The effective action radius RTC of coupling depends both on the total energy released during switching and on the average power. In the case of a strong thermal coupling, the limit of the supply current parameters, for which the oscillations exist, expands by 10 %. The effect of subharmonic synchronization hold promise for application in classification and pattern recognition.
  arXiv  Detail & Related papers  (2020-01-06T03:26:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.