Controlled Dynamics Attractor Transformer

• URL: http://arxiv.org/abs/2606.15207v1
• Date: Sat, 13 Jun 2026 09:04:22 GMT
• Title: Controlled Dynamics Attractor Transformer
• Abstract summary: Transformer architectures have advanced representation learning and inference in deep models through self-attention mechanisms.<n>In parallel,associative memory (AM) frameworks map representations onto energy landscapes, offering interpretable retrieval mechanisms.<n>We propose Controlled Dynamics Attractor Transformer (CCAT), which couples a mixture von Mises-Fisher (Mo-vMF) attention energy with a Hopfield refinement energy.<n>CCAT achieves state-of-the-art performance across multiple benchmarks in graph anomaly detection and graph classification.
• Score: 58.15454173718687
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Transformer architectures have dramatically advanced representation learning and inference in deep models through self-attention mechanisms. In parallel,associative memory (AM) frameworks map representations onto energy landscapes, offering interpretable retrieval mechanisms. However, their continuous-time inference dynamics lack the biological plausibility of classical Continuous Attractor Neural Networks (CANNs). To bridge this gap, we propose Controlled Dynamics Attractor Transformer (CDAT), which couples a mixture von Mises-Fisher (Mo-vMF) attention energy with a Hopfield refinement energy, while augmenting energy descent with a CANN-inspired excitation-inhibition modulation. CDAT instantiates a topology-constrained dynamical system whose couplings encode relational structure among tokens, thereby linking attractor-style dynamics to modern energy-based attention. We further provide a constructive dissipation analysis to formally establish their controlled inference dynamics. Benefiting from these robust and structured dynamics, CDAT achieves state-of-the-art performance across multiple benchmarks in graph anomaly detection and graph classification.

Related papers

• Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models [59.8742208800447]
  We propose DyGFM, a Dynamic Graph Foundation Model over multiple domains based on decoupled and divergence-conditioned prompting.<n>We show that DyGFM consistently outperforms 12 state-of-the-art baselines on both node classification and link prediction tasks.
  arXiv  Detail & Related papers  (2026-05-13T13:50:03Z)
• Emergence Transformer: Dynamical Temporal Attention Matters [8.268321002607737]
  The Transformer, a breakthrough architecture in artificial intelligence, owes its success to the attention mechanism.<n>We propose an Emergence Transformer, allowing each component to interact with its own or its neighbors' past states.<n>We show how DTA reshapes social coherence, suggesting strategies to either enhance agreement or preserve plurality.
  arXiv  Detail & Related papers  (2026-04-18T01:10:44Z)
• Hebbian Attractor Networks for Robot Locomotion [44.84033948079516]
  Hebbian plasticity offers a promising path toward rapid adaptation in changing environments.<n>Hebbian Attractor Networks (HANs) induce emergent attractor dynamics.<n>Results show that slower updates, combined with averaged pre- and postsynaptic activations, promote convergence to stable weight configurations.
  arXiv  Detail & Related papers  (2026-03-23T19:20:14Z)
• Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systems [38.4555621948915]
  Prismatic World Model (PRISM-WM) is designed to decompose complex hybrid dynamics into composable primitives.<n>PRISM-WM significantly reduces rollout drift by accurately modeling sharp mode transitions in system dynamics.
  arXiv  Detail & Related papers  (2025-12-09T09:40:34Z)
• Tensor Network Framework for Forecasting Nonlinear and Chaotic Dynamics [1.790605517028706]
  We present a tensor network model (TNM) for forecasting nonlinear and chaotic dynamics.<n>We show that the TNM accurately reconstructs short-term trajectories and faithfully captures the attractor geometry.
  arXiv  Detail & Related papers  (2025-11-12T11:49:38Z)
• Kuramoto Orientation Diffusion Models [67.0711709825854]
  Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular patterns.<n>Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model.<n>We implement competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures.
  arXiv  Detail & Related papers  (2025-09-18T18:18:49Z)
• PowerGrow: Feasible Co-Growth of Structures and Dynamics for Power Grid Synthesis [75.14189839277928]
  We present PowerGrow, a co-generative framework that significantly reduces computational overhead while maintaining operational validity.<n> Experiments across benchmark settings show that PowerGrow outperforms prior diffusion models in fidelity and diversity.<n>This demonstrates its ability to generate operationally valid and realistic power grid scenarios.
  arXiv  Detail & Related papers  (2025-08-29T01:47:27Z)
• Transformers from Diffusion: A Unified Framework for Neural Message Passing [79.9193447649011]
  Message passing neural networks (MPNNs) have become a de facto class of model solutions.<n>We propose an energy-constrained diffusion model, which integrates the inductive bias of diffusion with layer-wise constraints of energy.<n>Building on these insights, we devise a new class of message passing models, dubbed Transformers (DIFFormer), whose global attention layers are derived from the principled energy-constrained diffusion framework.
  arXiv  Detail & Related papers  (2024-09-13T17:54:41Z)
• Dynamical Mean-Field Theory of Self-Attention Neural Networks [0.0]
  Transformer-based models have demonstrated exceptional performance across diverse domains. Little is known about how they operate or what are their expected dynamics. We use methods for the study of asymmetric Hopfield networks in nonequilibrium regimes.
  arXiv  Detail & Related papers  (2024-06-11T13:29:34Z)
• SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases [66.61789780666727]
  We show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. We also offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states. Our model yields a significant improvement over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-08-25T07:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.