Versor: A Geometric Sequence Architecture

• URL: http://arxiv.org/abs/2602.10195v1
• Date: Tue, 10 Feb 2026 19:00:02 GMT
• Title: Versor: A Geometric Sequence Architecture
• Abstract summary: A novel sequence architecture design is introduced, Versor, which uses Conformal Geometric Algebra (CGA) in place of the traditional fundamental non-linear operations.<n>Versor is validated on chaotic N-body dynamics, topological reasoning, and standard multimodal benchmarks.
• Score: 0.0954904463032233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A novel sequence architecture design is introduced, Versor, which uses Conformal Geometric Algebra (CGA) in place of the traditional fundamental non-linear operations to achieve structural generalization and significant performance improvements on a variety of tasks, while offering improved interpretability and efficiency. By embedding states in the $Cl_{4,1}$ manifold and evolving them via geometric transformations (rotors), Versor natively represents $SE(3)$-equivariant relationships without requiring explicit structural encoding. Versor is validated on chaotic N-body dynamics, topological reasoning, and standard multimodal benchmarks (CIFAR-10, WikiText-103), consistently outperforming Transformers, Graph Networks, and geometric baselines (GATr, EGNN). Key results include: orders of magnitude fewer parameters ($200\times$ vs. Transformers); interpretable attention decomposing into proximity and orientational components; zero-shot scale generalization (99.3% MCC on topology vs. 50.4% for ViT); and $O(L)$ linear complexity via the novel Recursive Rotor Accumulator. In out-of-distribution tests, Versor maintains stable predictions while Transformers fail catastrophically. Custom Clifford kernels achieve up to $78\times$ speedup, providing a scalable foundation for geometrically-aware scientific modeling.

Related papers

• Mesh Graph Neural Network Framework for Accelerating Finite Element Simulation for Arbitrary Geometries [0.126928839393823]
  This work presents a mesh graph network (MGN) for predicting von Mises stress fields in 2D structural components with arbitrary hole geometries.<n>Unlike traditional machine learning approaches that use absolute node coordinates as features, the proposed model builds on existing MGN frameworks.<n>The model achieves $R2 geq 0.97$ on an unseen geometry and unseen load, compared to $R2 approx 0.01$--$0.86$ for conventional models.
  arXiv  Detail & Related papers  (2026-06-06T18:17:08Z)
• Layerwise LQR for Geometry-Aware Optimization of Deep Networks [38.84575015854741]
  We introduce Layerwise LQR, a framework for learning structured inverse preconditioners under a global layerwise optimal-control objective.<n>We derive a scalable relaxation that learns diagonal, (E-)Kronecker-factored, or other structured inverse preconditioners.<n> Experiments on ResNets and Transformers show that LLQR improves optimization dynamics and often translates these gains into improved final test performance.
  arXiv  Detail & Related papers  (2026-05-05T19:16:00Z)
• The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason [4.28787537081191]
  We find that large language models exhibit emphspectral phase transitions in their hidden activation spaces when engaging in reasoning versus factual recall.<n>We establish a comprehensive emphspectral theory of reasoning in transformers, revealing that the geometry of thought is universal in direction, architecture-specific, and predictive of outcome.
  arXiv  Detail & Related papers  (2026-04-03T09:18:57Z)
• Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networks [52.153950303594684]
  We propose a decoupled kinetic paradigm inspired by Alternating Gradient Flow (AGF)<n>AGF accurately captures the network's structural "kinetic utility"<n>We design a hybrid routing framework that decouples AGF-guided offline structural search from online execution via zero-cost physical priors.
  arXiv  Detail & Related papers  (2026-03-12T18:19:21Z)
• Plain Transformers are Surprisingly Powerful Link Predictors [57.01966734467712]
  Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.<n>While Graph Neural Networks (GNNs) are the standard solution, state-of-the-art pipelines often rely on explicit structurals or memory-intensive node embeddings.<n>We present PENCIL, an encoder-only plain Transformer that replaces hand-crafted priors with attention over sampled local subgraphs.
  arXiv  Detail & Related papers  (2026-02-02T02:45:52Z)
• Exploiting Function-Family Structure in Analog Circuit Optimization [22.334314861424943]
  Off-the-shelf Gaussian-process surrogates impose globally smooth, stationary priors that are misaligned with these regime-switching primitives.<n>We demonstrate that pre-trained models encoding these primitives enable reliable optimization without per-circuit engineering.
  arXiv  Detail & Related papers  (2025-11-30T03:21:13Z)
• ZeroSim: Zero-Shot Analog Circuit Evaluation with Unified Transformer Embeddings [36.00558662624576]
  ZeroSim is a transformer-based performance modeling framework designed to achieve robust in-distribution generalization across trained topologies.<n>We show that ZeroSim significantly outperforms baseline models such as multilayer perceptrons, graph neural networks and transformers.<n>When integrated into a reinforcement learning-based parameter optimization pipeline, ZeroSim achieves a remarkable speedup (13x) compared to conventional SPICE simulations.
  arXiv  Detail & Related papers  (2025-11-10T22:08:30Z)
• Constructive Lyapunov Functions via Topology-Preserving Neural Networks [0.0]
  ONN achieves order-optimal performance on convergence rate, edge efficiency, and computational complexity.<n> Empirical validation on 3M-node semantic networks demonstrates 99.75% improvement over baseline methods.<n>ORTSF integration into transformers achieves 14.7% perplexity reduction and 2.3 faster convergence.
  arXiv  Detail & Related papers  (2025-10-10T17:46:52Z)
• CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning [0.0]
  Curvature-Adaptive Transformer (CAT) learns per-token routing across three geometric attention branches through a lightweight, differentiable gating mechanism.<n>On knowledge graph completion benchmarks, CAT achieves approximately 10% improvements in MRR and Hits@10 over fixed-geometry baselines with minimal overhead.
  arXiv  Detail & Related papers  (2025-10-02T03:26:33Z)
• Geometrically Constrained and Token-Based Probabilistic Spatial Transformers [5.437226012505534]
  We revisit Spatial Transformer Networks (STNs) as a canonicalization tool for transformer-based vision pipelines.<n>We propose a probabilistic, component-wise extension that improves robustness.<n> Experiments on challenging moth classification benchmarks demonstrate that our method consistently improves robustness compared to other STNs.
  arXiv  Detail & Related papers  (2025-09-14T11:30:53Z)
• Graded Transformers [0.0]
  We introduce the Graded Transformer framework, a new class of sequence models that embed inductive biases through grading on vector spaces.<n>The framework enables adaptive feature prioritization, overcoming limitations of fixed grades in earlier models.<n>The Graded Transformer provides a mathematically principled approach to hierarchical learning and neuro-symbolic reasoning.
  arXiv  Detail & Related papers  (2025-07-27T02:34:08Z)
• Exact Sequence Interpolation with Transformers [0.0]
  We prove that transformers can exactly interpolate datasets of finite input sequences in $mathbbRd$, $dgeq 2$, with corresponding output sequences of smaller or equal length.<n>Specifically, given $N$ sequences of arbitrary but finite lengths in $mathbbRd$ and output sequences of lengths $m1, dots, mN in mathcalN$, we construct a transformer with $mathcalO(sum_j=1N mj)$ blocks and $
  arXiv  Detail & Related papers  (2025-02-04T12:31:00Z)
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems [57.58801785642868]
  Chain of thought (CoT) is a highly effective method to improve the accuracy of large language models (LLMs) on arithmetics and symbolic reasoning tasks. This work provides a theoretical understanding of the power of CoT for decoder-only transformers through the lens of expressiveness.
  arXiv  Detail & Related papers  (2024-02-20T10:11:03Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Full Stack Optimization of Transformer Inference: a Survey [58.55475772110702]
  Transformer models achieve superior accuracy across a wide range of applications. The amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate. There has been an increased focus on making Transformer models more efficient.
  arXiv  Detail & Related papers  (2023-02-27T18:18:13Z)
• Geometric Clifford Algebra Networks [53.456211342585824]
  We propose Geometric Clifford Algebra Networks (GCANs) for modeling dynamical systems. GCANs are based on symmetry group transformations using geometric (Clifford) algebras.
  arXiv  Detail & Related papers  (2023-02-13T18:48:33Z)
• DepGraph: Towards Any Structural Pruning [68.40343338847664]
  We study general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. We propose a general and fully automatic method, emphDependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a
  arXiv  Detail & Related papers  (2023-01-30T14:02:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.