Efficient Equivariant Transformer for Self-Driving Agent Modeling

• URL: http://arxiv.org/abs/2604.01466v1
• Date: Wed, 01 Apr 2026 23:20:06 GMT
• Title: Efficient Equivariant Transformer for Self-Driving Agent Modeling
• Abstract summary: DriveGATr is a transformer-based architecture for agent modeling.<n>It achieves SE(2)-equivariance without the computational cost of existing methods.<n>DriveGATr is comparable to the state-of-the-art in traffic simulation.
• Score: 23.858855004235128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurately modeling agent behaviors is an important task in self-driving. It is also a task with many symmetries, such as equivariance to the order of agents and objects in the scene or equivariance to arbitrary roto-translations of the entire scene as a whole; i.e., SE(2)-equivariance. The transformer architecture is a ubiquitous tool for modeling these symmetries. While standard self-attention is inherently permutation equivariant, explicit pairwise relative positional encodings have been the standard for introducing SE(2)-equivariance. However, this approach introduces an additional cost that is quadratic in the number of agents, limiting its scalability to larger scenes and batch sizes. In this work, we propose DriveGATr, a novel transformer-based architecture for agent modeling that achieves SE(2)-equivariance without the computational cost of existing methods. Inspired by recent advances in geometric deep learning, DriveGATr encodes scene elements as multivectors in the 2D projective geometric algebra $\mathbb{R}^*_{2,0,1}$ and processes them with a stack of equivariant transformer blocks. Crucially, DriveGATr models geometric relationships using standard attention between multivectors, eliminating the need for costly explicit pairwise relative positional encodings. Experiments on the Waymo Open Motion Dataset demonstrate that DriveGATr is comparable to the state-of-the-art in traffic simulation and establishes a superior Pareto front for performance vs computational cost.

Related papers

• A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$ [8.549379020585462]
  We introduce a family of vision transformers equivariant to arbitrary discrete subgroups of $mathrmO(2)$.<n>Our construction yields equivariant analogues of the core transformer components, together with expressivity guarantees for the resulting layers.
  arXiv  Detail & Related papers  (2026-06-26T09:05:47Z)
• Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't [78.36679389968772]
  We find that padded $textL$ constantprecision transformers are surprisingly robust to expressivity.<n>$logd-looped constant-precision transformers reach $textFO-uniform TCd$, and growing-precision ones reach $textFO-uniform TCd$.
  arXiv  Detail & Related papers  (2026-05-28T19:58:43Z)
• AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training [50.28798360093496]
  We propose to transform complex and diverse solution operators into simpler, better-aligned forms that are easier to model jointly.<n>We instantiate this idea as AOT-POT (adaptive operator-transformation for pre-training operator transformer), which expands hidden representations into multiple parallel streams.<n>AOT-POT achieves state-of-the-art performance on 12 PDE benchmarks with only 3% additional parameters, reducing relative L2 error by up to 77.6% (40.9% on average)
  arXiv  Detail & Related papers  (2026-05-15T09:50:34Z)
• Platonic Transformers: A Solid Choice For Equivariance [25.29042615187704]
  We introduce the Platonic Transformer to resolve this trade-off.<n>By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme.<n>We show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters.
  arXiv  Detail & Related papers  (2025-10-03T20:51:25Z)
• Clebsch-Gordan Transformer: Fast and Global Equivariant Attention [19.720202550140325]
  We propose Clebsch-Gordan Transformer, achieving efficient global attention by a novel Clebsch-Gordon Convolution on $SO(3)$ irreducible representations.<n>Our method enables equivariant modeling of features at all orders while achieving $O(N log N)$ input token complexity.<n>We benchmark our method on a diverse set of benchmarks including n-body simulation, QM9, ModelNet point cloud classification and a robotic grasping dataset.
  arXiv  Detail & Related papers  (2025-09-28T22:09:36Z)
• Linear Memory SE(2) Invariant Attention [8.734687343991364]
  We propose a mechanism for SE(2) invariant scaled dot-product attention that requires linear memory relative to the number of objects in the scene.<n>Our approach is practical to implement and improves performance compared to comparable non-invariant architectures.
  arXiv  Detail & Related papers  (2025-07-24T17:28:57Z)
• seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models [1.474723404975345]
  We propose seq-JEPA, a world modeling framework that introduces architectural biases into joint-embedding predictive architectures.<n>Seq-JEPA simultaneously learns two architecturally segregated representations: one equivariant to specified transformations and another invariant to them.<n>It excels at tasks that inherently require aggregating a sequence of observations, such as path integration across actions and predictive learning across eye movements.
  arXiv  Detail & Related papers  (2025-05-06T04:39:11Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation [59.91357714415056]
  We propose two Transformer variants: Context-Sharing Transformer (CST) and Semantic Gathering-Scattering Transformer (S GST) CST learns the global-shared contextual information within image frames with a lightweight computation; S GST models the semantic correlation separately for the foreground and background. Compared with the baseline that uses vanilla Transformers for multi-stage fusion, ours significantly increase the speed by 13 times and achieves new state-of-the-art ZVOS performance.
  arXiv  Detail & Related papers  (2023-08-13T06:12:00Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.