Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers

• URL: http://arxiv.org/abs/2607.00580v1
• Date: Wed, 01 Jul 2026 08:02:46 GMT
• Title: Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers
• Abstract summary: We propose Active Spatial Guidance (Guidance), a training-only objective that disables positional injection.<n>Guidance consistently improves performance on ImageNet-100 classification, ADE20K semantic segmentation, and Hypersim monocular depth estimation.<n>Our results suggest that spatial inductive bias in ViTs need not be architecturally injected, but can be shaped through training-time supervision.
• Score: 14.844683088763118
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Vision Transformers (ViTs) commonly rely on injected positional mechanisms to address self-attention's permutation invariance. Motivated by the spatial regularities of natural images, we ask whether spatial organization can be induced from data rather than explicitly injected. Under controlled, matched from-scratch training, we propose Active Spatial Guidance (Guidance), a training-only objective that disables positional injection and applies an auxiliary 2D coordinate-regression loss to the final-layer patch tokens. The guidance head is used only during training and removed for inference; the deployed model consists of a positional-injection-free ViT encoder and the task-specific prediction module. Using DINOv3 ViT backbones, Guidance consistently improves performance on ImageNet-100 classification, ADE20K semantic segmentation, and Hypersim monocular depth estimation, outperforming strong injected baselines such as learned absolute positional embeddings and rotary positional embeddings under identical training protocols. On ImageNet-100, broader comparisons against representative injected positional designs further support Guidance's effectiveness. Guidance also improves robustness under resolution transfer, and multi-resolution training further strengthens accuracy across input sizes. Overall, our results suggest that spatial inductive bias in ViTs need not be architecturally injected, but can be shaped through training-time supervision. The code used for training and evaluation is publicly available in https://github.com/cloudlc/asg.

Related papers

• Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity [6.827241106564254]
  Vision Transformers demonstrate remarkable global modeling capacity but often underperform in data-scarce regimes.<n>We propose iBKD, a framework that preserves the spatial grid throughout the entire transfer process.<n>iBKD consistently outperforms both locality-guidance methods and general knowledge distillation baselines.
  arXiv  Detail & Related papers  (2026-08-11T09:39:13Z)
• Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization [6.3170928681089995]
  Vision-Language-Action (VLA) models leverage large-scale vision-language pretraining for flexible robot manipulation.<n>They remain brittle along two axes: spatial generalization, when object positions differ from those seen during training, and task generalization, when a familiar scene is paired with a different language instruction.<n>We propose a lightweight, model-agnostic module that represents the grounding signal in 3D, computes its relative displacement to the gripper, and injects the resulting embedding directly into the action head.
  arXiv  Detail & Related papers  (2026-06-26T02:44:27Z)
• AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers [24.937985157569823]
  We argue that such timestep-agnostic alignment is suboptimal because the useful granularity of representation supervision changes systematically with the signal-to-noise ratio.<n>In high-noise regimes, diffusion models benefit more from coarse semantic and layout-level anchoring, whereas in low-noise regimes, the training signal should emphasize spatially detailed and structurally faithful refinement.<n>This non-stationary alignment behavior creates a representational mismatch for static single-level supervisors.
  arXiv  Detail & Related papers  (2026-05-05T03:07:29Z)
• Locality-Attending Vision Transformer [39.7650646299574]
  Global self-attention can obscure fine-grained spatial details crucial for tasks such as segmentation.<n>We present a simple yet effective add-on that improves performance on segmentation tasks while retaining vision transformers' image-level recognition capabilities.
  arXiv  Detail & Related papers  (2026-03-05T07:31:49Z)
• Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization [65.37179698521766]
  Vision-Language-Action (VLA) models show strong promise for generalist robot control.<n>It remains unclear whether -- and under what conditions -- the standard "scale data" recipe translates to robotics.<n>We present a systematic, controlled study of VLA scaling that revisits core training choices for pretraining across diverse robots.
  arXiv  Detail & Related papers  (2026-02-10T12:25:43Z)
• SeqPE: Transformer with Sequential Position Encoding [76.22159277300891]
  SeqPE represents each $n$-dimensional position index as a symbolic sequence and employs a lightweight sequential position encoder to learn their embeddings.<n> Experiments across language modeling, long-context question answering, and 2D image classification demonstrate that SeqPE not only surpasses strong baselines in perplexity, exact match (EM) and accuracy--but also enables seamless generalization to multi-dimensional inputs without requiring manual architectural redesign.
  arXiv  Detail & Related papers  (2025-06-16T09:16:40Z)
• GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction [5.647541727494757]
  GTransPDM was developed for pedestrian crossing intention prediction by leveraging multi-modal features.<n>It achieves 92% accuracy on the PIE dataset and 87% accuracy on the JAAD dataset, with a processing speed of 0.05ms.
  arXiv  Detail & Related papers  (2024-09-30T12:02:17Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes [86.2129580231191]
  Adjoint Rigid Transform (ART) Network is a neural module which can be integrated with a variety of 3D networks. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks. We will release our code and pre-trained models for further research.
  arXiv  Detail & Related papers  (2021-02-01T20:58:45Z)
• Understanding Self-Training for Gradual Domain Adaptation [107.37869221297687]
  We consider gradual domain adaptation, where the goal is to adapt an initial classifier trained on a source domain given only unlabeled data that shifts gradually in distribution towards a target domain. We prove the first non-vacuous upper bound on the error of self-training with gradual shifts, under settings where directly adapting to the target domain can result in unbounded error. The theoretical analysis leads to algorithmic insights, highlighting that regularization and label sharpening are essential even when we have infinite data, and suggesting that self-training works particularly well for shifts with small Wasserstein-infinity distance.
  arXiv  Detail & Related papers  (2020-02-26T08:59:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.