Advancing Vision Transformer with Enhanced Spatial Priors

• URL: http://arxiv.org/abs/2604.18549v1
• Date: Mon, 20 Apr 2026 17:41:00 GMT
• Title: Advancing Vision Transformer with Enhanced Spatial Priors
• Abstract summary: Vision Transformer (ViT) has garnered significant attention within the computer vision community.<n>We have proposed RMT, a robust vision backbone with explicit spatial priors for general purposes.<n>Building on the strengths of RMT, Euclidean enhanced Vision Transformer (EVT) is an expanded version that incorporates several key improvements.
• Score: 45.601974887796864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the Vision Transformer (ViT) has garnered significant attention within the computer vision community. However, the core component of ViT, Self-Attention, lacks explicit spatial priors and suffers from quadratic computational complexity, limiting its applicability. To address these issues, we have proposed RMT, a robust vision backbone with explicit spatial priors for general purposes. RMT utilizes Manhattan distance decay to introduce spatial information and employs a horizontal and vertical decomposition attention method to model global information. Building on the strengths of RMT, Euclidean enhanced Vision Transformer (EVT) is an expanded version that incorporates several key improvements. Firstly, EVT uses a more reasonable Euclidean distance decay to enhance the modeling of spatial information, allowing for a more accurate representation of spatial relationships compared to the Manhattan distance used in RMT. Secondly, EVT abandons the decomposed attention mechanism featured in RMT and instead adopts a simpler spatially-independent grouping approach, providing the model with greater flexibility in controlling the number of tokens within each group. By addressing these modifications, EVT offers a more sophisticated and adaptable approach to incorporating spatial priors into the Self-Attention mechanism, thus overcoming some of the limitations associated with RMT and further enhancing its applicability in various computer vision tasks. Extensive experiments on Image Classification, Object Detection, Instance Segmentation, and Semantic Segmentation demonstrate that EVT exhibits exceptional performance. Without additional training data, EVT achieves 86.6% top1-acc on ImageNet-1k.

Related papers

• OmniVaT: Single Domain Generalization for Multimodal Visual-Tactile Learning [66.4730970958238]
  Visual-tactile learning (VTL) enables embodied agents to perceive the physical world by integrating visual (VIS) and tactile (TAC) sensors.<n>We formulate these challenges as a new task, termed single domain generalization for multimodal VTL.<n>We propose an OmniVaT framework that, for the first time, successfully addresses this task.
  arXiv  Detail & Related papers  (2026-01-01T14:11:49Z)
• Integrating ConvNeXt and Vision Transformers for Enhancing Facial Age Estimation [10.995974662579124]
  We present a novel hybrid architecture that combines convolutional neural networks (CNNs) with Vision Transformers (ViT)<n>Our proposed ConvNeXt-ViT hybrid solution was thoroughly evaluated on benchmark age estimation datasets.<n>The results show that the ConvNeXt-ViT hybrid not only outperforms traditional methods, but also provides a robust foundation for future advances in age estimation.
  arXiv  Detail & Related papers  (2025-10-31T09:36:28Z)
• Learning Spatial Decay for Vision Transformers [50.63391799053993]
  Vision Transformers (ViTs) have revolutionized computer vision, yet their self-attention mechanism lacks explicit spatial inductive biases.<n>Existing approaches introduce data-independent spatial decay based on fixed distance metrics.<n>We present the first successful adaptation of data-dependent spatial decay to 2D vision transformers.
  arXiv  Detail & Related papers  (2025-08-13T06:18:32Z)
• DSFormer: A Dual-Scale Cross-Learning Transformer for Visual Place Recognition [16.386674597850778]
  We propose a novel framework that integrates Dual-Scale-Former (DSFormer), a Transformer-based cross-learning module, with an innovative block clustering strategy.<n>Our approach achieves state-of-the-art performance across most benchmark datasets.
  arXiv  Detail & Related papers  (2025-07-24T14:29:30Z)
• Efficient High-Resolution Visual Representation Learning with State Space Model for Human Pose Estimation [60.80423207808076]
  Capturing long-range dependencies while preserving high-resolution visual representations is crucial for dense prediction tasks such as human pose estimation.<n>We propose the Dynamic Visual State Space (DVSS) block, which augments visual state space models with multi-scale convolutional operations.<n>We build HRVMamba, a novel model for efficient high-resolution representation learning.
  arXiv  Detail & Related papers  (2024-10-04T06:19:29Z)
• RMT: Retentive Networks Meet Vision Transformers [55.76528783956601]
  Vision Transformer (ViT) has gained increasing attention in the computer vision community in recent years.<n>Self-Attention lacks explicit spatial priors and bears a quadratic computational complexity.<n>We propose RMT, a strong vision backbone with explicit spatial prior for general purposes.
  arXiv  Detail & Related papers  (2023-09-20T00:57:48Z)
• DAT++: Spatially Dynamic Vision Transformer with Deformable Attention [87.41016963608067]
  We present Deformable Attention Transformer ( DAT++), a vision backbone efficient and effective for visual recognition. DAT++ achieves state-of-the-art results on various visual recognition benchmarks, with 85.9% ImageNet accuracy, 54.5 and 47.0 MS-COCO instance segmentation mAP, and 51.5 ADE20K semantic segmentation mIoU.
  arXiv  Detail & Related papers  (2023-09-04T08:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.