bViT: Investigating Single-Block Recurrence in Vision Transformers for Image Recognition

• URL: http://arxiv.org/abs/2605.10661v1
• Date: Mon, 11 May 2026 14:43:36 GMT
• Title: bViT: Investigating Single-Block Recurrence in Vision Transformers for Image Recognition
• Abstract summary: We introduce bViT, a single-block recurrent ViT in which one transformer block is applied repeatedly to process an image.<n>On ImageNet-1K, a 12-step bViT-B achieves accuracy comparable to standard ViT-B under the same training recipe and computational budget.<n>We observe that recurrent performance improves with representation width, with wider bViTs recovering much more of the performance of standard ViTs than narrow variants.
• Score: 2.1465474237525113
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision Transformers (ViTs) are built by stacking independently parameterized blocks, but it remains unclear how much of this depth requires layer specific transformations and how much can be realized through recurrent computation. We study this question with bViT, a single-block recurrent ViT in which one transformer block is applied repeatedly to process an image. This architecture preserves the iterative structure of a deep ViT while removing layer specific block parameterization, providing a controlled setting for studying recurrence in vision. On ImageNet-1K, a 12-step bViT-B achieves accuracy comparable to standard ViT-B under the same training recipe and computational budget, while using an order of magnitude fewer parameters. We observe that recurrent performance improves with representation width, with wider bViTs recovering much more of the performance of standard ViTs than narrow variants. We interpret this behavior as implicit depth multiplexing, where a shared block expresses multiple step-dependent computations through the evolving hidden state. Beyond ImageNet classification, bViT transfers competitively to downstream tasks and enables parameter-efficient fine-tuning. Mechanistic analyses of activations, attention and step-specific pruning show that the shared block changes its effective behavior across recurrent steps rather than simply repeating the same computation. Our results suggest that a large fraction of ViT depth can be implemented through recurrent reuse, provided that the representation space is sufficiently wide.

Related papers

• Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision [2.1465474237525113]
  Single-block recurrent ViTs (bViT) remove this growth by repeatedly applying one shared block.<n>We find that standard ViTs remain preferable when FLOPs are the primary constraint, whereas recurrent ViTs offer a better accuracy-- parameter trade-off under memory constraints.
  arXiv  Detail & Related papers  (2026-08-05T14:06:50Z)
• High-Fidelity Differential-information Driven Binary Vision Transformer [38.19452875887032]
  Binarization of vision transformers (ViTs) offers a promising approach to addressing the trade-off between high computational/storage demands and the constraints of edge-device deployment.<n>We propose DIDB-ViT, a novel binary ViT that is highly informative while maintaining the original ViT architecture and computational efficiency.
  arXiv  Detail & Related papers  (2025-07-03T00:59:53Z)
• Your ViT is Secretly an Image Segmentation Model [50.71238842539735]
  Vision Transformers (ViTs) have shown remarkable performance and scalability across various computer vision tasks.<n>We show that inductive biases introduced by task-specific components can instead be learned by the ViT itself.<n>We introduce the Mask Transformer (EoMT), which repurposes the plain ViT architecture to conduct image segmentation.
  arXiv  Detail & Related papers  (2025-03-24T19:56:02Z)
• LPViT: Low-Power Semi-structured Pruning for Vision Transformers [43.126752035656196]
  Vision transformers have emerged as a promising alternative to convolutional neural networks for image analysis tasks.<n>One significant drawback of ViTs is their resource-intensive nature, leading to increased memory footprint, complexity, and power consumption.<n>We introduce a new block-structured pruning to address the resource-intensive issue for ViTs, offering a balanced trade-off between accuracy and hardware acceleration.
  arXiv  Detail & Related papers  (2024-07-02T08:58:19Z)
• PriViT: Vision Transformers for Fast Private Inference [55.36478271911595]
  Vision Transformer (ViT) architecture has emerged as the backbone of choice for state-of-the-art deep models for computer vision applications. ViTs are ill-suited for private inference using secure multi-party protocols, due to the large number of non-polynomial operations. We propose PriViT, an algorithm to selectively " Taylorize" nonlinearities in ViTs while maintaining their prediction accuracy.
  arXiv  Detail & Related papers  (2023-10-06T21:45:05Z)
• CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs [89.79139531731637]
  Vision Transformers (ViTs) have emerged as state-of-the-art models for various vision tasks.<n>We propose a joint underlinecompression method for ViTs that achieves a harmonious blend of high underlineaccuracy, fast underlineinference speed, and favorable underlinetransferability to downstream tasks.
  arXiv  Detail & Related papers  (2023-09-27T16:12:07Z)
• Self-slimmed Vision Transformer [52.67243496139175]
  Vision transformers (ViTs) have become the popular structures and outperformed convolutional neural networks (CNNs) on various vision tasks. We propose a generic self-slimmed learning approach for vanilla ViTs, namely SiT. Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs.
  arXiv  Detail & Related papers  (2021-11-24T16:48:57Z)
• Global Vision Transformer Pruning with Hessian-Aware Saliency [93.33895899995224]
  This work challenges the common design philosophy of the Vision Transformer (ViT) model with uniform dimension across all the stacked blocks in a model stage. We derive a novel Hessian-based structural pruning criteria comparable across all layers and structures, with latency-aware regularization for direct latency reduction. Performing iterative pruning on the DeiT-Base model leads to a new architecture family called NViT (Novel ViT), with a novel parameter that utilizes parameters more efficiently.
  arXiv  Detail & Related papers  (2021-10-10T18:04:59Z)
• Scalable Visual Transformers with Hierarchical Pooling [61.05787583247392]
  We propose a Hierarchical Visual Transformer (HVT) which progressively pools visual tokens to shrink the sequence length. It brings a great benefit by scaling dimensions of depth/width/resolution/patch size without introducing extra computational complexity. Our HVT outperforms the competitive baselines on ImageNet and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2021-03-19T03:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.