VPNeXt -- Rethinking Dense Decoding for Plain Vision Transformer

• URL: http://arxiv.org/abs/2502.16654v3
• Date: Sat, 27 Sep 2025 13:53:31 GMT
• Title: VPNeXt -- Rethinking Dense Decoding for Plain Vision Transformer
• Authors: Xikai Tang, Ye Huang, Guangqiang Yin, Lixin Duan,
• Abstract summary: We present VPNeXt, a new and simple model for the Plain Vision Transformer (ViT)<n>Unlike the many related studies that share the same homogeneous paradigms, VPNeXt offers a fresh perspective on dense representation based on ViT.
• Score: 24.91695334834226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present VPNeXt, a new and simple model for the Plain Vision Transformer (ViT). Unlike the many related studies that share the same homogeneous paradigms, VPNeXt offers a fresh perspective on dense representation based on ViT. In more detail, the proposed VPNeXt addressed two concerns about the existing paradigm: (1) Is it necessary to use a complex Transformer Mask Decoder architecture to obtain good representations? (2) Does the Plain ViT really need to depend on the mock pyramid feature for upsampling? For (1), we investigated the potential underlying reasons that contributed to the effectiveness of the Transformer Decoder and introduced the Visual Context Replay (VCR) to achieve similar effects efficiently. For (2), we introduced the ViTUp module. This module fully utilizes the previously overlooked ViT real pyramid feature to achieve better upsampling results compared to the earlier mock pyramid feature. This represents the first instance of such functionality in the field of semantic segmentation for Plain ViT. We performed ablation studies on related modules to verify their effectiveness gradually. We conducted relevant comparative experiments and visualizations to show that VPNeXt achieved state-of-the-art performance with a simple and effective design. Moreover, the proposed VPNeXt significantly exceeded the long-established mIoU wall/barrier of the VOC2012 dataset, setting a new state-of-the-art by a large margin, which also stands as the largest improvement since 2015.

Related papers

• VORTEX: Challenging CNNs at Texture Recognition by using Vision Transformers with Orderless and Randomized Token Encodings [1.6594406786473057]
  Vision Transformers (ViTs) were introduced a few years ago, but little is known about their texture recognition ability. We introduce VORTEX, a novel method that enables the effective use of ViTs for texture analysis. We evaluate VORTEX on nine diverse texture datasets, demonstrating its ability to achieve or surpass SOTA performance.
  arXiv  Detail & Related papers  (2025-03-09T00:36:02Z)
• Denoising Vision Transformers [43.03068202384091]
  We propose a two-stage denoising approach, termed Denoising Vision Transformers (DVT) In the first stage, we separate the clean features from those contaminated by positional artifacts by enforcing cross-view feature consistency with neural fields on a per-image basis. In the second stage, we train a lightweight transformer block to predict clean features from raw ViT outputs, leveraging the derived estimates of the clean features as supervision.
  arXiv  Detail & Related papers  (2024-01-05T18:59:52Z)
• Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers [10.72362704573323]
  We introduce the PlainSeg, a model comprising only three 3$times$3 convolutions in addition to the transformer layers. We also present the PlainSeg-Hier, which allows for the utilization of hierarchical features.
  arXiv  Detail & Related papers  (2023-10-19T14:01:40Z)
• VST++: Efficient and Stronger Visual Saliency Transformer [74.26078624363274]
  We develop an efficient and stronger VST++ model to explore global long-range dependencies. We evaluate our model across various transformer-based backbones on RGB, RGB-D, and RGB-T SOD benchmark datasets.
  arXiv  Detail & Related papers  (2023-10-18T05:44:49Z)
• PriViT: Vision Transformers for Fast Private Inference [55.36478271911595]
  Vision Transformer (ViT) architecture has emerged as the backbone of choice for state-of-the-art deep models for computer vision applications. ViTs are ill-suited for private inference using secure multi-party protocols, due to the large number of non-polynomial operations. We propose PriViT, an algorithm to selectively " Taylorize" nonlinearities in ViTs while maintaining their prediction accuracy.
  arXiv  Detail & Related papers  (2023-10-06T21:45:05Z)
• TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer [188.00681648113223]
  We explore neat yet effective Transformer-based frameworks for visual grounding. TransVG establishes multi-modal correspondences by Transformers and localizes referred regions by directly regressing box coordinates. We upgrade our framework to a purely Transformer-based one by leveraging Vision Transformer (ViT) for vision feature encoding.
  arXiv  Detail & Related papers  (2022-06-14T06:27:38Z)
• PVT v2: Improved Baselines with Pyramid Vision Transformer [112.0139637538858]
  We improve the original Pyramid Vision Transformer (PVT v1) PVT v2 reduces the computational complexity of PVT v1 to linear. It achieves significant improvements on fundamental vision tasks such as classification, detection, and segmentation.
  arXiv  Detail & Related papers  (2021-06-25T17:51:09Z)
• Visual Saliency Transformer [127.33678448761599]
  We develop a novel unified model based on a pure transformer, Visual Saliency Transformer (VST), for both RGB and RGB-D salient object detection (SOD) It takes image patches as inputs and leverages the transformer to propagate global contexts among image patches. Experimental results show that our model outperforms existing state-of-the-art results on both RGB and RGB-D SOD benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-25T08:24:06Z)
• Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding [81.07894629034767]
  This paper presents a new Vision Transformer (ViT) architecture Multi-Scale Vision Longformer. It significantly enhances the ViT of citedosovitskiy 2020image for encoding high-resolution images using two techniques.
  arXiv  Detail & Related papers  (2021-03-29T06:23:20Z)
• TransReID: Transformer-based Object Re-Identification [20.02035310635418]
  Vision Transformer (ViT) is a pure transformer-based model for the object re-identification (ReID) task. With several adaptations, a strong baseline ViT-BoT is constructed with ViT as backbone. We propose a pure-transformer framework dubbed as TransReID, which is the first work to use a pure Transformer for ReID research.
  arXiv  Detail & Related papers  (2021-02-08T17:33:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.