ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation

• URL: http://arxiv.org/abs/2507.09299v1
• Date: Sat, 12 Jul 2025 14:19:04 GMT
• Title: ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation
• Authors: Abdulvahap Mutlu, Şengül Doğan, Türker Tuncer,
• Abstract summary: We introduce ViT-ProtoNet, which integrates a ViT-Small backbone into the Prototypical Network framework.<n>Our results establish ViT-ProtoNet as a powerful, flexible approach for few-shot classification.
• Score: 4.884338536663712
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The remarkable representational power of Vision Transformers (ViTs) remains underutilized in few-shot image classification. In this work, we introduce ViT-ProtoNet, which integrates a ViT-Small backbone into the Prototypical Network framework. By averaging class conditional token embeddings from a handful of support examples, ViT-ProtoNet constructs robust prototypes that generalize to novel categories under 5-shot settings. We conduct an extensive empirical evaluation on four standard benchmarks: Mini-ImageNet, FC100, CUB-200, and CIFAR-FS, including overlapped support variants to assess robustness. Across all splits, ViT-ProtoNet consistently outperforms CNN-based prototypical counterparts, achieving up to a 3.2\% improvement in 5-shot accuracy and demonstrating superior feature separability in latent space. Furthermore, it outperforms or is competitive with transformer-based competitors using a more lightweight backbone. Comprehensive ablations examine the impact of transformer depth, patch size, and fine-tuning strategy. To foster reproducibility, we release code and pretrained weights. Our results establish ViT-ProtoNet as a powerful, flexible approach for few-shot classification and set a new baseline for transformer-based meta-learners.

Related papers

• Slicing Vision Transformer for Flexible Inference [79.35046907288518]
  We propose a general framework, named Scala, to enable a single network to represent multiple smaller ViTs.<n>S Scala achieves an average improvement of 1.6% on ImageNet-1K with fewer parameters.
  arXiv  Detail & Related papers  (2024-12-06T05:31:42Z)
• Applying ViT in Generalized Few-shot Semantic Segmentation [0.0]
  This paper explores the capability of ViT-based models under the generalized few-shot semantic segmentation (GFSS) framework. We conduct experiments with various combinations of backbone models, including ResNets and pretrained Vision Transformer (ViT)-based models. We demonstrate the great potential of large pretrained ViT-based model on GFSS task, and expect further improvement on testing benchmarks.
  arXiv  Detail & Related papers  (2024-08-27T11:04:53Z)
• Semi-supervised Vision Transformers at Scale [93.0621675558895]
  We study semi-supervised learning (SSL) for vision transformers (ViT) We propose a new SSL pipeline, consisting of first un/self-supervised pre-training, followed by supervised fine-tuning, and finally semi-supervised fine-tuning. Our proposed method, dubbed Semi-ViT, achieves comparable or better performance than the CNN counterparts in the semi-supervised classification setting.
  arXiv  Detail & Related papers  (2022-08-11T08:11:54Z)
• Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing [71.06718651013965]
  We present adaptive vision transformers (ViT) for robust cross-domain face antispoofing. We adopt ViT as a backbone to exploit its strength to account for long-range dependencies among pixels. Experiments on several benchmark datasets show that the proposed models achieve both robust and competitive performance.
  arXiv  Detail & Related papers  (2022-03-23T03:37:44Z)
• Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning [50.95116994162883]
  Vision transformers have been thought of as a promising alternative to convolutional neural networks for visual recognition. This paper presents hierarchically cascaded transformers that exploit intrinsic image structures through spectral tokens pooling. HCTransformers surpass the DINO baseline by a large margin of 9.7% 5-way 1-shot accuracy and 9.17% 5-way 5-shot accuracy on miniImageNet.
  arXiv  Detail & Related papers  (2022-03-17T03:49:58Z)
• A Unified Pruning Framework for Vision Transformers [40.7622551128182]
  Vision transformer (ViT) and its variants have achieved promising performances in various computer vision tasks. We propose a unified framework for structural pruning of both ViTs and its variants, namely UP-ViTs. Our method focuses on pruning all ViTs components while maintaining the consistency of the model structure.
  arXiv  Detail & Related papers  (2021-11-30T05:01:02Z)
• Discrete Representations Strengthen Vision Transformer Robustness [43.821734467553554]
  Vision Transformer (ViT) is emerging as the state-of-the-art architecture for image recognition. We present a simple and effective architecture modification to ViT's input layer by adding discrete tokens produced by a vector-quantized encoder. Experimental results demonstrate that adding discrete representation on four architecture variants strengthens ViT robustness by up to 12% across seven ImageNet robustness benchmarks.
  arXiv  Detail & Related papers  (2021-11-20T01:49:56Z)
• Rethinking the Design Principles of Robust Vision Transformer [28.538786330184642]
  Vision Transformers (ViT) have shown that self-attention-based networks surpassed traditional convolution neural networks (CNNs) in most vision tasks. In this paper, we rethink the design principles of ViTs based on the robustness. By combining the robust design components, we propose Robust Vision Transformer (RVT)
  arXiv  Detail & Related papers  (2021-05-17T15:04:15Z)
• Vision Transformers are Robust Learners [65.91359312429147]
  We study the robustness of the Vision Transformer (ViT) against common corruptions and perturbations, distribution shifts, and natural adversarial examples. We present analyses that provide both quantitative and qualitative indications to explain why ViTs are indeed more robust learners.
  arXiv  Detail & Related papers  (2021-05-17T02:39:22Z)
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions [103.03973037619532]
  This work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the recently-proposed Transformer model (e.g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer(PVT) PVT can be not only trained on dense partitions of the image to achieve high output resolution, which is important for dense predictions.
  arXiv  Detail & Related papers  (2021-02-24T08:33:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.