Hands-on Evaluation of Visual Transformers for Object Recognition and Detection

• URL: http://arxiv.org/abs/2512.09579v1
• Date: Wed, 10 Dec 2025 12:15:48 GMT
• Title: Hands-on Evaluation of Visual Transformers for Object Recognition and Detection
• Authors: Dimitrios N. Vlachogiannis, Dimitrios A. Koutsomitropoulos,
• Abstract summary: We compare different types of Vision Transformers (ViTs) against traditional CNN models across various tasks.<n>We find that hybrid and hierarchical transformers, especially Swin and CvT, offer a strong balance between accuracy and computational resources.<n>Overall, our results indicate that Vision Transformers are competitive and, in many cases, outperform traditional CNNs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional Neural Networks (CNNs) for computer vision sometimes struggle with understanding images in a global context, as they mainly focus on local patterns. On the other hand, Vision Transformers (ViTs), inspired by models originally created for language processing, use self-attention mechanisms, which allow them to understand relationships across the entire image. In this paper, we compare different types of ViTs (pure, hierarchical, and hybrid) against traditional CNN models across various tasks, including object recognition, detection, and medical image classification. We conduct thorough tests on standard datasets like ImageNet for image classification and COCO for object detection. Additionally, we apply these models to medical imaging using the ChestX-ray14 dataset. We find that hybrid and hierarchical transformers, especially Swin and CvT, offer a strong balance between accuracy and computational resources. Furthermore, by experimenting with data augmentation techniques on medical images, we discover significant performance improvements, particularly with the Swin Transformer model. Overall, our results indicate that Vision Transformers are competitive and, in many cases, outperform traditional CNNs, especially in scenarios requiring the understanding of global visual contexts like medical imaging.

Related papers

• DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformer [1.456352735394398]
  We propose a novel hierarchical transformer model that adeptly integrates the feature extraction capabilities of Convolutional Neural Networks (CNNs) with the advanced representational potential of Vision Transformers (ViTs)<n> Addressing the lack of inductive biases and dependence on extensive training datasets in ViTs, our model employs a CNN backbone to generate hierarchical visual representations.<n>These representations are adapted for transformer input through an innovative patch tokenization process, preserving the inherited multi-scale inductive biases.
  arXiv  Detail & Related papers  (2025-06-15T22:42:57Z)
• Scaling Federated Learning Solutions with Kubernetes for Synthesizing Histopathology Images [1.232097230344824]
  In the histological domain, tissue images are expensive to obtain and constitute sensitive medical information.<n> Vision Transformers are state-of-the-art computer vision models that have proven helpful in many tasks, including image classification.
  arXiv  Detail & Related papers  (2025-04-05T10:32:56Z)
• Sensitive Image Classification by Vision Transformers [1.9598097298813262]
  Vision transformer models leverage a self-attention mechanism to capture global interactions among contextual local elements.<n>In our study, we conducted a comparative analysis between various popular vision transformer models and traditional pre-trained ResNet models.<n>The findings demonstrated that vision transformer networks surpassed the benchmark pre-trained models, showcasing their superior classification and detection capabilities.
  arXiv  Detail & Related papers  (2024-12-21T02:34:24Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• ViTs are Everywhere: A Comprehensive Study Showcasing Vision Transformers in Different Domain [0.0]
  Vision Transformers (ViTs) are becoming more popular and dominant solutions for many vision problems. ViTs can overcome several possible difficulties with convolutional neural networks (CNNs)
  arXiv  Detail & Related papers  (2023-10-09T12:31:30Z)
• Vision Transformers in Medical Imaging: A Review [0.0]
  Transformer, a model comprising attention-based encoder-decoder architecture, have gained prevalence in the field of natural language processing (NLP) In this paper, we attempt to provide a comprehensive and recent review on the application of transformers in medical imaging by; describing the transformer model comparing it with a diversity of convolutional neural networks (CNNs)
  arXiv  Detail & Related papers  (2022-11-18T05:52:37Z)
• XCiT: Cross-Covariance Image Transformers [73.33400159139708]
  We propose a "transposed" version of self-attention that operates across feature channels rather than tokens. The resulting cross-covariance attention (XCA) has linear complexity in the number of tokens, and allows efficient processing of high-resolution images.
  arXiv  Detail & Related papers  (2021-06-17T17:33:35Z)
• Intriguing Properties of Vision Transformers [114.28522466830374]
  Vision transformers (ViT) have demonstrated impressive performance across various machine vision problems. We systematically study this question via an extensive set of experiments and comparisons with a high-performing convolutional neural network (CNN) We show effective features of ViTs are due to flexible receptive and dynamic fields possible via the self-attention mechanism.
  arXiv  Detail & Related papers  (2021-05-21T17:59:18Z)
• Medical Transformer: Gated Axial-Attention for Medical Image Segmentation [73.98974074534497]
  We study the feasibility of using Transformer-based network architectures for medical image segmentation tasks. We propose a Gated Axial-Attention model which extends the existing architectures by introducing an additional control mechanism in the self-attention module. To train the model effectively on medical images, we propose a Local-Global training strategy (LoGo) which further improves the performance.
  arXiv  Detail & Related papers  (2021-02-21T18:35:14Z)
• Transformers in Vision: A Survey [101.07348618962111]
  Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence. Transformers require minimal inductive biases for their design and are naturally suited as set-functions. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.
  arXiv  Detail & Related papers  (2021-01-04T18:57:24Z)
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale [112.94212299087653]
  Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.
  arXiv  Detail & Related papers  (2020-10-22T17:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.