CoSwin: Convolution Enhanced Hierarchical Shifted Window Attention For Small-Scale Vision

• URL: http://arxiv.org/abs/2509.08959v1
• Date: Wed, 10 Sep 2025 19:43:16 GMT
• Title: CoSwin: Convolution Enhanced Hierarchical Shifted Window Attention For Small-Scale Vision
• Authors: Puskal Khadka, Rodrigue Rizk, Longwei Wang, KC Santosh,
• Abstract summary: CoSwin is a novel feature-fusion architecture that augments the hierarchical shifted window attention with localized convolutional feature learning.<n>We evaluate CoSwin on multiple image classification benchmarks including CIFAR-10, CIFAR-100, MNIST, SVHN, and Tiny ImageNet.
• Score: 2.558238597112103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Transformers (ViTs) have achieved impressive results in computer vision by leveraging self-attention to model long-range dependencies. However, their emphasis on global context often comes at the expense of local feature extraction in small datasets, particularly due to the lack of key inductive biases such as locality and translation equivariance. To mitigate this, we propose CoSwin, a novel feature-fusion architecture that augments the hierarchical shifted window attention with localized convolutional feature learning. Specifically, CoSwin integrates a learnable local feature enhancement module into each attention block, enabling the model to simultaneously capture fine-grained spatial details and global semantic structure. We evaluate CoSwin on multiple image classification benchmarks including CIFAR-10, CIFAR-100, MNIST, SVHN, and Tiny ImageNet. Our experimental results show consistent performance gains over state-of-the-art convolutional and transformer-based models. Notably, CoSwin achieves improvements of 2.17% on CIFAR-10, 4.92% on CIFAR-100, 0.10% on MNIST, 0.26% on SVHN, and 4.47% on Tiny ImageNet over the baseline Swin Transformer. These improvements underscore the effectiveness of local-global feature fusion in enhancing the generalization and robustness of transformers for small-scale vision. Code and pretrained weights available at https://github.com/puskal-khadka/coswin

Related papers

• EVCC: Enhanced Vision Transformer-ConvNeXt-CoAtNet Fusion for Classification [0.5394291557377919]
  Hybrid vision architectures combining Transformers and CNNs have significantly advanced image classification, but they usually do so at significant computational cost.<n>We introduce EVCC, a novel multi-branch architecture integrating the Vision Transformer, lightweight ConvNeXt, and CoAtNet.<n> Experiments across the CIFAR-100, Tobacco3482, CelebA, and Brain Cancer datasets demonstrate EVCC's superiority over powerful models.
  arXiv  Detail & Related papers  (2025-11-24T02:11:19Z)
• Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasets [11.95214938154427]
  Vision Transformer (ViT) captures global information by dividing images into patches. ViT lacks inductive bias during image or video dataset training. We present a lightweight Depth-Wise Convolution module as a shortcut in ViT models.
  arXiv  Detail & Related papers  (2024-07-28T04:23:40Z)
• ACC-ViT : Atrous Convolution's Comeback in Vision Transformers [5.224344210588584]
  We introduce Atrous Attention, a fusion of regional and sparse attention, which can adaptively consolidate both local and global information. We also propose a general vision transformer backbone, named ACC-ViT, following conventional practices for standard vision tasks. ACC-ViT is therefore a strong vision backbone, which is also competitive in mobile-scale versions, ideal for niche applications with small datasets.
  arXiv  Detail & Related papers  (2024-03-07T04:05:16Z)
• Lightweight Vision Transformer with Bidirectional Interaction [59.39874544410419]
  We propose a Fully Adaptive Self-Attention (FASA) mechanism for vision transformer to model the local and global information.<n>Based on FASA, we develop a family of lightweight vision backbones, Fully Adaptive Transformer (FAT) family.
  arXiv  Detail & Related papers  (2023-06-01T06:56:41Z)
• DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition [62.95223898214866]
  We explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field. With a pyramid architecture, we construct a Multi-Scale Dilated Transformer (DilateFormer) by stacking MSDA blocks at low-level stages and global multi-head self-attention blocks at high-level stages. Our experiment results show that our DilateFormer achieves state-of-the-art performance on various vision tasks.
  arXiv  Detail & Related papers  (2023-02-03T14:59:31Z)
• A Close Look at Spatial Modeling: From Attention to Convolution [70.5571582194057]
  Vision Transformers have shown great promise recently for many vision tasks due to the insightful architecture design and attention mechanism. We generalize self-attention formulation to abstract a queryirrelevant global context directly and integrate the global context into convolutions. With less than 14M parameters, our FCViT-S12 outperforms related work ResT-Lite by 3.7% top1 accuracy on ImageNet-1K.
  arXiv  Detail & Related papers  (2022-12-23T19:13:43Z)
• Adaptive Split-Fusion Transformer [90.04885335911729]
  We propose an Adaptive Split-Fusion Transformer (ASF-former) to treat convolutional and attention branches differently with adaptive weights. Experiments on standard benchmarks, such as ImageNet-1K, show that our ASF-former outperforms its CNN, transformer counterparts, and hybrid pilots in terms of accuracy.
  arXiv  Detail & Related papers  (2022-04-26T10:00:28Z)
• EdgeFormer: Improving Light-weight ConvNets by Learning from Vision Transformers [29.09883780571206]
  We propose EdgeFormer, a pure ConvNet based backbone model. We combine the global circular convolution (GCC) with position embeddings, a light-weight convolution op. Experiment results show that the proposed EdgeFormer achieves better performance than popular light-weight ConvNets and vision transformer based models.
  arXiv  Detail & Related papers  (2022-03-08T09:25:17Z)
• UniFormer: Unifying Convolution and Self-attention for Visual Recognition [69.68907941116127]
  Convolution neural networks (CNNs) and vision transformers (ViTs) have been two dominant frameworks in the past few years. We propose a novel Unified transFormer (UniFormer) which seamlessly integrates the merits of convolution and self-attention in a concise transformer format. Our UniFormer achieves 86.3 top-1 accuracy on ImageNet-1K classification.
  arXiv  Detail & Related papers  (2022-01-24T04:39:39Z)
• CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows [99.36226415086243]
  We present CSWin Transformer, an efficient and effective Transformer-based backbone for general-purpose vision tasks. A challenging issue in Transformer design is that global self-attention is very expensive to compute whereas local self-attention often limits the field of interactions of each token.
  arXiv  Detail & Related papers  (2021-07-01T17:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.