CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer

• URL: http://arxiv.org/abs/2511.14111v2
• Date: Fri, 21 Nov 2025 20:51:20 GMT
• Title: CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer
• Authors: Srivathsan Sivakumar, Faisal Z. Qureshi,
• Abstract summary: Vision Transformers (ViTs) have demonstrated remarkable performance across a range of computer vision tasks.<n>We propose emph Cascaded-ViT (CViT), a lightweight and compute-efficient vision transformer architecture.<n>CCFFN improves parameter and FLOP efficiency without sacrificing accuracy.
• Score: 1.9336815376402718
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision Transformers (ViTs) have demonstrated remarkable performance across a range of computer vision tasks; however, their high computational, memory, and energy demands hinder deployment on resource-constrained platforms. In this paper, we propose \emph{Cascaded-ViT (CViT)}, a lightweight and compute-efficient vision transformer architecture featuring a novel feedforward network design called \emph{Cascaded-Chunk Feed Forward Network (CCFFN)}. By splitting input features, CCFFN improves parameter and FLOP efficiency without sacrificing accuracy. Experiments on ImageNet-1K show that our \emph{CViT-XL} model achieves 75.5\% Top-1 accuracy while reducing FLOPs by 15\% and energy consumption by 3.3\% compared to EfficientViT-M5. Across various model sizes, the CViT family consistently exhibits the lowest energy consumption, making it suitable for deployment on battery-constrained devices such as mobile phones and drones. Furthermore, when evaluated using a new metric called \emph{Accuracy-Per-FLOP (APF)}, which quantifies compute efficiency relative to accuracy, CViT models consistently achieve top-ranking efficiency. Particularly, CViT-L is 2.2\% more accurate than EfficientViT-M2 while having comparable APF scores.

Related papers

• LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons [1.213604453116022]
  Vision Transformers have been tremendously successful in computer vision tasks.<n>Large computational, memory, and energy demands are a challenge for edge inference on FPGAs.<n>We propose LL-ViT, a novel edge optimized vision transformer design.
  arXiv  Detail & Related papers  (2025-11-02T05:51:48Z)
• TReX- Reusing Vision Transformer's Attention for Efficient Xbar-based Computing [12.583079680322156]
  We propose TReX, an attention-reuse-driven ViT optimization framework. We find that TReX achieves 2.3x (2.19x) EDAP reduction and 1.86x (1.79x) TOPS/mm2 improvement with 1% accuracy drop in case of DeiT-S (LV-ViT-S) ViT models. On NLP tasks such as CoLA, TReX leads to 2% higher non-ideal accuracy compared to baseline at 1.6x lower EDAP.
  arXiv  Detail & Related papers  (2024-08-22T21:51:38Z)
• ParFormer: A Vision Transformer with Parallel Mixer and Sparse Channel Attention Patch Embedding [9.144813021145039]
  This paper introduces ParFormer, a vision transformer that incorporates a Parallel Mixer and a Sparse Channel Attention Patch Embedding (SCAPE) ParFormer improves feature extraction by combining convolutional and attention mechanisms. For edge device deployment, ParFormer-T excels with a throughput of 278.1 images/sec, which is 1.38 $times$ higher than EdgeNeXt-S. The larger variant, ParFormer-L, reaches 83.5% Top-1 accuracy, offering a balanced trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-03-22T07:32:21Z)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers [52.199303258423306]
  We propose a novel density loss that encourages higher activation sparsity in pre-trained models. Our proposed method, textbfDEFT, can consistently reduce activation density by up to textbf44.94% on RoBERTa$_mathrmLarge$ and by textbf53.19% (encoder density) and textbf90.60% (decoder density) on Flan-T5$_mathrmXXL$.
  arXiv  Detail & Related papers  (2024-02-02T21:25:46Z)
• EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention [44.148667664413004]
  We propose a family of high-speed vision transformers named EfficientViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations. To address this, we present a cascaded group attention module feeding attention heads with different splits.
  arXiv  Detail & Related papers  (2023-05-11T17:59:41Z)
• Rethinking Vision Transformers for MobileNet Size and Speed [58.01406896628446]
  We propose a novel supernet with low latency and high parameter efficiency. We also introduce a novel fine-grained joint search strategy for transformer models. This work demonstrate that properly designed and optimized vision transformers can achieve high performance even with MobileNet-level size and speed.
  arXiv  Detail & Related papers  (2022-12-15T18:59:12Z)
• Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer [56.87383229709899]
  We develop an information rectification module (IRM) and a distribution guided distillation scheme for fully quantized vision transformers (Q-ViT) Our method achieves a much better performance than the prior arts.
  arXiv  Detail & Related papers  (2022-10-13T04:00:29Z)
• CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Prediction [16.578899848650675]
  Vision transformer (ViT) has achieved competitive accuracy on a variety of computer vision applications, but its computational cost impedes the deployment on resource-limited mobile devices. We propose a cascade pruning framework named CP-ViT by predicting sparsity in ViT models progressively and dynamically to reduce computational redundancy while minimizing the accuracy loss.
  arXiv  Detail & Related papers  (2022-03-09T08:15:14Z)
• AdaViT: Adaptive Tokens for Efficient Vision Transformer [91.88404546243113]
  We introduce AdaViT, a method that adaptively adjusts the inference cost of vision transformer (ViT) for images of different complexity. AdaViT achieves this by automatically reducing the number of tokens in vision transformers that are processed in the network as inference proceeds.
  arXiv  Detail & Related papers  (2021-12-14T18:56:07Z)
• Global Vision Transformer Pruning with Hessian-Aware Saliency [93.33895899995224]
  This work challenges the common design philosophy of the Vision Transformer (ViT) model with uniform dimension across all the stacked blocks in a model stage. We derive a novel Hessian-based structural pruning criteria comparable across all layers and structures, with latency-aware regularization for direct latency reduction. Performing iterative pruning on the DeiT-Base model leads to a new architecture family called NViT (Novel ViT), with a novel parameter that utilizes parameters more efficiently.
  arXiv  Detail & Related papers  (2021-10-10T18:04:59Z)
• Highly Efficient Salient Object Detection with 100K Parameters [137.74898755102387]
  We propose a flexible convolutional module, namely generalized OctConv (gOctConv), to efficiently utilize both in-stage and cross-stages multi-scale features. We build an extremely light-weighted model, namely CSNet, which achieves comparable performance with about 0.2% (100k) of large models on popular object detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-12T07:00:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.