CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction

• URL: http://arxiv.org/abs/2410.11428v1
• Date: Tue, 15 Oct 2024 09:27:26 GMT
• Title: CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction
• Authors: Chunlei Meng, Jiacheng Yang, Wei Lin, Bowen Liu, Hongda Zhang, chun ouyang, Zhongxue Gan,
• Abstract summary: CTA-Net combines CNNs and ViTs, with transformers capturing long-range dependencies and CNNs extracting localized features. This integration enables efficient processing of detailed local and broader contextual information. Experiments on small-scale datasets with fewer than 100,000 samples show that CTA-Net achieves superior performance.
• Score: 14.377544481394013
• License:
• Abstract: Convolutional neural networks (CNNs) and vision transformers (ViTs) have become essential in computer vision for local and global feature extraction. However, aggregating these architectures in existing methods often results in inefficiencies. To address this, the CNN-Transformer Aggregation Network (CTA-Net) was developed. CTA-Net combines CNNs and ViTs, with transformers capturing long-range dependencies and CNNs extracting localized features. This integration enables efficient processing of detailed local and broader contextual information. CTA-Net introduces the Light Weight Multi-Scale Feature Fusion Multi-Head Self-Attention (LMF-MHSA) module for effective multi-scale feature integration with reduced parameters. Additionally, the Reverse Reconstruction CNN-Variants (RRCV) module enhances the embedding of CNNs within the transformer architecture. Extensive experiments on small-scale datasets with fewer than 100,000 samples show that CTA-Net achieves superior performance (TOP-1 Acc 86.76\%), fewer parameters (20.32M), and greater efficiency (FLOPs 2.83B), making it a highly efficient and lightweight solution for visual tasks on small-scale datasets (fewer than 100,000).

Related papers

• Efficient Semantic Segmentation via Lightweight Multiple-Information Interaction Network [37.84039482457571]
  We propose a lightweight multiple-information interaction network for real-time semantic segmentation, called LMIINet. It effectively combines CNNs and Transformers while reducing redundant computations and memory footprint. With only 0.72M parameters and 11.74G FLOPs, LMIINet achieves 72.0% mIoU at 100 FPS on the Cityscapes test set and 69.94% mIoU at 160 FPS on the CamVid dataset.
  arXiv  Detail & Related papers  (2024-10-03T05:45:24Z)
• HAFormer: Unleashing the Power of Hierarchy-Aware Features for Lightweight Semantic Segmentation [11.334990474402915]
  We introduce HAFormer, a model that combines the hierarchical features extraction ability of CNNs with the global dependency modeling capability of Transformers. HAFormer achieves high performance with minimal computational overhead and compact model size.
  arXiv  Detail & Related papers  (2024-07-10T07:53:24Z)
• ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection [65.59969454655996]
  We propose an efficient change detection framework, ELGC-Net, which leverages rich contextual information to precisely estimate change regions. Our proposed ELGC-Net sets a new state-of-the-art performance in remote sensing change detection benchmarks. We also introduce ELGC-Net-LW, a lighter variant with significantly reduced computational complexity, suitable for resource-constrained settings.
  arXiv  Detail & Related papers  (2024-03-26T17:46:25Z)
• OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentation [70.17681136234202]
  We reexamine the design distinctions and test the limits of what a sparse CNN can achieve. We propose two key components, i.e., adaptive receptive fields (spatially) and adaptive relation, to bridge the gap. This exploration led to the creation of Omni-Adaptive 3D CNNs (OA-CNNs), a family of networks that integrates a lightweight module.
  arXiv  Detail & Related papers  (2024-03-21T14:06:38Z)
• Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks [13.815116154370834]
  We introduce a novel framework, the Multiscale Low-Frequency Memory (MLFM) Network. The MLFM efficiently preserves low-frequency information, enhancing performance in targeted computer vision tasks. Our work builds upon the existing CNN foundations and paves the way for future advancements in computer vision.
  arXiv  Detail & Related papers  (2024-03-13T00:48:41Z)
• Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN [34.020978009518245]
  We propose a lightweight real-time semantic segmentation network called LETNet. LETNet combines a U-shaped CNN with Transformer effectively in a capsule embedding style to compensate for respective deficiencies. Experiments performed on challenging datasets demonstrate that LETNet achieves superior performances in accuracy and efficiency balance.
  arXiv  Detail & Related papers  (2023-02-21T07:16:53Z)
• MCTNet: A Multi-Scale CNN-Transformer Network for Change Detection in Optical Remote Sensing Images [7.764449276074902]
  We propose a hybrid network based on multi-scale CNN-transformer structure, termed MCTNet. We show that our MCTNet achieves better detection performance than existing state-of-the-art CD methods.
  arXiv  Detail & Related papers  (2022-10-14T07:54:28Z)
• Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets [91.25055890980084]
  There still remains an extreme performance gap between Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) when training from scratch on small datasets. We propose Dynamic Hybrid Vision Transformer (DHVT) as the solution to enhance the two inductive biases. Our DHVT achieves a series of state-of-the-art performance with a lightweight model, 85.68% on CIFAR-100 with 22.8M parameters, 82.3% on ImageNet-1K with 24.0M parameters.
  arXiv  Detail & Related papers  (2022-10-12T06:54:39Z)
• Vision Transformer with Convolutions Architecture Search [72.70461709267497]
  We propose an architecture search method-Vision Transformer with Convolutions Architecture Search (VTCAS) The high-performance backbone network searched by VTCAS introduces the desirable features of convolutional neural networks into the Transformer architecture. It enhances the robustness of the neural network for object recognition, especially in the low illumination indoor scene.
  arXiv  Detail & Related papers  (2022-03-20T02:59:51Z)
• Rich CNN-Transformer Feature Aggregation Networks for Super-Resolution [50.10987776141901]
  Recent vision transformers along with self-attention have achieved promising results on various computer vision tasks. We introduce an effective hybrid architecture for super-resolution (SR) tasks, which leverages local features from CNNs and long-range dependencies captured by transformers. Our proposed method achieves state-of-the-art SR results on numerous benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-15T06:52:25Z)
• Container: Context Aggregation Network [83.12004501984043]
  Recent finding shows that a simple based solution without any traditional convolutional or Transformer components can produce effective visual representations. We present the model (CONText Ion NERtwok), a general-purpose building block for multi-head context aggregation. In contrast to Transformer-based methods that do not scale well to downstream tasks that rely on larger input image resolutions, our efficient network, named modellight, can be employed in object detection and instance segmentation networks.
  arXiv  Detail & Related papers  (2021-06-02T18:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.