UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition

• URL: http://arxiv.org/abs/2311.15599v2
• Date: Mon, 18 Mar 2024 08:37:24 GMT
• Title: UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition
• Authors: Xiaohan Ding, Yiyuan Zhang, Yixiao Ge, Sijie Zhao, Lin Song, Xiangyu Yue, Ying Shan,
• Abstract summary: We propose four architectural guidelines for designing large- Kernel-based convolutional neural networks (ConvNets) Our proposed large- Kernel-based ConvNet shows leading performance in image recognition. We discover large kernels are the key to unlocking the exceptional performance of ConvNets in domains where they were originally not proficient.
• Score: 61.01408259741114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-kernel convolutional neural networks (ConvNets) have recently received extensive research attention, but two unresolved and critical issues demand further investigation. 1) The architectures of existing large-kernel ConvNets largely follow the design principles of conventional ConvNets or transformers, while the architectural design for large-kernel ConvNets remains under-addressed. 2) As transformers have dominated multiple modalities, it remains to be investigated whether ConvNets also have a strong universal perception ability in domains beyond vision. In this paper, we contribute from two aspects. 1) We propose four architectural guidelines for designing large-kernel ConvNets, the core of which is to exploit the essential characteristics of large kernels that distinguish them from small kernels - they can see wide without going deep. Following such guidelines, our proposed large-kernel ConvNet shows leading performance in image recognition (ImageNet accuracy of 88.0%, ADE20K mIoU of 55.6%, and COCO box AP of 56.4%), demonstrating better performance and higher speed than the recent powerful competitors. 2) We discover large kernels are the key to unlocking the exceptional performance of ConvNets in domains where they were originally not proficient. With certain modality-related preprocessing approaches, the proposed model achieves state-of-the-art performance on time-series forecasting and audio recognition tasks even without modality-specific customization to the architecture. All the code and models are publicly available on GitHub and Huggingface.

Related papers

• Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations [17.41381592056492]
  This paper proposes the paradigm of large convolutional kernels in designing modern Convolutional Neural Networks (ConvNets) We establish that employing a few large kernels, instead of stacking multiple smaller ones, can be a superior design strategy. We propose the UniRepLKNet architecture, which offers systematical architecture design principles specifically crafted for large- Kernel ConvNets.
  arXiv  Detail & Related papers  (2024-10-10T15:43:55Z)
• Designing Concise ConvNets with Columnar Stages [33.248031676529635]
  We introduce a refreshing ConvNet macro design called Columnar Stage Network (CoSNet) CoSNet has a systematically developed simple and concise structure, smaller depth, low parameter count, low FLOPs, and attention-less operations. Our evaluations show that CoSNet rivals many renowned ConvNets and Transformer designs under resource-constrained scenarios.
  arXiv  Detail & Related papers  (2024-10-05T09:03:42Z)
• Are Large Kernels Better Teachers than Transformers for ConvNets? [82.4742785108714]
  This paper reveals a new appeal of the recently emerged large-kernel Convolutional Neural Networks (ConvNets): as the teacher in Knowledge Distillation (KD) for small- Kernel ConvNets.
  arXiv  Detail & Related papers  (2023-05-30T21:05:23Z)
• InceptionNeXt: When Inception Meets ConvNeXt [167.61042926444105]
  We build a series of networks, namely IncepitonNeXt, which not only enjoy high throughputs but also maintain competitive performance. InceptionNeXt achieves 1.6x higher training throughputs than ConvNeX-T, as well as attains 0.2% top-1 accuracy improvement on ImageNet-1K.
  arXiv  Detail & Related papers  (2023-03-29T17:59:58Z)
• ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders [104.05133094625137]
  We propose a fully convolutional masked autoencoder framework and a new Global Response Normalization layer. This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets.
  arXiv  Detail & Related papers  (2023-01-02T18:59:31Z)
• MogaNet: Multi-order Gated Aggregation Network [64.16774341908365]
  We propose a new family of modern ConvNets, dubbed MogaNet, for discriminative visual representation learning. MogaNet encapsulates conceptually simple yet effective convolutions and gated aggregation into a compact module. MogaNet exhibits great scalability, impressive efficiency of parameters, and competitive performance compared to state-of-the-art ViTs and ConvNets on ImageNet.
  arXiv  Detail & Related papers  (2022-11-07T04:31:17Z)
• Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs [35.39701561076837]
  We propose a new basic neural network operator named position-aware circular convolution (ParC) and its accelerated version Fast-ParC. Our Fast-ParC further reduces the O(n2) time complexity of ParC to O(n log n) using Fast Fourier Transform. Experiment results show that our ParC op can effectively enlarge the receptive field of traditional ConvNets.
  arXiv  Detail & Related papers  (2022-10-08T13:14:02Z)
• Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs [148.0476219278875]
  We revisit large kernel design in modern convolutional neural networks (CNNs) Inspired by recent advances of vision transformers (ViTs), in this paper, we demonstrate that using a few large convolutional kernels instead of a stack of small kernels could be a more powerful paradigm. We propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31x31, in contrast to commonly used 3x3.
  arXiv  Detail & Related papers  (2022-03-13T17:22:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.