Efficient Modulation for Vision Networks

• URL: http://arxiv.org/abs/2403.19963v1
• Date: Fri, 29 Mar 2024 03:48:35 GMT
• Title: Efficient Modulation for Vision Networks
• Authors: Xu Ma, Xiyang Dai, Jianwei Yang, Bin Xiao, Yinpeng Chen, Yun Fu, Lu Yuan,
• Abstract summary: We propose efficient modulation, a novel design for efficient vision networks. We demonstrate that the modulation mechanism is particularly well suited for efficient networks. Our network can accomplish better trade-offs between accuracy and efficiency.
• Score: 122.1051910402034
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this work, we present efficient modulation, a novel design for efficient vision networks. We revisit the modulation mechanism, which operates input through convolutional context modeling and feature projection layers, and fuses features via element-wise multiplication and an MLP block. We demonstrate that the modulation mechanism is particularly well suited for efficient networks and further tailor the modulation design by proposing the efficient modulation (EfficientMod) block, which is considered the essential building block for our networks. Benefiting from the prominent representational ability of modulation mechanism and the proposed efficient design, our network can accomplish better trade-offs between accuracy and efficiency and set new state-of-the-art performance in the zoo of efficient networks. When integrating EfficientMod with the vanilla self-attention block, we obtain the hybrid architecture which further improves the performance without loss of efficiency. We carry out comprehensive experiments to verify EfficientMod's performance. With fewer parameters, our EfficientMod-s performs 0.6 top-1 accuracy better than EfficientFormerV2-s2 and is 25% faster on GPU, and 2.9 better than MobileViTv2-1.0 at the same GPU latency. Additionally, our method presents a notable improvement in downstream tasks, outperforming EfficientFormerV2-s by 3.6 mIoU on the ADE20K benchmark. Code and checkpoints are available at https://github.com/ma-xu/EfficientMod.

Related papers

• EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference [49.94169109038806]
  This paper introduces EPS-MoE, a novel expert pipeline scheduler for MoE. Our results demonstrate an average 21% improvement in prefill throughput over existing parallel inference methods.
  arXiv  Detail & Related papers  (2024-10-16T05:17:49Z)
• Rapid and Power-Aware Learned Optimization for Modular Receive Beamforming [27.09017677987757]
  Multiple-input multiple-output (MIMO) systems play a key role in wireless communication technologies. We propose a power-oriented optimization algorithm for beamforming in modular hybrid systems. We show how power efficient beamforming can be encouraged by the learned, via boosting computation with low-resolution phase shifts.
  arXiv  Detail & Related papers  (2024-08-01T10:19:25Z)
• Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation [67.13876021157887]
  Dynamic Tuning (DyT) is a novel approach to improve both parameter and inference efficiency for ViT adaptation. DyT achieves superior performance compared to existing PEFT methods while evoking only 71% of their FLOPs on the VTAB-1K benchmark.
  arXiv  Detail & Related papers  (2024-03-18T14:05:52Z)
• EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention [44.148667664413004]
  We propose a family of high-speed vision transformers named EfficientViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations. To address this, we present a cascaded group attention module feeding attention heads with different splits.
  arXiv  Detail & Related papers  (2023-05-11T17:59:41Z)
• Rethinking Mobile Block for Efficient Attention-based Models [60.0312591342016]
  This paper focuses on developing modern, efficient, lightweight models for dense predictions while trading off parameters, FLOPs, and performance. Inverted Residual Block (IRB) serves as the infrastructure for lightweight CNNs, but no counterpart has been recognized by attention-based studies. We extend CNN-based IRB to attention-based models and abstracting a one-residual Meta Mobile Block (MMB) for lightweight model design.
  arXiv  Detail & Related papers  (2023-01-03T15:11:41Z)
• Rethinking Vision Transformers for MobileNet Size and Speed [58.01406896628446]
  We propose a novel supernet with low latency and high parameter efficiency. We also introduce a novel fine-grained joint search strategy for transformer models. This work demonstrate that properly designed and optimized vision transformers can achieve high performance even with MobileNet-level size and speed.
  arXiv  Detail & Related papers  (2022-12-15T18:59:12Z)
• Virtuoso: Video-based Intelligence for real-time tuning on SOCs [24.086595996055074]
  Underlying Virtuoso is a multi-branch execution kernel capable of running at different operating points in the accuracy-energy-latency axes. We benchmark 15 state-of-the-art or widely used protocols, including Faster R-CNN (FRCNN), YOLO v3, SSD, EfficientDet, SELSA, MEGA, REPP, FastAdapt, and our in-house adaptive variants of FRCNN+, YOLO+, SSD+, and EfficientDet+.
  arXiv  Detail & Related papers  (2021-12-24T14:47:41Z)
• HANT: Hardware-Aware Network Transformation [82.54824188745887]
  We propose hardware-aware network transformation (HANT) HANT replaces inefficient operations with more efficient alternatives using a neural architecture search like approach. Our results on accelerating the EfficientNet family show that HANT can accelerate them by up to 3.6x with 0.4% drop in the top-1 accuracy on the ImageNet dataset.
  arXiv  Detail & Related papers  (2021-07-12T18:46:34Z)
• Weight-dependent Gates for Network Pruning [24.795174721078528]
  This paper argues that the pruning decision should depend on the convolutional weights, and thus proposes novel weight-dependent gates (W-Gates) to learn the information from filter weights and obtain binary gates to prune or keep the filters automatically. We have demonstrated the effectiveness of the proposed method on ResNet34, ResNet50, and MobileNet V2, respectively achieving up to 1.33/1.28/1.1 higher Top-1 accuracy with lower hardware latency on ImageNet.
  arXiv  Detail & Related papers  (2020-07-04T10:29:07Z)
• An Efficient Accelerator Design Methodology for Deformable Convolutional Networks [16.392643034008348]
  We present a novel approach to accelerate deformable convolution on FPGA. By optimizing the receptive field, we can compress the maximum size of the receptive field by 12.6 times. Our accelerator achieves up to 17.25 times speedup over the state-of-the-art accelerator.
  arXiv  Detail & Related papers  (2020-06-09T13:16:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.