FemtoDet: An Object Detection Baseline for Energy Versus Performance Tradeoffs

• URL: http://arxiv.org/abs/2301.06719v5
• Date: Sun, 13 Aug 2023 17:25:45 GMT
• Title: FemtoDet: An Object Detection Baseline for Energy Versus Performance Tradeoffs
• Authors: Peng Tu, Xu Xie, Guo AI, Yuexiang Li, Yawen Huang, Yefeng Zheng
• Abstract summary: Vision applications of convolutional neural networks, such as always-on surveillance cameras, are critical for energy constraints. This paper aims to serve as a baseline by designing detectors to reach tradeoffs between energy and performance from two perspectives.
• Score: 27.006082622843653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient detectors for edge devices are often optimized for parameters or speed count metrics, which remain in weak correlation with the energy of detectors. However, some vision applications of convolutional neural networks, such as always-on surveillance cameras, are critical for energy constraints. This paper aims to serve as a baseline by designing detectors to reach tradeoffs between energy and performance from two perspectives: 1) We extensively analyze various CNNs to identify low-energy architectures, including selecting activation functions, convolutions operators, and feature fusion structures on necks. These underappreciated details in past work seriously affect the energy consumption of detectors; 2) To break through the dilemmatic energy-performance problem, we propose a balanced detector driven by energy using discovered low-energy components named \textit{FemtoDet}. In addition to the novel construction, we improve FemtoDet by considering convolutions and training strategy optimizations. Specifically, we develop a new instance boundary enhancement (IBE) module for convolution optimization to overcome the contradiction between the limited capacity of CNNs and detection tasks in diverse spatial representations, and propose a recursive warm-restart (RecWR) for optimizing training strategy to escape the sub-optimization of light-weight detectors by considering the data shift produced in popular augmentations. As a result, FemtoDet with only 68.77k parameters achieves a competitive score of 46.3 AP50 on PASCAL VOC and 1.11 W $\&$ 64.47 FPS on Qualcomm Snapdragon 865 CPU platforms. Extensive experiments on COCO and TJU-DHD datasets indicate that the proposed method achieves competitive results in diverse scenes.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Towards Energy-Efficiency by Navigating the Trilemma of Energy, Latency, and Accuracy [10.597365664501135]
  Extended Reality (XR) enables immersive experiences through untethered headsets but suffers from stringent battery and resource constraints. This paper examines scene reconstruction, a key building block for immersive XR experiences, and demonstrates how energy efficiency can be achieved.
  arXiv  Detail & Related papers  (2024-09-06T04:10:33Z)
• Federated Learning With Energy Harvesting Devices: An MDP Framework [5.852486435612777]
  Federated learning (FL) requires edge devices to perform local training and exchange information with a parameter server. A critical challenge in practical FL systems is the rapid energy depletion of battery-limited edge devices. We apply energy harvesting technique in FL systems to extract ambient energy for continuously powering edge devices.
  arXiv  Detail & Related papers  (2024-05-17T03:41:40Z)
• MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection [54.545054873239295]
  Deepfakes have recently raised significant trust issues and security concerns among the public. ViT-based methods take advantage of the expressivity of transformers, achieving superior detection performance. This work introduces Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized yet parameter-efficient ViT-based approach.
  arXiv  Detail & Related papers  (2024-04-12T13:02:08Z)
• PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices [10.01838504586422]
  The continuous operation of ML-powered systems leads to significant energy use during inference. This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, affects energy consumption for NN inference with regular fine-tuning. We propose PolyThrottle, a solution that optimize configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner.
  arXiv  Detail & Related papers  (2023-10-30T20:19:41Z)
• CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion [11.313017866190622]
  We propose CARMA: a context-aware sensor fusion approach that uses context to reconfigure computation flow at runtime. CARMA significantly reduces the energy used by a multi-sensory object detector without compromising performance. We evaluate multiple context-identification strategies, propose a novel system-wide energy-performance joint optimization, and evaluate scenario-specific perception performance.
  arXiv  Detail & Related papers  (2023-06-27T19:00:07Z)
• Efficient Decoder-free Object Detection with Transformers [75.00499377197475]
  Vision transformers (ViTs) are changing the landscape of object detection approaches. We propose a decoder-free fully transformer-based (DFFT) object detector. DFFT_SMALL achieves high efficiency in both training and inference stages.
  arXiv  Detail & Related papers  (2022-06-14T13:22:19Z)
• Federated Learning for Energy-limited Wireless Networks: A Partial Model Aggregation Approach [79.59560136273917]
  limited communication resources, bandwidth and energy, and data heterogeneity across devices are main bottlenecks for federated learning (FL) We first devise a novel FL framework with partial model aggregation (PMA) The proposed PMA-FL improves 2.72% and 11.6% accuracy on two typical heterogeneous datasets.
  arXiv  Detail & Related papers  (2022-04-20T19:09:52Z)
• Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer [42.30741737568212]
  We propose the solution of powering devices using wireless power transfer (WPT) This work aims at the derivation of guidelines on deploying the resultant wirelessly powered FEEL (WP-FEEL) system. The results provide useful guidelines on WPT provisioning to provide a guaranteer on learning performance.
  arXiv  Detail & Related papers  (2021-02-24T15:47:34Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• Anchor-free Small-scale Multispectral Pedestrian Detection [88.7497134369344]
  We propose a method for effective and efficient multispectral fusion of the two modalities in an adapted single-stage anchor-free base architecture. We aim at learning pedestrian representations based on object center and scale rather than direct bounding box predictions. Results show our method's effectiveness in detecting small-scaled pedestrians.
  arXiv  Detail & Related papers  (2020-08-19T13:13:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.