CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion

• URL: http://arxiv.org/abs/2306.15748v1
• Date: Tue, 27 Jun 2023 19:00:07 GMT
• Title: CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion
• Authors: Yifan Zhang, Arnav Vaibhav Malawade, Xiaofang Zhang, Yuhui Li, DongHwan Seong, Mohammad Abdullah Al Faruque and Sitao Huang
• Abstract summary: We propose CARMA: a context-aware sensor fusion approach that uses context to reconfigure computation flow at runtime. CARMA significantly reduces the energy used by a multi-sensory object detector without compromising performance. We evaluate multiple context-identification strategies, propose a novel system-wide energy-performance joint optimization, and evaluate scenario-specific perception performance.
• Score: 11.313017866190622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous systems (AS) are systems that can adapt and change their behavior in response to unanticipated events and include systems such as aerial drones, autonomous vehicles, and ground/aquatic robots. AS require a wide array of sensors, deep-learning models, and powerful hardware platforms to perceive and safely operate in real-time. However, in many contexts, some sensing modalities negatively impact perception while increasing the system's overall energy consumption. Since AS are often energy-constrained edge devices, energy-efficient sensor fusion methods have been proposed. However, existing methods either fail to adapt to changing scenario conditions or to optimize energy efficiency system-wide. We propose CARMA: a context-aware sensor fusion approach that uses context to dynamically reconfigure the computation flow on a Field-Programmable Gate Array (FPGA) at runtime. By clock-gating unused sensors and model sub-components, CARMA significantly reduces the energy used by a multi-sensory object detector without compromising performance. We use a Deep-learning Processor Unit (DPU) based reconfiguration approach to minimize the latency of model reconfiguration. We evaluate multiple context-identification strategies, propose a novel system-wide energy-performance joint optimization, and evaluate scenario-specific perception performance. Across challenging real-world sensing contexts, CARMA outperforms state-of-the-art methods with up to 1.3x speedup and 73% lower energy consumption.

Related papers

• Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes [56.52618054240197]
  We propose a novel, condition-aware multimodal fusion approach for robust semantic perception of driving scenes. Our method, CAFuser, uses an RGB camera input to classify environmental conditions and generate a Condition Token that guides the fusion of multiple sensor modalities. We set the new state of the art with CAFuser on the MUSES dataset with 59.7 PQ for multimodal panoptic segmentation and 78.2 mIoU for semantic segmentation, ranking first on the public benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T17:56:20Z)
• Event-based vision on FPGAs -- a survey [0.0]
  Field programmable gate Arrays (FPGAs) have enabled fast data processing (even in real-time) and energy efficiency. This paper gives an overview of the most important works, where FPGAs have been used in different contexts to process event data. It covers applications in the following areas: filtering, stereovision, optical flow, acceleration of AI-based algorithms for object classification, detection and tracking, and applications in robotics and inspection systems.
  arXiv  Detail & Related papers  (2024-07-11T10:07:44Z)
• Federated Learning With Energy Harvesting Devices: An MDP Framework [5.852486435612777]
  Federated learning (FL) requires edge devices to perform local training and exchange information with a parameter server. A critical challenge in practical FL systems is the rapid energy depletion of battery-limited edge devices. We apply energy harvesting technique in FL systems to extract ambient energy for continuously powering edge devices.
  arXiv  Detail & Related papers  (2024-05-17T03:41:40Z)
• Context-aware Multi-Model Object Detection for Diversely Heterogeneous Compute Systems [0.32634122554914]
  One-size-fits-all approach to object detection using deep neural networks (DNNs) leads to inefficient utilization of computational resources. We propose SHIFT which continuously selects from a variety of DNN-based OD models depending on the dynamically changing contextual information and computational constraints. Our proposed methodology results in improvements of up to 7.5x in energy usage and 2.8x in latency compared to state-of-the-art GPU-based single model OD approaches.
  arXiv  Detail & Related papers  (2024-02-12T05:38:11Z)
• Random resistive memory-based deep extreme point learning machine for unified visual processing [67.51600474104171]
  We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM) Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
  arXiv  Detail & Related papers  (2023-12-14T09:46:16Z)
• PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices [10.01838504586422]
  The continuous operation of ML-powered systems leads to significant energy use during inference. This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, affects energy consumption for NN inference with regular fine-tuning. We propose PolyThrottle, a solution that optimize configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner.
  arXiv  Detail & Related papers  (2023-10-30T20:19:41Z)
• Multiagent Reinforcement Learning with an Attention Mechanism for Improving Energy Efficiency in LoRa Networks [52.96907334080273]
  As the network scale increases, the energy efficiency of LoRa networks decreases sharply due to severe packet collisions. We propose a transmission parameter allocation algorithm based on multiagent reinforcement learning (MALoRa) Simulation results demonstrate that MALoRa significantly improves the system EE compared with baseline algorithms.
  arXiv  Detail & Related papers  (2023-09-16T11:37:23Z)
• FemtoDet: An Object Detection Baseline for Energy Versus Performance Tradeoffs [27.006082622843653]
  Vision applications of convolutional neural networks, such as always-on surveillance cameras, are critical for energy constraints. This paper aims to serve as a baseline by designing detectors to reach tradeoffs between energy and performance from two perspectives.
  arXiv  Detail & Related papers  (2023-01-17T06:24:08Z)
• Multi-Point Integrated Sensing and Communication: Fusion Model and Functionality Selection [99.67715229413986]
  This paper presents a multi-point ISAC (MPISAC) system that fuses the outputs from multiple ISAC devices for achieving higher sensing performance. We adopt a fusion model that predicts the fusion accuracy via hypothesis testing and optimal voting analysis.
  arXiv  Detail & Related papers  (2022-08-16T08:09:54Z)
• LEAF + AIO: Edge-Assisted Energy-Aware Object Detection for Mobile Augmented Reality [77.00418462388525]
  Mobile augmented reality (MAR) applications are significantly energy-guzzling. We design an edge-based energy-aware MAR system that enables MAR devices to dynamically change their configurations. Our proposed dynamic MAR configuration adaptations can minimize the per frame energy consumption of multiple MAR clients.
  arXiv  Detail & Related papers  (2022-05-27T06:11:50Z)
• Energy Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space [62.39318039798564]
  We propose a scheduling mechanism capable of taking advantage of correlated information. The proposed mechanism is capable of determining the frequency with which sensors should transmit their updates. We show that our solution can significantly extend the sensors' lifetime.
  arXiv  Detail & Related papers  (2020-11-19T09:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.