Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices

• URL: http://arxiv.org/abs/2410.10847v1
• Date: Tue, 01 Oct 2024 18:00:09 GMT
• Title: Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices
• Abstract summary: Two-stage object detectors exhibit high accuracy and precise localization, especially for identifying small objects. The computation costs associated with two-stage detection methods cause more severe thermal issues on edge devices. We propose Lotus, a novel framework that is tailored for two-stage detectors to dynamically scale CPU and GPU frequencies jointly.
• Score: 35.05167987125874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Two-stage object detectors exhibit high accuracy and precise localization, especially for identifying small objects that are favorable for various edge applications. However, the high computation costs associated with two-stage detection methods cause more severe thermal issues on edge devices, incurring dynamic runtime frequency change and thus large inference latency variations. Furthermore, the dynamic number of proposals in different frames leads to various computations over time, resulting in further latency variations. The significant latency variations of detectors on edge devices can harm user experience and waste hardware resources. To avoid thermal throttling and provide stable inference speed, we propose Lotus, a novel framework that is tailored for two-stage detectors to dynamically scale CPU and GPU frequencies jointly in an online manner based on deep reinforcement learning (DRL). To demonstrate the effectiveness of Lotus, we implement it on NVIDIA Jetson Orin Nano and Mi 11 Lite mobile platforms. The results indicate that Lotus can consistently and significantly reduce latency variation, achieve faster inference, and maintain lower CPU and GPU temperatures under various settings.

Related papers

• Lightweight Multi-Scale Anomaly Detection for Resource-Constrained Edge Devices [0.12891210250935145]
  Time-series anomaly detection is increasingly important in IoT systems, sensor networks, and edge monitoring applications.<n>Recent deep-learning approaches have improved detection accuracy, but many remain computationally expensive and often fail to capture subtle anomalies.<n>We propose a Lightweight MultiScale AutoEncoder network for univariate time-series anomaly detection, designed to be compact and computationally efficient.
  arXiv  Detail & Related papers  (2026-07-14T10:19:07Z)
• Taming Asynchronous CPU-GPU Coupling for Frequency-aware Latency Estimation on Mobile Edge [5.2773045718276705]
  We introduce FLAME to accurately estimate inference latency across frequency combinations.<n>It features a novel layer-wise modeling that quantifies the overlapping parallelism and then aggregates dynamic pipeline bubbles.<n>FLAME's precise modeling allows for profiling a sparse subset of samples, cutting profiling from hours to minutes and SLM profiling from days to mere minutes.
  arXiv  Detail & Related papers  (2026-04-11T12:26:06Z)
• Real-Time LiDAR Super-Resolution via Frequency-Aware Multi-Scale Fusion [0.4078247440919472]
  FLASH (Frequency-aware LiDAR Adaptive Super-resolution with Hierarchical fusion) is a novel framework that overcomes limitations through dual-domain processing.<n> FLASH integrates two key innovations: (i) Frequency-Aware Window Attention that combines local spatial attention with global frequency-domain analysis via FFT, capturing both fine-grained geometry and periodic scanning patterns at log-linear complexity, and (ii) Adaptive Multi-Scale Fusion that replaces conventional skip connections with learned position-specific feature aggregation, enhanced by CBAM attention for dynamic feature selection.
  arXiv  Detail & Related papers  (2025-11-10T18:38:15Z)
• Faster than Fast: Accelerating Oriented FAST Feature Detection on Low-end Embedded GPUs [11.639825636679454]
  This paper presents two methods to accelerate the Oriented FAST feature detection on low-end embedded GPU.<n>Experiments on a Jetson TX2 embedded GPU demonstrate an average speedup of over 7.3 times compared to widely used OpenCV with GPU support.
  arXiv  Detail & Related papers  (2025-06-08T14:30:30Z)
• SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model [52.47816604709358]
  Video anomaly detection (VAD) aims to identify unexpected events in videos and has wide applications in safety-critical domains. vision-language models (VLMs) have demonstrated strong multimodal reasoning capabilities, offering new opportunities for anomaly detection. We propose SlowFastVAD, a hybrid framework that integrates a fast anomaly detector with a slow anomaly detector.
  arXiv  Detail & Related papers  (2025-04-14T15:30:03Z)
• E2USD: Efficient-yet-effective Unsupervised State Detection for Multivariate Time Series [18.02694168117277]
  We propose E2Usd that enables efficient-yet-accurate unsupervised state detection. E2Usd exploits a Fast Fourier Transform-based Time Series and a Decomposed Dual-view Embedding Module. We also propose a False Negative Cancellation Contrastive Learning method to counteract the effects of false negatives.
  arXiv  Detail & Related papers  (2024-02-21T10:16:57Z)
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach [76.45949280328838]
  We propose a Laplacian enhanced low-rank tensor (LETC) framework featuring both lowrankness and multi-temporal correlations for large-scale traffic speed kriging. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging.
  arXiv  Detail & Related papers  (2022-10-21T07:25:57Z)
• Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design [66.39546326221176]
  Attention-based neural networks have become pervasive in many AI tasks. The use of the attention mechanism and feed-forward network (FFN) demands excessive computational and memory resources. This paper proposes a hardware-friendly variant that adopts a unified butterfly sparsity pattern to approximate both the attention mechanism and the FFNs.
  arXiv  Detail & Related papers  (2022-09-20T09:28:26Z)
• Robust Time Series Dissimilarity Measure for Outlier Detection and Periodicity Detection [16.223509730658513]
  We propose a novel time series dissimilarity measure named RobustDTW to reduce the effects of noises and outliers. Specifically, the RobustDTW estimates the trend and optimize the time warp in an alternating manner by utilizing our designed temporal graph trend filtering. Experiments on real-world datasets demonstrate the superior performance of RobustDTW compared to DTW variants in both outlier time series detection and periodicity detection.
  arXiv  Detail & Related papers  (2022-06-07T00:49:16Z)
• Hardware-Robust In-RRAM-Computing for Object Detection [0.15113576014047125]
  In-RRAM computing suffered from large device variation and numerous nonideal effects in hardware. This paper proposes a joint hardware and software optimization strategy to design a hardware-robust IRC macro for object detection. The proposed approach has been successfully applied to a complex object detection task with only 3.85% mAP drop.
  arXiv  Detail & Related papers  (2022-05-09T01:46:24Z)
• Taking ROCKET on an Efficiency Mission: Multivariate Time Series Classification with LightWaveS [3.5786621294068373]
  We present LightWaveS, a framework for accurate multivariate time series classification. It employs just 2.5% of the ROCKET features, while achieving accuracy comparable to recent deep learning models. We show that we achieve speedup ranging from 9x to 65x compared to ROCKET during inference on an edge device.
  arXiv  Detail & Related papers  (2022-04-04T10:52:20Z)
• Real-Time GPU-Accelerated Machine Learning Based Multiuser Detection for 5G and Beyond [70.81551587109833]
  nonlinear beamforming filters can significantly outperform linear approaches in stationary scenarios with massive connectivity. One of the main challenges comes from the real-time implementation of these algorithms. This paper explores the acceleration of APSM-based algorithms through massive parallelization.
  arXiv  Detail & Related papers  (2022-01-13T15:20:45Z)
• FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation [81.76975488010213]
  Dense optical flow estimation plays a key role in many robotic vision tasks. Current networks often occupy large number of parameters and require heavy computation costs. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
  arXiv  Detail & Related papers  (2021-03-08T03:09:37Z)
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference [82.1584439276834]
  Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. We present EdgeBERT, an in-depth algorithm- hardware co-design for latency-aware energy optimization for multi-task NLP.
  arXiv  Detail & Related papers  (2020-11-28T19:21:47Z)
• Joint Detection and Tracking in Videos with Identification Features [36.55599286568541]
  We propose the first joint optimization of detection, tracking and re-identification features for videos. Our method reaches the state-of-the-art on MOT, it ranks 1st in the UA-DETRAC'18 tracking challenge among online trackers, and 3rd overall.
  arXiv  Detail & Related papers  (2020-05-21T21:06:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.