Towards Latency-Aware 3D Streaming Perception for Autonomous Driving

• URL: http://arxiv.org/abs/2504.19115v1
• Date: Sun, 27 Apr 2025 05:49:52 GMT
• Title: Towards Latency-Aware 3D Streaming Perception for Autonomous Driving
• Authors: Jiaqi Peng, Tai Wang, Jiangmiao Pang, Yuan Shen,
• Abstract summary: We propose a new benchmark tailored for online evaluation by considering runtime latency.<n>Based on the benchmark, we build a latency-aware 3D Streaming Perception framework.<n>Our method shows generalization across various latency levels, achieving an online performance that closely aligns with 80% of its offline evaluation.
• Score: 25.879279738510398
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although existing 3D perception algorithms have demonstrated significant improvements in performance, their deployment on edge devices continues to encounter critical challenges due to substantial runtime latency. We propose a new benchmark tailored for online evaluation by considering runtime latency. Based on the benchmark, we build a Latency-Aware 3D Streaming Perception (LASP) framework that addresses the latency issue through two primary components: 1) latency-aware history integration, which extends query propagation into a continuous process, ensuring the integration of historical feature regardless of varying latency; 2) latency-aware predictive detection, a module that compensates the detection results with the predicted trajectory and the posterior accessed latency. By incorporating the latency-aware mechanism, our method shows generalization across various latency levels, achieving an online performance that closely aligns with 80\% of its offline evaluation on the Jetson AGX Orin without any acceleration techniques.

Related papers

• On Latency Predictors for Neural Architecture Search [8.564763702766776]
  We introduce a comprehensive suite of latency prediction tasks obtained in a principled way through automated partitioning of hardware device sets. We then design a general latency predictor to comprehensively study (1) the predictor architecture, (2) NN sample selection methods, (3) hardware device representations, and (4) NN operation encoding schemes. Building on conclusions from our study, we present an end-to-end latency predictor training strategy.
  arXiv  Detail & Related papers  (2024-03-04T19:59:32Z)
• Latency-aware Unified Dynamic Networks for Efficient Image Recognition [72.8951331472913]
  LAUDNet is a framework to bridge the theoretical and practical efficiency gap in dynamic networks. It integrates three primary dynamic paradigms-spatially adaptive computation, dynamic layer skipping, and dynamic channel skipping. It can notably reduce the latency of models like ResNet by over 50% on platforms such as V100,3090, and TX2 GPUs.
  arXiv  Detail & Related papers  (2023-08-30T10:57:41Z)
• An Intelligent Deterministic Scheduling Method for Ultra-Low Latency Communication in Edge Enabled Industrial Internet of Things [19.277349546331557]
  Time Sensitive Network (TSN) is recently researched to realize low latency communication via deterministic scheduling. Non-collision theory based deterministic scheduling (NDS) method is proposed to achieve ultra-low latency communication for the time-sensitive flows. Experiment results demonstrate that NDS/DQS can well support deterministic ultra-low latency services and guarantee efficient bandwidth utilization.
  arXiv  Detail & Related papers  (2022-07-17T16:52:51Z)
• MAPLE-X: Latency Prediction with Explicit Microprocessor Prior Knowledge [87.41163540910854]
  Deep neural network (DNN) latency characterization is a time-consuming process. We propose MAPLE-X which extends MAPLE by incorporating explicit prior knowledge of hardware devices and DNN architecture latency.
  arXiv  Detail & Related papers  (2022-05-25T11:08:20Z)
• MAPLE-Edge: A Runtime Latency Predictor for Edge Devices [80.01591186546793]
  We propose MAPLE-Edge, an edge device-oriented extension of MAPLE, the state-of-the-art latency predictor for general purpose hardware. Compared to MAPLE, MAPLE-Edge can describe the runtime and target device platform using a much smaller set of CPU performance counters. We also demonstrate that unlike MAPLE which performs best when trained on a pool of devices sharing a common runtime, MAPLE-Edge can effectively generalize across runtimes.
  arXiv  Detail & Related papers  (2022-04-27T14:00:48Z)
• Real-time Object Detection for Streaming Perception [84.2559631820007]
  Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. We build a simple and effective framework for streaming perception. Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
  arXiv  Detail & Related papers  (2022-03-23T11:33:27Z)
• DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-based Systems [0.76146285961466]
  We present DeLag, a novel automated search-based approach for diagnosing performance issues in service-based systems. DeLag simultaneously searches for multiple latency patterns while optimizing precision, recall and dissimilarity.
  arXiv  Detail & Related papers  (2021-10-21T13:59:32Z)
• LC-NAS: Latency Constrained Neural Architecture Search for Point Cloud Networks [73.78551758828294]
  LC-NAS is able to find state-of-the-art architectures for point cloud classification with minimal computational cost. We show how our searched architectures achieve any desired latency with a reasonably low drop in accuracy.
  arXiv  Detail & Related papers  (2020-08-24T10:30:21Z)
• Towards Streaming Perception [70.68520310095155]
  We present an approach that coherently integrates latency and accuracy into a single metric for real-time online perception. The key insight behind this metric is to jointly evaluate the output of the entire perception stack at every time instant. We focus on the illustrative tasks of object detection and instance segmentation in urban video streams, and contribute a novel dataset with high-quality and temporally-dense annotations.
  arXiv  Detail & Related papers  (2020-05-21T01:51:35Z)
• Streaming Object Detection for 3-D Point Clouds [29.465873948076766]
  LiDAR provides a prominent sensory modality that informs many existing perceptual systems. The latency for perceptual systems based on point cloud data can be dominated by the amount of time for a complete rotational scan. We show how operating on LiDAR data in its native streaming formulation offers several advantages for self driving object detection.
  arXiv  Detail & Related papers  (2020-05-04T21:55:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.