Latency-Aware Collaborative Perception

• URL: http://arxiv.org/abs/2207.08560v2
• Date: Thu, 21 Jul 2022 01:35:53 GMT
• Title: Latency-Aware Collaborative Perception
• Authors: Zixing Lei, Shunli Ren, Yue Hu, Wenjun Zhang and Siheng Chen
• Abstract summary: We present the first latency-aware collaborative perception system, which actively adapts asynchronous perceptual features from multiple agents to the same time stamp. Experiments show that the proposed latency aware collaborative perception system with SyncNet can outperforms the state-of-the-art collaborative perception method by 15.6% in the communication latency scenario.
• Score: 31.421282624961883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative perception has recently shown great potential to improve perception capabilities over single-agent perception. Existing collaborative perception methods usually consider an ideal communication environment. However, in practice, the communication system inevitably suffers from latency issues, causing potential performance degradation and high risks in safety-critical applications, such as autonomous driving. To mitigate the effect caused by the inevitable latency, from a machine learning perspective, we present the first latency-aware collaborative perception system, which actively adapts asynchronous perceptual features from multiple agents to the same time stamp, promoting the robustness and effectiveness of collaboration. To achieve such a feature-level synchronization, we propose a novel latency compensation module, called SyncNet, which leverages feature-attention symbiotic estimation and time modulation techniques. Experiments results show that the proposed latency aware collaborative perception system with SyncNet can outperforms the state-of-the-art collaborative perception method by 15.6% in the communication latency scenario and keep collaborative perception being superior to single agent perception under severe latency.

Related papers

• CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridization [23.958663737034318]
  We propose a novel communication-efficient collaborative perception framework based on supply-demand awareness and intermediate-late hybridization. Experiments on multiple datasets, including both simulated and real-world scenarios, demonstrate that mymethodname achieves state-of-the-art detection accuracy and optimal bandwidth trade-offs.
  arXiv  Detail & Related papers  (2025-03-05T12:02:04Z)
• CoDynTrust: Robust Asynchronous Collaborative Perception via Dynamic Feature Trust Modulus [9.552300496606644]
  Collaborative perception, fusing information from multiple agents, can extend perception range so as to improve performance. temporal asynchrony in real-world environments, caused by communication delays, clock misalignment, or sampling configuration differences, can lead to information mismatches. We propose CoDynTrust, an uncertainty-encoded asynchronous fusion perception framework that is robust to the information mismatches caused by temporal asynchrony.
  arXiv  Detail & Related papers  (2025-02-12T07:23:26Z)
• mmCooper: A Multi-agent Multi-stage Communication-efficient and Collaboration-robust Cooperative Perception Framework [12.896563384343889]
  mmCooper is a novel multi-agent, multi-stage, communication-efficient, and collaboration-robust cooperative perception framework. Our framework balances intermediate- and late-stage information to share among agents, enhancing perceptual performance while maintaining communication efficiency.
  arXiv  Detail & Related papers  (2025-01-21T16:34:16Z)
• OFDM-Standard Compatible SC-NOFS Waveforms for Low-Latency and Jitter-Tolerance Industrial IoT Communications [53.398544571833135]
  This work proposes a spectrally efficient irregular Sinc (irSinc) shaping technique, revisiting the traditional Sinc back to 1924. irSinc yields a signal with increased spectral efficiency without sacrificing error performance. Our signal achieves faster data transmission within the same spectral bandwidth through 5G standard signal configuration.
  arXiv  Detail & Related papers  (2024-06-07T09:20:30Z)
• Robust Collaborative Perception without External Localization and Clock Devices [52.32342059286222]
  A consistent spatial-temporal coordination across multiple agents is fundamental for collaborative perception. Traditional methods depend on external devices to provide localization and clock signals. We propose a novel approach: aligning by recognizing the inherent geometric patterns within the perceptual data of various agents.
  arXiv  Detail & Related papers  (2024-05-05T15:20:36Z)
• Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunities [10.749959052350594]
  Collaborative perception with connected and autonomous vehicles (CAVs) shows a promising solution to overcoming these limitations. In this article, we first identify the challenges of collaborative perception, such as data sharing asynchrony, data volume, and pose errors. We propose a scheme to deal with communication efficiency and latency problems, which is a channel-aware collaborative perception framework.
  arXiv  Detail & Related papers  (2024-01-03T05:33:14Z)
• Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detection [9.967263440745432]
  Occlusion is a major challenge for LiDAR-based object detection methods. State-of-the-art V2X methods resolve the performance-bandwidth tradeoff using a mid-collaboration approach. We devise a simple yet effective collaboration method that achieves a better bandwidth-performance tradeoff than prior methods.
  arXiv  Detail & Related papers  (2023-07-04T03:49:42Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism [33.116006446428756]
  We study multi-agent online learning problems in the presence of delays and asynchronicities. We derive adaptive learning strategies with optimal regret bounds, at both the agent and network levels.
  arXiv  Detail & Related papers  (2020-12-21T18:55:55Z)
• Learning to Communicate and Correct Pose Errors [75.03747122616605]
  We study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. We propose a novel neural reasoning framework that learns to communicate, to estimate potential errors, and to reach a consensus about those errors.
  arXiv  Detail & Related papers  (2020-11-10T18:19:40Z)
• Towards Streaming Perception [70.68520310095155]
  We present an approach that coherently integrates latency and accuracy into a single metric for real-time online perception. The key insight behind this metric is to jointly evaluate the output of the entire perception stack at every time instant. We focus on the illustrative tasks of object detection and instance segmentation in urban video streams, and contribute a novel dataset with high-quality and temporally-dense annotations.
  arXiv  Detail & Related papers  (2020-05-21T01:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.