Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps

• URL: http://arxiv.org/abs/2209.12836v1
• Date: Mon, 26 Sep 2022 16:41:18 GMT
• Title: Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
• Authors: Yue Hu, Shaoheng Fang, Zixing Lei, Yiqi Zhong, Siheng Chen
• Abstract summary: Multi-agent collaborative perception could significantly upgrade the perception performance. It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. We propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. We propose Where2comm, a communication-efficient collaborative perception framework.
• Score: 24.47241495415147
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent collaborative perception could significantly upgrade the perception performance by enabling agents to share complementary information with each other through communication. It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. To tackle this bottleneck issue, we propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. It empowers agents to only share spatially sparse, yet perceptually critical information, contributing to where to communicate. Based on this novel spatial confidence map, we propose Where2comm, a communication-efficient collaborative perception framework. Where2comm has two distinct advantages: i) it considers pragmatic compression and uses less communication to achieve higher perception performance by focusing on perceptually critical areas; and ii) it can handle varying communication bandwidth by dynamically adjusting spatial areas involved in communication. To evaluate Where2comm, we consider 3D object detection in both real-world and simulation scenarios with two modalities (camera/LiDAR) and two agent types (cars/drones) on four datasets: OPV2V, V2X-Sim, DAIR-V2X, and our original CoPerception-UAVs. Where2comm consistently outperforms previous methods; for example, it achieves more than $100,000 \times$ lower communication volume and still outperforms DiscoNet and V2X-ViT on OPV2V. Our code is available at https://github.com/MediaBrain-SJTU/where2comm.

Related papers

• Deep Reinforcement Learning-Based User Scheduling for Collaborative Perception [24.300126250046894]
  Collaborative perception is envisioned to improve perceptual accuracy by using vehicle-to-everything (V2X) communication. Due to limited communication resources, it is impractical for all units to transmit sensing data such as point clouds or high-definition video. We propose a deep reinforcement learning-based V2X user scheduling algorithm for collaborative perception.
  arXiv  Detail & Related papers  (2025-02-12T04:45:00Z)
• Communication-Efficient Collaborative Perception via Information Filling with Codebook [48.087934650038044]
  Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents. To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection. By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system.
  arXiv  Detail & Related papers  (2024-05-08T11:12:37Z)
• V2X-PC: Vehicle-to-everything Collaborative Perception via Point Cluster [58.79477191603844]
  We introduce a new message unit, namely point cluster, to represent the scene sparsely with a combination of low-level structure information and high-level semantic information. This framework includes a Point Cluster Packing (PCP) module to keep object feature and manage bandwidth. Experiments on two widely recognized collaborative perception benchmarks showcase the superior performance of our method compared to the previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-25T11:24:02Z)
• Pragmatic Communication in Multi-Agent Collaborative Perception [80.14322755297788]
  Collaborative perception results in a trade-off between perception ability and communication costs. We propose PragComm, a multi-agent collaborative perception system with two key components. PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume.
  arXiv  Detail & Related papers  (2024-01-23T11:58:08Z)
• Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Driving [49.42873226593071]
  We propose V2X communication INterruption-aware COoperative Perception (V2X-INCOP) for V2X communication-aided autonomous driving. We use historical cooperation information to recover missing information due to the interruptions and alleviate the impact of the interruption issue. Experiments on three public cooperative perception datasets demonstrate that the proposed method is effective in alleviating the impacts of communication interruption on cooperative perception.
  arXiv  Detail & Related papers  (2023-04-24T04:59:13Z)
• V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer [58.71845618090022]
  We build a holistic attention model, namely V2X-ViT, to fuse information across on-road agents. V2X-ViT consists of alternating layers of heterogeneous multi-agent self-attention and multi-scale window self-attention. To validate our approach, we create a large-scale V2X perception dataset.
  arXiv  Detail & Related papers  (2022-03-20T20:18:25Z)
• Learning to Communicate and Correct Pose Errors [75.03747122616605]
  We study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. We propose a novel neural reasoning framework that learns to communicate, to estimate potential errors, and to reach a consensus about those errors.
  arXiv  Detail & Related papers  (2020-11-10T18:19:40Z)
• Who2com: Collaborative Perception via Learnable Handshake Communication [34.29310680302486]
  We propose the problem of collaborative perception, where robots can combine their local observations with those of neighboring agents in a learnable way to improve accuracy on a perception task. Inspired by networking communication protocols, we propose a multi-stage handshake communication mechanism where the neural network can learn to compress relevant information needed for each stage. We show that for the semantic segmentation task, our handshake communication method significantly improves accuracy by approximately 20% over decentralized baselines, and is comparable to centralized ones using a quarter of the bandwidth.
  arXiv  Detail & Related papers  (2020-03-21T04:16:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.