Communication-Efficient Collaborative Perception via Information Filling with Codebook

• URL: http://arxiv.org/abs/2405.04966v1
• Date: Wed, 8 May 2024 11:12:37 GMT
• Title: Communication-Efficient Collaborative Perception via Information Filling with Codebook
• Authors: Yue Hu, Juntong Peng, Sifei Liu, Junhao Ge, Si Liu, Siheng Chen,
• Abstract summary: Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents. To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection. By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system.
• Score: 48.087934650038044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents. It inherently results in a fundamental trade-off between perception ability and communication cost. To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection. The proposed codebook-based message representation enables the transmission of integer codes, rather than high-dimensional feature maps. The proposed information-filling-driven message selection optimizes local messages to collectively fill each agent's information demand, preventing information overflow among multiple agents. By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system, which significantly advances the perception-communication trade-off and is inclusive to both homogeneous and heterogeneous collaboration settings. We evaluate CodeFilling in both a real-world dataset, DAIR-V2X, and a new simulation dataset, OPV2VH+. Results show that CodeFilling outperforms previous SOTA Where2comm on DAIR-V2X/OPV2VH+ with 1,333/1,206 times lower communication volume. Our code is available at https://github.com/PhyllisH/CodeFilling.

Related papers

• Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification [49.66272783945571]
  Federated Knowledge Graphs Embedding learning (FKGE) encounters challenges in communication efficiency stemming from the considerable size of parameters and extensive communication rounds. We propose bidirectional communication-efficient FedS based on Entity-Wise Top-K Sparsification strategy.
  arXiv  Detail & Related papers  (2024-06-19T05:26:02Z)
• V2X-PC: Vehicle-to-everything Collaborative Perception via Point Cluster [58.79477191603844]
  We introduce a new message unit, namely point cluster, to represent the scene sparsely with a combination of low-level structure information and high-level semantic information. This framework includes a Point Cluster Packing (PCP) module to keep object feature and manage bandwidth. Experiments on two widely recognized collaborative perception benchmarks showcase the superior performance of our method compared to the previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-25T11:24:02Z)
• What Makes Good Collaborative Views? Contrastive Mutual Information Maximization for Multi-Agent Perception [52.41695608928129]
  Multi-agent perception (MAP) allows autonomous systems to understand complex environments by interpreting data from multiple sources. This paper investigates intermediate collaboration for MAP with a specific focus on exploring "good" properties of collaborative view. We propose a novel framework named CMiMC for intermediate collaboration.
  arXiv  Detail & Related papers  (2024-03-15T07:18:55Z)
• Pragmatic Communication in Multi-Agent Collaborative Perception [80.14322755297788]
  Collaborative perception results in a trade-off between perception ability and communication costs. We propose PragComm, a multi-agent collaborative perception system with two key components. PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume.
  arXiv  Detail & Related papers  (2024-01-23T11:58:08Z)
• Context-aware Communication for Multi-agent Reinforcement Learning [6.109127175562235]
  We develop a context-aware communication scheme for multi-agent reinforcement learning (MARL) In the first stage, agents exchange coarse representations in a broadcast fashion, providing context for the second stage. Following this, agents utilize attention mechanisms in the second stage to selectively generate messages personalized for the receivers. To evaluate the effectiveness of CACOM, we integrate it with both actor-critic and value-based MARL algorithms.
  arXiv  Detail & Related papers  (2023-12-25T03:33:08Z)
• Select2Col: Leveraging Spatial-Temporal Importance of Semantic Information for Efficient Collaborative Perception [21.043094544649733]
  Collaborative perception by leveraging the shared semantic information plays a crucial role in overcoming the individual limitations of isolated agents. Existing collaborative perception methods tend to focus solely on the spatial features of semantic information, while neglecting the importance of the temporal dimension. We propose Select2Col, a novel collaborative perception framework that takes into account the underlinespatial-tunderlinee of semantiunderlinec informaunderlinetion.
  arXiv  Detail & Related papers  (2023-07-31T09:33:19Z)
• Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps [24.47241495415147]
  Multi-agent collaborative perception could significantly upgrade the perception performance. It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. We propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. We propose Where2comm, a communication-efficient collaborative perception framework.
  arXiv  Detail & Related papers  (2022-09-26T16:41:18Z)
• Multi-agent Communication with Graph Information Bottleneck under Limited Bandwidth (a position paper) [92.11330289225981]
  In many real-world scenarios, communication can be expensive and the bandwidth of the multi-agent system is subject to certain constraints. Redundant messages who occupy the communication resources can block the transmission of informative messages and thus jeopardize the performance. We propose a novel multi-agent communication module, CommGIB, which effectively compresses the structure information and node information in the communication graph to deal with bandwidth-constrained settings.
  arXiv  Detail & Related papers  (2021-12-20T07:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.