Pragmatic Communication in Multi-Agent Collaborative Perception
- URL: http://arxiv.org/abs/2401.12694v1
- Date: Tue, 23 Jan 2024 11:58:08 GMT
- Title: Pragmatic Communication in Multi-Agent Collaborative Perception
- Authors: Yue Hu, Xianghe Pang, Xiaoqi Qin, Yonina C. Eldar, Siheng Chen, Ping
Zhang, Wenjun Zhang
- Abstract summary: Collaborative perception results in a trade-off between perception ability and communication costs.
We propose PragComm, a multi-agent collaborative perception system with two key components.
PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume.
- Score: 80.14322755297788
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Collaborative perception allows each agent to enhance its perceptual
abilities by exchanging messages with others. It inherently results in a
trade-off between perception ability and communication costs. Previous works
transmit complete full-frame high-dimensional feature maps among agents,
resulting in substantial communication costs. To promote communication
efficiency, we propose only transmitting the information needed for the
collaborator's downstream task. This pragmatic communication strategy focuses
on three key aspects: i) pragmatic message selection, which selects
task-critical parts from the complete data, resulting in spatially and
temporally sparse feature vectors; ii) pragmatic message representation, which
achieves pragmatic approximation of high-dimensional feature vectors with a
task-adaptive dictionary, enabling communicating with integer indices; iii)
pragmatic collaborator selection, which identifies beneficial collaborators,
pruning unnecessary communication links. Following this strategy, we first
formulate a mathematical optimization framework for the
perception-communication trade-off and then propose PragComm, a multi-agent
collaborative perception system with two key components: i) single-agent
detection and tracking and ii) pragmatic collaboration. The proposed PragComm
promotes pragmatic communication and adapts to a wide range of communication
conditions. We evaluate PragComm for both collaborative 3D object detection and
tracking tasks in both real-world, V2V4Real, and simulation datasets, OPV2V and
V2X-SIM2.0. PragComm consistently outperforms previous methods with more than
32.7K times lower communication volume on OPV2V. Code is available at
github.com/PhyllisH/PragComm.
Related papers
- Communication Learning in Multi-Agent Systems from Graph Modeling Perspective [62.13508281188895]
We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph.
We introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time.
arXiv Detail & Related papers (2024-11-01T05:56:51Z) - Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph.
Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
arXiv Detail & Related papers (2024-05-14T12:40:25Z) - Communication-Efficient Collaborative Perception via Information Filling with Codebook [48.087934650038044]
Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents.
To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection.
By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system.
arXiv Detail & Related papers (2024-05-08T11:12:37Z) - Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
We propose a novel agent-driven generative semantic communication framework based on reinforcement learning.
In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling.
The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
arXiv Detail & Related papers (2024-04-10T13:24:27Z) - PACE: A Pragmatic Agent for Enhancing Communication Efficiency Using
Large Language Models [29.016842120305892]
This paper proposes an image pragmatic communication framework based on a Pragmatic Agent for Communication Efficiency (PACE) using Large Language Models (LLM)
PACE sequentially performs semantic perception, intention resolution, and intention-oriented coding.
For experimental validation, this paper constructs an image pragmatic communication dataset along with corresponding evaluation standards.
arXiv Detail & Related papers (2024-01-30T06:55:17Z) - On the Role of Emergent Communication for Social Learning in Multi-Agent
Reinforcement Learning [0.0]
Social learning uses cues from experts to align heterogeneous policies, reduce sample complexity, and solve partially observable tasks.
This paper proposes an unsupervised method based on the information bottleneck to capture both referential complexity and task-specific utility.
arXiv Detail & Related papers (2023-02-28T03:23:27Z) - Learning Practical Communication Strategies in Cooperative Multi-Agent
Reinforcement Learning [5.539117319607963]
Communication in realistic wireless networks can be highly unreliable due to network conditions varying with agents' mobility.
We propose a framework to learn practical communication strategies by addressing three fundamental questions.
We show significant improvements in game performance, convergence speed and communication efficiency compared with state-of-the-art.
arXiv Detail & Related papers (2022-09-02T22:18:43Z) - Learning Individually Inferred Communication for Multi-Agent Cooperation [37.56115000150748]
We propose Individually Inferred Communication (I2C) to enable agents to learn a prior for agent-agent communication.
The prior knowledge is learned via causal inference and realized by a feed-forward neural network.
I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios.
arXiv Detail & Related papers (2020-06-11T14:07:57Z) - Emergence of Pragmatics from Referential Game between Theory of Mind
Agents [64.25696237463397]
We propose an algorithm, using which agents can spontaneously learn the ability to "read between lines" without any explicit hand-designed rules.
We integrate the theory of mind (ToM) in a cooperative multi-agent pedagogical situation and propose an adaptive reinforcement learning (RL) algorithm to develop a communication protocol.
arXiv Detail & Related papers (2020-01-21T19:37:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.