QUEST: Query Stream for Practical Cooperative Perception

• URL: http://arxiv.org/abs/2308.01804v3
• Date: Wed, 22 May 2024 09:08:16 GMT
• Title: QUEST: Query Stream for Practical Cooperative Perception
• Authors: Siqi Fan, Haibao Yu, Wenxian Yang, Jirui Yuan, Zaiqing Nie,
• Abstract summary: We propose the concept of query cooperation to enable interpretable instance-level flexible feature interaction. The cross-agent queries are interacted via fusion for co-aware instances and complementation for individual unaware instances.
• Score: 5.750142092931156
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cooperative perception can effectively enhance individual perception performance by providing additional viewpoint and expanding the sensing field. Existing cooperation paradigms are either interpretable (result cooperation) or flexible (feature cooperation). In this paper, we propose the concept of query cooperation to enable interpretable instance-level flexible feature interaction. To specifically explain the concept, we propose a cooperative perception framework, termed QUEST, which let query stream flow among agents. The cross-agent queries are interacted via fusion for co-aware instances and complementation for individual unaware instances. Taking camera-based vehicle-infrastructure perception as a typical practical application scene, the experimental results on the real-world dataset, DAIR-V2X-Seq, demonstrate the effectiveness of QUEST and further reveal the advantage of the query cooperation paradigm on transmission flexibility and robustness to packet dropout. We hope our work can further facilitate the cross-agent representation interaction for better cooperative perception in practice.

Related papers

• Estimate collective cooperativeness of driving agents in mixed traffic flow [21.67640928933297]
  Cooperation is a ubiquitous phenomenon in many natural, social, and engineered systems that contain multiple agents. We propose a unified conceptual framework to estimate collective cooperativeness of driving agents. Our case study indicates the existence of collective cooperativeness between human-driven passenger cars and trucks in real-world traffic.
  arXiv  Detail & Related papers  (2024-07-31T00:15:54Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learning [10.932974027102619]
  This study introduces a computational framework based on multi-agent reinforcement learning in the spatial Prisoner's Dilemma game. By modelling each agent using two distinct Q-networks, we disentangle the coevolutionary dynamics between cooperation and interaction.
  arXiv  Detail & Related papers  (2024-05-04T12:42:55Z)
• What Makes Good Collaborative Views? Contrastive Mutual Information Maximization for Multi-Agent Perception [52.41695608928129]
  Multi-agent perception (MAP) allows autonomous systems to understand complex environments by interpreting data from multiple sources. This paper investigates intermediate collaboration for MAP with a specific focus on exploring "good" properties of collaborative view. We propose a novel framework named CMiMC for intermediate collaboration.
  arXiv  Detail & Related papers  (2024-03-15T07:18:55Z)
• Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
  Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server. We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
  arXiv  Detail & Related papers  (2024-03-11T09:21:11Z)
• Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction [10.1305370182537]
  This paper introduces a novel framework, CooperKGC, for knowledge graph construction. CooperKGC establishes a collaborative processing network, assembling a KGC collaboration team capable of concurrently addressing entity, relation, and event extraction tasks. Our experiments unequivocally demonstrate that fostering collaboration and information interaction among diverse agents within CooperKGC yields superior results compared to individual cognitive processes operating in isolation.
  arXiv  Detail & Related papers  (2023-12-05T07:27:08Z)
• CORE: Cooperative Reconstruction for Multi-Agent Perception [24.306731432524227]
  CORE is a conceptually simple, effective and communication-efficient model for multi-agent cooperative perception. It addresses the task from a novel perspective of cooperative reconstruction, based on two key insights. We validate CORE on OPV2V, a large-scale multi-agent percetion dataset.
  arXiv  Detail & Related papers  (2023-07-21T11:50:05Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement [80.94378602238432]
  We propose an efficient structure named Correspondence Efficient Transformer (ECO-TR) by finding correspondences in a coarse-to-fine manner. To achieve this, multiple transformer blocks are stage-wisely connected to gradually refine the predicted coordinates. Experiments on various sparse and dense matching tasks demonstrate the superiority of our method in both efficiency and effectiveness against existing state-of-the-arts.
  arXiv  Detail & Related papers  (2022-09-25T13:05:33Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)
• Sequential Cooperative Bayesian Inference [16.538512182336827]
  Cooperation implies that the agent selecting the data, and the agent learning from the data, have the same goal, that the learner infer the intended hypothesis. Recent models in human and machine learning have demonstrated the possibility of cooperation.
  arXiv  Detail & Related papers  (2020-02-13T18:48:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.