Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sources

• URL: http://arxiv.org/abs/2402.04273v2
• Date: Tue, 20 Feb 2024 20:46:28 GMT
• Title: Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sources
• Authors: Jinlong Li, Baolu Li, Xinyu Liu, Runsheng Xu, Jiaqi Ma, Hongkai Yu
• Abstract summary: The diverse agents in multi-agent perception systems may be from different companies. Each company might use the identical classic neural network architecture based encoder for feature extraction. The data silos by the above Distribution Gap could result in a significant performance decline in multi-agent perception. We introduce the Feature Distribution-aware Aggregation (FDA) framework for cross-domain learning to mitigate the above Distribution Gap in multi-agent perception.
• Score: 39.22864188785987
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The diverse agents in multi-agent perception systems may be from different companies. Each company might use the identical classic neural network architecture based encoder for feature extraction. However, the data source to train the various agents is independent and private in each company, leading to the Distribution Gap of different private data for training distinct agents in multi-agent perception system. The data silos by the above Distribution Gap could result in a significant performance decline in multi-agent perception. In this paper, we thoroughly examine the impact of the distribution gap on existing multi-agent perception systems. To break the data silos, we introduce the Feature Distribution-aware Aggregation (FDA) framework for cross-domain learning to mitigate the above Distribution Gap in multi-agent perception. FDA comprises two key components: Learnable Feature Compensation Module and Distribution-aware Statistical Consistency Module, both aimed at enhancing intermediate features to minimize the distribution gap among multi-agent features. Intensive experiments on the public OPV2V and V2XSet datasets underscore FDA's effectiveness in point cloud-based 3D object detection, presenting it as an invaluable augmentation to existing multi-agent perception systems.

Related papers

• M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training [5.128670847334003]
  multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. The distribution of data in the training set differs from the distribution of data in the real world, the deployment environment. We propose M3BAT, an unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training.
  arXiv  Detail & Related papers  (2024-04-26T13:09:35Z)
• AdapTraj: A Multi-Source Domain Generalization Framework for Multi-Agent Trajectory Prediction [29.489714129095862]
  Multi-agent trajectory prediction is a critical task in modeling complex interactions of objects in dynamic systems. We propose AdapTraj, a multi-source domain generalization framework specifically tailored for multi-agent trajectory prediction. AdapTraj consistently outperforms other baselines by a substantial margin.
  arXiv  Detail & Related papers  (2023-12-22T02:49:56Z)
• FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clients [32.59184269562571]
  We propose a multi-modal collaborative diffusion federated learning framework called FedDiff. Our framework establishes a dual-branch diffusion model feature extraction setup, where the two modal data are inputted into separate branches of the encoder. Considering the challenge of private and efficient communication between multiple clients, we embed the diffusion model into the federated learning communication structure.
  arXiv  Detail & Related papers  (2023-11-16T02:29:37Z)
• Scalable Multi-agent Covering Option Discovery based on Kronecker Graphs [49.71319907864573]
  In this paper, we propose multi-agent skill discovery which enables the ease of decomposition. Our key idea is to approximate the joint state space as a Kronecker graph, based on which we can directly estimate its Fiedler vector. Considering that directly computing the Laplacian spectrum is intractable for tasks with infinite-scale state spaces, we further propose a deep learning extension of our method.
  arXiv  Detail & Related papers  (2023-07-21T14:53:12Z)
• Source-free Domain Adaptation Requires Penalized Diversity [60.04618512479438]
  Source-free domain adaptation (SFDA) was introduced to address knowledge transfer between different domains in the absence of source data. In unsupervised SFDA, the diversity is limited to learning a single hypothesis on the source or learning multiple hypotheses with a shared feature extractor. We propose a novel unsupervised SFDA algorithm that promotes representational diversity through the use of separate feature extractors.
  arXiv  Detail & Related papers  (2023-04-06T00:20:19Z)
• Learning From Good Trajectories in Offline Multi-Agent Reinforcement Learning [98.07495732562654]
  offline multi-agent reinforcement learning (MARL) aims to learn effective multi-agent policies from pre-collected datasets. One agent learned by offline MARL often inherits this random policy, jeopardizing the performance of the entire team. We propose a novel framework called Shared Individual Trajectories (SIT) to address this problem.
  arXiv  Detail & Related papers  (2022-11-28T18:11:26Z)
• Bridging the Domain Gap for Multi-Agent Perception [19.724227909352976]
  We propose the first lightweight framework to bridge domain gaps for multi-agent perception. Our framework consists of a learnable feature resizer to align features in multiple dimensions and a sparse cross-domain transformer for domain adaption. Our method can effectively bridge the gap for features from different domains and outperform other baseline methods significantly by at least 8% for point-cloud-based 3D object detection.
  arXiv  Detail & Related papers  (2022-10-16T05:29:21Z)
• Adversarial Dual Distinct Classifiers for Unsupervised Domain Adaptation [67.83872616307008]
  Unversarial Domain adaptation (UDA) attempts to recognize the unlabeled target samples by building a learning model from a differently-distributed labeled source domain. In this paper, we propose a novel Adrial Dual Distincts Network (AD$2$CN) to align the source and target domain data distribution simultaneously with matching task-specific category boundaries. To be specific, a domain-invariant feature generator is exploited to embed the source and target data into a latent common space with the guidance of discriminative cross-domain alignment.
  arXiv  Detail & Related papers  (2020-08-27T01:29:10Z)
• Brainstorming Generative Adversarial Networks (BGANs): Towards Multi-Agent Generative Models with Distributed Private Datasets [70.62568022925971]
  generative adversarial networks (GANs) must be fed by large datasets that adequately represent the data space. In many scenarios, the available datasets may be limited and distributed across multiple agents, each of which is seeking to learn the distribution of the data on its own. In this paper, a novel brainstorming GAN (BGAN) architecture is proposed using which multiple agents can generate real-like data samples while operating in a fully distributed manner.
  arXiv  Detail & Related papers  (2020-02-02T02:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.