HeteroPROPMT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework

• URL: http://arxiv.org/abs/2607.26283v1
• Date: Tue, 28 Jul 2026 21:24:43 GMT
• Title: HeteroPROPMT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework
• Abstract summary: Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results.<n>Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreters.<n>We propose HeteroPROMPT, a real-time and privacy-preserving framework for heterogeneous collaborative perception.
• Score: 4.904389322133549
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results. In real-world deployments, however, collaborating vehicles often use heterogeneous sensors, perception models, datasets, and training domains, creating feature-space shifts that degrade downstream fusion and detection. Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreters. These methods scale poorly to newly joining agents and often require access to proprietary metadata, raising privacy concerns. We propose HeteroPROMPT, a real-time and privacy-preserving framework for heterogeneous collaborative perception. HeteroPROMPT rapidly aligns each heterogeneous agent's features with an ego-centric unified feature space through modular prompts and lightweight learning-based tuning, while keeping agent encoders and the collaborative fusion and detection stacks frozen. Its visual prompt-based training and inference modulate Bird's Eye View (BEV) features across channels and spatial locations with low computational overhead. For metadata-free deployment, an autoencoder learns a compact unified representation and extracts modality cues from shared features, enabling real-time modality classification and routing to the appropriate HeteroPROMPT modules without exposing proprietary agent information. Experiments on the OPV2V-H and V2XSet datasets show that HeteroPROMPT improves Average Precision over state-of-the-art heterogeneous CP methods while using orders of magnitude fewer trainable parameters. This offers a scalable and practical CP solution. The proposed modality classifier also predicts the joining agent's modality from compact features with greater than 99.99 percent accuracy during deployment. Code will be available at https://github.com/arminmaleki007/HeteroPROMPT.

Related papers

• INTACT: Ego-Guided Typed Sparse Evidence Retrieval for Heterogeneous Collaborative Perception [50.64511548643398]
  We propose an ego-guided typed sparse evidence retrieval framework for heterogeneous perception.<n>IntACT lets the ego vehicle issue typed evidence queries that express suspected objects and evidence-deficient regions.<n>Experiments on simulated and real-world heterogeneous perception benchmarks validate the effectiveness and deployability of INTACT.
  arXiv  Detail & Related papers  (2026-06-03T04:34:53Z)
• FedRio: Personalized Federated Social Bot Detection via Cooperative Reinforced Contrastive Adversarial Distillation [52.56850762009136]
  Social bot detection is critical to the stability and security of online social platforms.<n>Current state-of-the-art bot detection models are largely developed in isolation.<n>We propose FedRio as a cross-platform and cross-model detection framework.
  arXiv  Detail & Related papers  (2026-04-12T15:13:41Z)
• GT-Space: Enhancing Heterogeneous Collaborative Perception with Ground Truth Feature Space [30.468574984070894]
  In autonomous driving, multi-agent collaborative perception enhances sensing capabilities by enabling agents to share perceptual data.<n>Existing approaches often require retraining encoders or designing interpreter modules for pairwise feature alignment.<n>We propose em GT-Space, a flexible and scalable collaborative perception framework for heterogeneous agents.
  arXiv  Detail & Related papers  (2026-03-13T14:39:40Z)
• Faster-HEAL: An Efficient and Privacy-Preserving Collaborative Perception Framework for Heterogeneous Autonomous Vehicles [4.904389322133549]
  Collaborative perception is a promising paradigm for improving situational awareness in autonomous vehicles.<n>We propose Faster-HEAL, a framework that fine-tunes a low-rank visual prompt to align heterogeneous features with a unified feature space.<n>This approach reduces the trainable parameters by 94%, enabling efficient adaptation to new agents without retraining large models.
  arXiv  Detail & Related papers  (2026-03-07T19:18:25Z)
• Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism [14.40993352402385]
  We present a novel Generative Communication mechanism (GenComm) that facilitates seamless perception across heterogeneous multi-agent systems.<n>Experiments conducted on the OPV2V-H, DAIR-V2X and V2X-Real datasets demonstrate that GenComm outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-22T14:15:20Z)
• Perfectly-Private Analog Secure Aggregation in Federated Learning [51.61616734974475]
  In federated learning, multiple parties train models locally and share their parameters with a central server, which aggregates them to update a global model.<n>In this paper, a novel secure parameter aggregation method is proposed that employs the torus rather than a finite field.
  arXiv  Detail & Related papers  (2025-09-10T15:22:40Z)
• STAMP: Scalable Task And Model-agnostic Collaborative Perception [24.890993164334766]
  STAMP is a task- and model-agnostic, collaborative perception pipeline for heterogeneous agents.<n>It minimizes computational overhead, enhances scalability, and preserves model security.<n>As a first-of-its-kind framework, STAMP aims to advance research in scalable and secure mobility systems towards Level 5 autonomy.
  arXiv  Detail & Related papers  (2025-01-24T16:27:28Z)
• Self-Localized Collaborative Perception [49.86110931859302]
  We propose$mathttCoBEVGlue$, a novel self-localized collaborative perception system. $mathttCoBEVGlue$ is a novel spatial alignment module, which provides the relative poses between agents. $mathttCoBEVGlue$ achieves state-of-the-art detection performance under arbitrary localization noises and attacks.
  arXiv  Detail & Related papers  (2024-06-18T15:26:54Z)
• Robust Collaborative Perception without External Localization and Clock Devices [52.32342059286222]
  A consistent spatial-temporal coordination across multiple agents is fundamental for collaborative perception. Traditional methods depend on external devices to provide localization and clock signals. We propose a novel approach: aligning by recognizing the inherent geometric patterns within the perceptual data of various agents.
  arXiv  Detail & Related papers  (2024-05-05T15:20:36Z)
• Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding [121.08841110022607]
  Existing agent-centric methods have demonstrated outstanding performance on public benchmarks. We introduce the K-nearest neighbor attention with relative pose encoding (KNARPE), a novel attention mechanism allowing the pairwise-relative representation to be used by Transformers. By sharing contexts among agents and reusing the unchanged contexts, our approach is as efficient as scene-centric methods, while performing on par with state-of-the-art agent-centric methods.
  arXiv  Detail & Related papers  (2023-10-19T17:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.