GT-Space: Enhancing Heterogeneous Collaborative Perception with Ground Truth Feature Space

• URL: http://arxiv.org/abs/2603.19308v1
• Date: Fri, 13 Mar 2026 14:39:40 GMT
• Title: GT-Space: Enhancing Heterogeneous Collaborative Perception with Ground Truth Feature Space
• Abstract summary: In autonomous driving, multi-agent collaborative perception enhances sensing capabilities by enabling agents to share perceptual data.<n>Existing approaches often require retraining encoders or designing interpreter modules for pairwise feature alignment.<n>We propose em GT-Space, a flexible and scalable collaborative perception framework for heterogeneous agents.
• Score: 30.468574984070894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In autonomous driving, multi-agent collaborative perception enhances sensing capabilities by enabling agents to share perceptual data. A key challenge lies in handling {\em heterogeneous} features from agents equipped with different sensing modalities or model architectures, which complicates data fusion. Existing approaches often require retraining encoders or designing interpreter modules for pairwise feature alignment, but these solutions are not scalable in practice. To address this, we propose {\em GT-Space}, a flexible and scalable collaborative perception framework for heterogeneous agents. GT-Space constructs a common feature space from ground-truth labels, providing a unified reference for feature alignment. With this shared space, agents only need a single adapter module to project their features, eliminating the need for pairwise interactions with other agents. Furthermore, we design a fusion network trained with contrastive losses across diverse modality combinations. Extensive experiments on simulation datasets (OPV2V and V2XSet) and a real-world dataset (RCooper) demonstrate that GT-Space consistently outperforms baselines in detection accuracy while delivering robust performance. Our code will be released at https://github.com/KingScar/GT-Space.

Related papers

• What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents [81.23160457293714]
  LLM agents increasingly rely on generated interaction data to learn how to interact with external environments.<n>Existing work spans many agent domains, but domain-centered organization and heterogeneous evaluation often obscure common generation mechanisms.<n>This work develops a two-level framework for the field.
  arXiv  Detail & Related papers  (2026-08-27T15:43:50Z)
• HeteroPROPMT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework [4.904389322133549]
  Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results.<n>Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreters.<n>We propose HeteroPROMPT, a real-time and privacy-preserving framework for heterogeneous collaborative perception.
  arXiv  Detail & Related papers  (2026-07-28T21:24:43Z)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning [99.49739831030388]
  We study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.<n>We propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface.
  arXiv  Detail & Related papers  (2026-06-11T17:59:36Z)
• INTACT: Ego-Guided Typed Sparse Evidence Retrieval for Heterogeneous Collaborative Perception [50.64511548643398]
  We propose an ego-guided typed sparse evidence retrieval framework for heterogeneous perception.<n>IntACT lets the ego vehicle issue typed evidence queries that express suspected objects and evidence-deficient regions.<n>Experiments on simulated and real-world heterogeneous perception benchmarks validate the effectiveness and deployability of INTACT.
  arXiv  Detail & Related papers  (2026-06-03T04:34:53Z)
• Linking Modality Isolation in Heterogeneous Collaborative Perception [41.68601421239159]
  We propose CodeAlign, a framework that smoothly aligns modalities via cross-modal featurecode-feature(FCF) translation.<n>CodeAlign learns FCF translations that map features to the corresponding codes of other modalities, which are then decoded back into features in the target code space.<n> Experiments show that, when integrating three modalities, CodeAlign requires only 8% of the training parameters of prior alignment methods, reduces communication load by 1024x, and state-of-the-art perception performance on both OPV2V and DAIRV2X dataset achieves.
  arXiv  Detail & Related papers  (2026-02-28T12:09:08Z)
• InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs [72.5651722107621]
  InterAgent is an end-to-end framework for text-driven physics-based multi-agent humanoid control.<n>We introduce an autoregressive diffusion transformer equipped with multi-stream blocks, which decouples proprioception, exteroception, and action to cross-modal interference.<n>We also propose a novel interaction graph exteroception representation that explicitly captures fine-grained joint-to-joint spatial dependencies.
  arXiv  Detail & Related papers  (2025-12-08T10:46:01Z)
• Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism [14.40993352402385]
  We present a novel Generative Communication mechanism (GenComm) that facilitates seamless perception across heterogeneous multi-agent systems.<n>Experiments conducted on the OPV2V-H, DAIR-V2X and V2X-Real datasets demonstrate that GenComm outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-22T14:15:20Z)
• Spatial-Temporal-Spectral Unified Modeling for Remote Sensing Dense Prediction [20.1863553357121]
  Current deep learning architectures for remote sensing are fundamentally rigid.<n>We introduce the Spatial-Temporal-Spectral Unified Network (STSUN) for unified modeling.<n> STSUN can adapt to input and output data with arbitrary spatial sizes, temporal lengths, and spectral bands.<n>It unifies various dense prediction tasks and diverse semantic class predictions.
  arXiv  Detail & Related papers  (2025-05-18T07:39:17Z)
• STAMP: Scalable Task And Model-agnostic Collaborative Perception [24.890993164334766]
  STAMP is a task- and model-agnostic, collaborative perception pipeline for heterogeneous agents.<n>It minimizes computational overhead, enhances scalability, and preserves model security.<n>As a first-of-its-kind framework, STAMP aims to advance research in scalable and secure mobility systems towards Level 5 autonomy.
  arXiv  Detail & Related papers  (2025-01-24T16:27:28Z)
• Self-Localized Collaborative Perception [49.86110931859302]
  We propose$mathttCoBEVGlue$, a novel self-localized collaborative perception system. $mathttCoBEVGlue$ is a novel spatial alignment module, which provides the relative poses between agents. $mathttCoBEVGlue$ achieves state-of-the-art detection performance under arbitrary localization noises and attacks.
  arXiv  Detail & Related papers  (2024-06-18T15:26:54Z)
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.<n>Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction.<n>We benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)
• Scalable Multi-agent Covering Option Discovery based on Kronecker Graphs [49.71319907864573]
  In this paper, we propose multi-agent skill discovery which enables the ease of decomposition. Our key idea is to approximate the joint state space as a Kronecker graph, based on which we can directly estimate its Fiedler vector. Considering that directly computing the Laplacian spectrum is intractable for tasks with infinite-scale state spaces, we further propose a deep learning extension of our method.
  arXiv  Detail & Related papers  (2023-07-21T14:53:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.