Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection

• URL: http://arxiv.org/abs/2409.20558v1
• Date: Mon, 30 Sep 2024 17:57:50 GMT
• Title: Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection
• Authors: Yubin Wang, Zhikang Zou, Xiaoqing Ye, Xiao Tan, Errui Ding, Cairong Zhao,
• Abstract summary: Uni$2$Det is a framework for unified and universal multi-dataset training on 3D detection. We introduce multi-stage prompting modules for multi-dataset 3D detection. Results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.
• Score: 64.08296187555095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Uni$^2$Det, a brand new framework for unified and universal multi-dataset training on 3D detection, enabling robust performance across diverse domains and generalization to unseen domains. Due to substantial disparities in data distribution and variations in taxonomy across diverse domains, training such a detector by simply merging datasets poses a significant challenge. Motivated by this observation, we introduce multi-stage prompting modules for multi-dataset 3D detection, which leverages prompts based on the characteristics of corresponding datasets to mitigate existing differences. This elegant design facilitates seamless plug-and-play integration within various advanced 3D detection frameworks in a unified manner, while also allowing straightforward adaptation for universal applicability across datasets. Experiments are conducted across multiple dataset consolidation scenarios involving KITTI, Waymo, and nuScenes, demonstrating that our Uni$^2$Det outperforms existing methods by a large margin in multi-dataset training. Notably, results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.

Related papers

• One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detection [71.78795573911512]
  We propose textbfOneDet3D, a universal one-for-all model that addresses 3D detection across different domains. We propose the domain-aware in scatter and context, guided by a routing mechanism, to address the data interference issue. The fully sparse structure and anchor-free head further accommodate point clouds with significant scale disparities.
  arXiv  Detail & Related papers  (2024-11-03T14:21:56Z)
• Multimodality Helps Few-Shot 3D Point Cloud Semantic Segmentation [61.91492500828508]
  Few-shot 3D point cloud segmentation (FS-PCS) aims at generalizing models to segment novel categories with minimal support samples. We introduce a cost-free multimodal FS-PCS setup, utilizing textual labels and the potentially available 2D image modality. We propose a simple yet effective Test-time Adaptive Cross-modal Seg (TACC) technique to mitigate training bias.
  arXiv  Detail & Related papers  (2024-10-29T19:28:41Z)
• Boosting 3D Object Detection with Semantic-Aware Multi-Branch Framework [44.44329455757931]
  In autonomous driving, LiDAR sensors are vital for acquiring 3D point clouds, providing reliable geometric information. Traditional sampling methods of preprocessing often ignore semantic features, leading to detail loss and ground point interference. We propose a multi-branch two-stage 3D object detection framework using a Semantic-aware Multi-branch Sampling (SMS) module and multi-view constraints.
  arXiv  Detail & Related papers  (2024-07-08T09:25:45Z)
• Multi-Space Alignments Towards Universal LiDAR Segmentation [50.992103482269016]
  M3Net is a one-of-a-kind framework for fulfilling multi-task, multi-dataset, multi-modality LiDAR segmentation. We first combine large-scale driving datasets acquired by different types of sensors from diverse scenes. We then conduct alignments in three spaces, namely data, feature, and label spaces, during the training.
  arXiv  Detail & Related papers  (2024-05-02T17:59:57Z)
• Multimodal 3D Object Detection on Unseen Domains [37.142470149311904]
  Domain adaptation approaches assume access to unannotated samples from the test distribution to address this problem. We propose CLIX$text3D$, a multimodal fusion and supervised contrastive learning framework for 3D object detection. We show that CLIX$text3D$ yields state-of-the-art domain generalization performance under multiple dataset shifts.
  arXiv  Detail & Related papers  (2024-04-17T21:47:45Z)
• Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training [44.790636524264]
  Point Prompt Training is a novel framework for multi-dataset synergistic learning in the context of 3D representation learning. It can overcome the negative transfer associated with synergistic learning and produce generalizable representations. It achieves state-of-the-art performance on each dataset using a single weight-shared model with supervised multi-dataset training.
  arXiv  Detail & Related papers  (2023-08-18T17:59:57Z)
• Uni3D: A Unified Baseline for Multi-dataset 3D Object Detection [34.2238222373818]
  Current 3D object detection models follow a single dataset-specific training and testing paradigm. In this paper, we study the task of training a unified 3D detector from multiple datasets. We present a Uni3D which leverages a simple data-level correction operation and a designed semantic-level coupling-and-recoupling module.
  arXiv  Detail & Related papers  (2023-03-13T05:54:13Z)
• MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation [104.48766162008815]
  We propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation. To design a framework that can take full advantage of multi-modality, each modality provides regularized self-supervisory signals to other modalities. Our regularized pseudo labels produce stable self-learning signals in numerous multi-modal test-time adaptation scenarios.
  arXiv  Detail & Related papers  (2022-04-27T02:28:12Z)
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN [117.80737222754306]
  We present a novel universal object detector called Universal-RCNN. We first generate a global semantic pool by integrating all high-level semantic representation of all the categories. An Intra-Domain Reasoning Module learns and propagates the sparse graph representation within one dataset guided by a spatial-aware GCN.
  arXiv  Detail & Related papers  (2020-02-18T07:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.