One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detection

• URL: http://arxiv.org/abs/2411.01584v1
• Date: Sun, 03 Nov 2024 14:21:56 GMT
• Title: One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detection
• Authors: Zhenyu Wang, Yali Li, Hengshuang Zhao, Shengjin Wang,
• Abstract summary: We propose textbfOneDet3D, a universal one-for-all model that addresses 3D detection across different domains. We propose the domain-aware in scatter and context, guided by a routing mechanism, to address the data interference issue. The fully sparse structure and anchor-free head further accommodate point clouds with significant scale disparities.
• Score: 71.78795573911512
• License:
• Abstract: The current trend in computer vision is to utilize one universal model to address all various tasks. Achieving such a universal model inevitably requires incorporating multi-domain data for joint training to learn across multiple problem scenarios. In point cloud based 3D object detection, however, such multi-domain joint training is highly challenging, because large domain gaps among point clouds from different datasets lead to the severe domain-interference problem. In this paper, we propose \textbf{OneDet3D}, a universal one-for-all model that addresses 3D detection across different domains, including diverse indoor and outdoor scenes, within the \emph{same} framework and only \emph{one} set of parameters. We propose the domain-aware partitioning in scatter and context, guided by a routing mechanism, to address the data interference issue, and further incorporate the text modality for a language-guided classification to unify the multi-dataset label spaces and mitigate the category interference issue. The fully sparse structure and anchor-free head further accommodate point clouds with significant scale disparities. Extensive experiments demonstrate the strong universal ability of OneDet3D to utilize only one trained model for addressing almost all 3D object detection tasks.

Related papers

• Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection [64.08296187555095]
  Uni$2$Det is a framework for unified and universal multi-dataset training on 3D detection. We introduce multi-stage prompting modules for multi-dataset 3D detection. Results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.
  arXiv  Detail & Related papers  (2024-09-30T17:57:50Z)
• PointDGMamba: Domain Generalization of Point Cloud Classification via Generalized State Space Model [77.00221501105788]
  Domain Generalization (DG) has been recently explored to improve the generalizability of point cloud classification (PCC) models toward unseen domains. We present the first work that studies the generalizability of state space models (SSMs) in DG PCC. We propose a novel framework, PointDGMamba, that excels in strong generalizability toward unseen domains.
  arXiv  Detail & Related papers  (2024-08-24T12:53:48Z)
• Revisiting Cross-Domain Problem for LiDAR-based 3D Object Detection [5.149095033945412]
  We deeply analyze the cross-domain performance of the state-of-the-art models. We observe that most models will overfit the training domains and it is challenging to adapt them to other domains directly. We propose additional evaluation metrics -- the side-view and front-view AP -- to better analyze the core issues of the methods' heavy drops in accuracy levels.
  arXiv  Detail & Related papers  (2024-08-22T19:52:44Z)
• Multimodal 3D Object Detection on Unseen Domains [37.142470149311904]
  Domain adaptation approaches assume access to unannotated samples from the test distribution to address this problem. We propose CLIX$text3D$, a multimodal fusion and supervised contrastive learning framework for 3D object detection. We show that CLIX$text3D$ yields state-of-the-art domain generalization performance under multiple dataset shifts.
  arXiv  Detail & Related papers  (2024-04-17T21:47:45Z)
• Learning to Adapt SAM for Segmenting Cross-domain Point Clouds [25.66755977728181]
  Unsupervised domain adaptation (UDA) in 3D segmentation tasks presents a formidable challenge. We propose an innovative hybrid feature augmentation methodology, which significantly enhances the alignment between the 3D feature space and SAM's feature space. Our method is evaluated on many widely-recognized datasets and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-13T02:28:40Z)
• SUG: Single-dataset Unified Generalization for 3D Point Cloud Classification [44.27324696068285]
  We propose a Single-dataset Unified Generalization (SUG) framework to alleviate the unforeseen domain differences faced by a well-trained source model. Specifically, we first design a Multi-grained Sub-domain Alignment (MSA) method, which can constrain the learned representations to be domain-agnostic and discriminative. Then, a Sample-level Domain-aware Attention (SDA) strategy is presented, which can selectively enhance easy-to-adapt samples from different sub-domains.
  arXiv  Detail & Related papers  (2023-05-16T04:36:04Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency [90.71745178767203]
  Deep learning-based 3D object detection has achieved unprecedented success with the advent of large-scale autonomous driving datasets. Existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world. We study a more realistic setting, unsupervised 3D domain adaptive detection, which only utilizes source domain annotations.
  arXiv  Detail & Related papers  (2021-07-23T17:19:23Z)
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN [117.80737222754306]
  We present a novel universal object detector called Universal-RCNN. We first generate a global semantic pool by integrating all high-level semantic representation of all the categories. An Intra-Domain Reasoning Module learns and propagates the sparse graph representation within one dataset guided by a spatial-aware GCN.
  arXiv  Detail & Related papers  (2020-02-18T07:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.