Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis

• URL: http://arxiv.org/abs/2410.20406v2
• Date: Sat, 02 Nov 2024 09:09:59 GMT
• Title: Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis
• Authors: Hongyu Sun, Qiuhong Ke, Yongcai Wang, Wang Chen, Kang Yang, Deying Li, Jianfei Cai,
• Abstract summary: Recent works demonstrate the performances of 3D point cloud recognition can be boosted remarkably by parameter-efficient prompt tuning. We present a comprehensive regulation framework that allows the learnable prompts to actively interact with the well-learned general knowledge in large 3D models. Surprisingly, our method not only realizes consistently increasing generalization ability but also enhances task-specific 3D recognition performances across various 3DDG benchmarks by a clear margin.
• Score: 39.85109385954641
• License:
• Abstract: This paper investigates the 3D domain generalization (3DDG) ability of large 3D models based on prevalent prompt learning. Recent works demonstrate the performances of 3D point cloud recognition can be boosted remarkably by parameter-efficient prompt tuning. However, we observe that the improvement on downstream tasks comes at the expense of a severe drop in 3D domain generalization. To resolve this challenge, we present a comprehensive regulation framework that allows the learnable prompts to actively interact with the well-learned general knowledge in large 3D models to maintain good generalization. Specifically, the proposed framework imposes multiple explicit constraints on the prompt learning trajectory by maximizing the mutual agreement between task-specific predictions and task-agnostic knowledge. We design the regulation framework as a plug-and-play module to embed into existing representative large 3D models. Surprisingly, our method not only realizes consistently increasing generalization ability but also enhances task-specific 3D recognition performances across various 3DDG benchmarks by a clear margin. Considering the lack of study and evaluation on 3DDG, we also create three new benchmarks, namely base-to-new, cross-dataset and few-shot generalization benchmarks, to enrich the field and inspire future research. Code and benchmarks are available at \url{https://github.com/auniquesun/Point-PRC}.

Related papers

• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Point-DETR3D: Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection [32.86369670395974]
  We introduce Point-DETR3D, a teacher-student framework for weakly semi-supervised 3D detection. With only 5% of labeled data, Point-DETR3D achieves over 90% performance of its fully supervised counterpart.
  arXiv  Detail & Related papers  (2024-03-22T16:11:29Z)
• A Survey on 3D Gaussian Splatting [51.96747208581275]
  3D Gaussian splatting (GS) has emerged as a transformative technique in the realm of explicit radiance field and computer graphics. We provide the first systematic overview of the recent developments and critical contributions in the domain of 3D GS. By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond.
  arXiv  Detail & Related papers  (2024-01-08T13:42:59Z)
• A Review and A Robust Framework of Data-Efficient 3D Scene Parsing with Traditional/Learned 3D Descriptors [10.497309421830671]
  Existing state-of-the-art 3D point cloud understanding methods merely perform well in a fully supervised manner. This work presents a general and simple framework to tackle point cloud understanding when labels are limited.
  arXiv  Detail & Related papers  (2023-12-03T02:51:54Z)
• Multi-View Fusion and Distillation for Subgrade Distresses Detection based on 3D-GPR [19.49863426864145]
  We introduce a novel methodology for the subgrade distress detection task by leveraging the multi-view information from 3D-GPR data. We develop a novel textbfMulti-textbfView textbfVusion and textbfDistillation framework, textbfGPR-MVFD, specifically designed to optimally utilize the multi-view GPR dataset.
  arXiv  Detail & Related papers  (2023-08-09T08:06:28Z)
• 3D Spatial Recognition without Spatially Labeled 3D [127.6254240158249]
  We introduce WyPR, a Weakly-supervised framework for Point cloud Recognition. We show that WyPR can detect and segment objects in point cloud data without access to any spatial labels at training time.
  arXiv  Detail & Related papers  (2021-05-13T17:58:07Z)
• Gram Regularization for Multi-view 3D Shape Retrieval [3.655021726150368]
  We propose a novel regularization term called Gram regularization. By forcing the variance between weight kernels to be large, the regularizer can help to extract discriminative features. The proposed Gram regularization is data independent and can converge stably and quickly without bells and whistles.
  arXiv  Detail & Related papers  (2020-11-16T05:37:24Z)
• Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection [102.62963605429508]
  Point cloud semantic segmentation plays an essential role in autonomous driving. Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes. We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
  arXiv  Detail & Related papers  (2020-09-22T14:17:40Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.