UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting

• URL: http://arxiv.org/abs/2506.09952v1
• Date: Wed, 11 Jun 2025 17:23:21 GMT
• Title: UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting
• Authors: Ziyi Wang, Yanran Zhang, Jie Zhou, Jiwen Lu,
• Abstract summary: No existing pre-training method is equally effective for both object- and scene-level point clouds.<n>We introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture.
• Score: 64.31900521467362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The scale diversity of point cloud data presents significant challenges in developing unified representation learning techniques for 3D vision. Currently, there are few unified 3D models, and no existing pre-training method is equally effective for both object- and scene-level point clouds. In this paper, we introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture. Our approach predicts Gaussian primitives as the pre-training task and employs differentiable Gaussian splatting to render images, enabling precise pixel-level supervision and end-to-end optimization. To further regulate the complexity of the pre-training task and direct the model's focus toward geometric structures, we integrate 2D features from pre-trained image models to incorporate well-established texture knowledge. We validate the universal effectiveness of our proposed method through extensive experiments across a variety of object- and scene-level tasks, using diverse point cloud models as backbones. Code is available at https://github.com/wangzy22/UniPre3D.

Related papers

• 3D Point Cloud Generation via Autoregressive Up-sampling [60.05226063558296]
  We introduce a pioneering autoregressive generative model for 3D point cloud generation.<n>Inspired by visual autoregressive modeling, we conceptualize point cloud generation as an autoregressive up-sampling process.<n>PointARU progressively refines 3D point clouds from coarse to fine scales.
  arXiv  Detail & Related papers  (2025-03-11T16:30:45Z)
• CLAP: Unsupervised 3D Representation Learning for Fusion 3D Perception via Curvature Sampling and Prototype Learning [43.7594705101778]
  We propose a joint unsupervised differentiable-rendering-based pre-training method for images and point clouds, termed CLAP.<n>Our method overcomes the computational hurdle by Curvature Sampling to select the more informative points/pixels for pre-training.<n>Experiments show that CLAP achieves up to 100% more performance gain as compared to previous SOTA pre-training methods.
  arXiv  Detail & Related papers  (2024-12-04T06:26:12Z)
• Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation [67.07112533415116]
  We present a novel framework that adapts various foundational models for the 3D point cloud segmentation task. Our approach involves making initial predictions of 2D semantic masks using different large vision models. To generate robust 3D semantic pseudo labels, we introduce a semantic label fusion strategy that effectively combines all the results via voting.
  arXiv  Detail & Related papers  (2023-11-03T15:41:15Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [111.16358607889609]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation.<n>For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models [97.58685709663287]
  generative pre-training can boost the performance of fundamental models in 2D vision. In 3D vision, the over-reliance on Transformer-based backbones and the unordered nature of point clouds have restricted the further development of generative pre-training. We propose a novel 3D-to-2D generative pre-training method that is adaptable to any point cloud model.
  arXiv  Detail & Related papers  (2023-07-27T16:07:03Z)
• Flow-based GAN for 3D Point Cloud Generation from a Single Image [16.04710129379503]
  We introduce a hybrid explicit-implicit generative modeling scheme, which inherits the flow-based explicit generative models for sampling point clouds with arbitrary resolutions. We evaluate on the large-scale synthetic dataset ShapeNet, with the experimental results demonstrating the superior performance of the proposed method.
  arXiv  Detail & Related papers  (2022-10-08T17:58:20Z)
• P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting [94.11915008006483]
  We propose a novel Point-to-Pixel prompting for point cloud analysis. Our method attains 89.3% accuracy on the hardest setting of ScanObjectNN. Our framework also exhibits very competitive performance on ModelNet classification and ShapeNet Part Code.
  arXiv  Detail & Related papers  (2022-08-04T17:59:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.