DC3DO: Diffusion Classifier for 3D Objects

• URL: http://arxiv.org/abs/2408.06693v1
• Date: Tue, 13 Aug 2024 07:35:56 GMT
• Title: DC3DO: Diffusion Classifier for 3D Objects
• Authors: Nursena Koprucu, Meher Shashwat Nigam, Shicheng Xu, Biruk Abere, Gabriele Dominici, Andrew Rodriguez, Sharvaree Vadgam, Berfin Inal, Alberto Tono,
• Abstract summary: Inspired by Geoffrey Hinton emphasis on generative modeling, we explore the use of 3D diffusion models for object classification. Our approach, the Diffusion for 3D Objects (DC3DO), enables zero-shot classification of 3D shapes without additional training.
• Score: 3.265023962374139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inspired by Geoffrey Hinton emphasis on generative modeling, To recognize shapes, first learn to generate them, we explore the use of 3D diffusion models for object classification. Leveraging the density estimates from these models, our approach, the Diffusion Classifier for 3D Objects (DC3DO), enables zero-shot classification of 3D shapes without additional training. On average, our method achieves a 12.5 percent improvement compared to its multiview counterparts, demonstrating superior multimodal reasoning over discriminative approaches. DC3DO employs a class-conditional diffusion model trained on ShapeNet, and we run inferences on point clouds of chairs and cars. This work highlights the potential of generative models in 3D object classification.

Related papers

• DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data [50.164670363633704]
  We present DIRECT-3D, a diffusion-based 3D generative model for creating high-quality 3D assets from text prompts. Our model is directly trained on extensive noisy and unaligned in-the-wild' 3D assets. We achieve state-of-the-art performance in both single-class generation and text-to-3D generation.
  arXiv  Detail & Related papers  (2024-06-06T17:58:15Z)
• SC-Diff: 3D Shape Completion with Latent Diffusion Models [4.913210912019975]
  This paper introduces a 3D shape completion approach using a 3D latent diffusion model optimized for completing shapes. Our method combines image-based conditioning through cross-attention and spatial conditioning through the integration of 3D features from captured partial scans.
  arXiv  Detail & Related papers  (2024-03-19T06:01:11Z)
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance [76.7746870349809]
  We present ComboVerse, a 3D generation framework that produces high-quality 3D assets with complex compositions by learning to combine multiple models. Our proposed framework emphasizes spatial alignment of objects, compared with standard score distillation sampling.
  arXiv  Detail & Related papers  (2024-03-19T03:39:43Z)
• Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection [77.23918785277404]
  We present Diffusion-SS3D, a new perspective of enhancing the quality of pseudo-labels via the diffusion model for semi-supervised 3D object detection. Specifically, we include noises to produce corrupted 3D object size and class label, distributions, and then utilize the diffusion model as a denoising process to obtain bounding box outputs. We conduct experiments on the ScanNet and SUN RGB-D benchmark datasets to demonstrate that our approach achieves state-of-the-art performance against existing methods.
  arXiv  Detail & Related papers  (2023-12-05T18:54:03Z)
• Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation [67.07112533415116]
  We present a novel framework that adapts various foundational models for the 3D point cloud segmentation task. Our approach involves making initial predictions of 2D semantic masks using different large vision models. To generate robust 3D semantic pseudo labels, we introduce a semantic label fusion strategy that effectively combines all the results via voting.
  arXiv  Detail & Related papers  (2023-11-03T15:41:15Z)
• Diffusion-based 3D Object Detection with Random Boxes [58.43022365393569]
  Existing anchor-based 3D detection methods rely on empiricals setting of anchors, which makes the algorithms lack elegance. Our proposed Diff3Det migrates the diffusion model to proposal generation for 3D object detection by considering the detection boxes as generative targets. In the inference stage, the model progressively refines a set of random boxes to the prediction results.
  arXiv  Detail & Related papers  (2023-09-05T08:49:53Z)
• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation [28.25023686484727]
  A diffusion model learns to predict a vector field of gradients. We propose a chain rule on the learned gradients, and back-propagate the score of a diffusion model through the Jacobian of a differentiable field. We run our algorithm on several off-the-shelf diffusion image generative models, including the recently released Stable Diffusion trained on the large-scale LAION dataset.
  arXiv  Detail & Related papers  (2022-12-01T18:56:37Z)
• 3D Neural Field Generation using Triplane Diffusion [37.46688195622667]
  We present an efficient diffusion-based model for 3D-aware generation of neural fields. Our approach pre-processes training data, such as ShapeNet meshes, by converting them to continuous occupancy fields. We demonstrate state-of-the-art results on 3D generation on several object classes from ShapeNet.
  arXiv  Detail & Related papers  (2022-11-30T01:55:52Z)
• Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis [143.22192229456306]
  This paper proposes a deep 3D energy-based model to represent volumetric shapes. The benefits of the proposed model are six-fold. Experiments demonstrate that the proposed model can generate high-quality 3D shape patterns.
  arXiv  Detail & Related papers  (2020-12-25T06:09:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.