Hash3D: Training-free Acceleration for 3D Generation

• URL: http://arxiv.org/abs/2404.06091v1
• Date: Tue, 9 Apr 2024 07:49:30 GMT
• Title: Hash3D: Training-free Acceleration for 3D Generation
• Authors: Xingyi Yang, Xinchao Wang,
• Abstract summary: Hash3D is a universal acceleration for 3D generation without model training. By effectively hashing and reusing feature maps across neighboring timesteps and camera angles, Hash3D substantially prevents redundant calculations. Surprisingly, this feature-sharing mechanism not only speed up the generation but also enhances the smoothness and view consistency of the synthesized 3D objects.
• Score: 72.88137795439407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The evolution of 3D generative modeling has been notably propelled by the adoption of 2D diffusion models. Despite this progress, the cumbersome optimization process per se presents a critical hurdle to efficiency. In this paper, we introduce Hash3D, a universal acceleration for 3D generation without model training. Central to Hash3D is the insight that feature-map redundancy is prevalent in images rendered from camera positions and diffusion time-steps in close proximity. By effectively hashing and reusing these feature maps across neighboring timesteps and camera angles, Hash3D substantially prevents redundant calculations, thus accelerating the diffusion model's inference in 3D generation tasks. We achieve this through an adaptive grid-based hashing. Surprisingly, this feature-sharing mechanism not only speed up the generation but also enhances the smoothness and view consistency of the synthesized 3D objects. Our experiments covering 5 text-to-3D and 3 image-to-3D models, demonstrate Hash3D's versatility to speed up optimization, enhancing efficiency by 1.3 to 4 times. Additionally, Hash3D's integration with 3D Gaussian splatting largely speeds up 3D model creation, reducing text-to-3D processing to about 10 minutes and image-to-3D conversion to roughly 30 seconds. The project page is at https://adamdad.github.io/hash3D/.

Related papers

• MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Model [34.245635412589806]
  MeshFormer is a sparse-view reconstruction model that explicitly leverages 3D native structure, input guidance, and training supervision. It can be integrated with 2D diffusion models to enable fast single-image-to-3D and text-to-3D tasks.
  arXiv  Detail & Related papers  (2024-08-19T17:55:17Z)
• DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data [50.164670363633704]
  We present DIRECT-3D, a diffusion-based 3D generative model for creating high-quality 3D assets from text prompts. Our model is directly trained on extensive noisy and unaligned in-the-wild' 3D assets. We achieve state-of-the-art performance in both single-class generation and text-to-3D generation.
  arXiv  Detail & Related papers  (2024-06-06T17:58:15Z)
• Compress3D: a Compressed Latent Space for 3D Generation from a Single Image [27.53099431097921]
  Triplane autoencoder encodes 3D models into a compact triplane latent space to compress both the 3D geometry and texture information. We introduce a 3D-aware cross-attention mechanism, which utilizes low-resolution latent representations to query features from a high-resolution 3D feature volume. Our approach enables the generation of high-quality 3D assets in merely 7 seconds on a single A100 GPU.
  arXiv  Detail & Related papers  (2024-03-20T11:51:04Z)
• LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation [73.36690511083894]
  This paper introduces a novel framework called LN3Diff to address a unified 3D diffusion pipeline. Our approach harnesses a 3D-aware architecture and variational autoencoder to encode the input image into a structured, compact, and 3D latent space. It achieves state-of-the-art performance on ShapeNet for 3D generation and demonstrates superior performance in monocular 3D reconstruction and conditional 3D generation.
  arXiv  Detail & Related papers  (2024-03-18T17:54:34Z)
• AGG: Amortized Generative 3D Gaussians for Single Image to 3D [108.38567665695027]
  We introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image. AGG decomposes the generation of 3D Gaussian locations and other appearance attributes for joint optimization. We propose a cascaded pipeline that first generates a coarse representation of the 3D data and later upsamples it with a 3D Gaussian super-resolution module.
  arXiv  Detail & Related papers  (2024-01-08T18:56:33Z)
• Instant3D: Instant Text-to-3D Generation [101.25562463919795]
  We propose a novel framework for fast text-to-3D generation, dubbed Instant3D. Instant3D is able to create a 3D object for an unseen text prompt in less than one second with a single run of a feedforward network.
  arXiv  Detail & Related papers  (2023-11-14T18:59:59Z)
• GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models [102.22388340738536]
  2D and 3D diffusion models can generate decent 3D objects based on prompts. 3D diffusion models have good 3D consistency, but their quality and generalization are limited as trainable 3D data is expensive and hard to obtain. This paper attempts to bridge the power from the two types of diffusion models via the recent explicit and efficient 3D Gaussian splatting representation.
  arXiv  Detail & Related papers  (2023-10-12T17:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.