Neural Wavelet-domain Diffusion for 3D Shape Generation
- URL: http://arxiv.org/abs/2209.08725v1
- Date: Mon, 19 Sep 2022 02:51:48 GMT
- Title: Neural Wavelet-domain Diffusion for 3D Shape Generation
- Authors: Ka-Hei Hui, Ruihui Li, Jingyu Hu, Chi-Wing Fu
- Abstract summary: This paper presents a new approach for 3D shape generation, enabling direct generative modeling on a continuous implicit representation in wavelet domain.
Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets.
- Score: 52.038346313823524
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: This paper presents a new approach for 3D shape generation, enabling direct
generative modeling on a continuous implicit representation in wavelet domain.
Specifically, we propose a compact wavelet representation with a pair of coarse
and detail coefficient volumes to implicitly represent 3D shapes via truncated
signed distance functions and multi-scale biorthogonal wavelets, and formulate
a pair of neural networks: a generator based on the diffusion model to produce
diverse shapes in the form of coarse coefficient volumes; and a detail
predictor to further produce compatible detail coefficient volumes for
enriching the generated shapes with fine structures and details. Both
quantitative and qualitative experimental results manifest the superiority of
our approach in generating diverse and high-quality shapes with complex
topology and structures, clean surfaces, and fine details, exceeding the 3D
generation capabilities of the state-of-the-art models.
Related papers
- Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields [50.12118098874321]
We introduce a latent 3D diffusion process for neural voxel fields, enabling generation at significantly higher resolutions.
A part-aware shape decoder is introduced to integrate the part codes into the neural voxel fields, guiding the accurate part decomposition.
The results demonstrate the superior generative capabilities of our proposed method in part-aware shape generation, outperforming existing state-of-the-art methods.
arXiv Detail & Related papers (2024-05-02T04:31:17Z) - GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling [55.05713977022407]
We introduce a radiance representation that is both structured and fully explicit and thus greatly facilitates 3D generative modeling.
We derive GaussianCube by first using a novel densification-constrained Gaussian fitting algorithm, which yields high-accuracy fitting.
Experiments conducted on unconditional and class-conditioned object generation, digital avatar creation, and text-to-3D all show that our model synthesis achieves state-of-the-art generation results.
arXiv Detail & Related papers (2024-03-28T17:59:50Z) - Topology-Aware Latent Diffusion for 3D Shape Generation [20.358373670117537]
We introduce a new generative model that combines latent diffusion with persistent homology to create 3D shapes with high diversity.
Our method involves representing 3D shapes as implicit fields, then employing persistent homology to extract topological features.
arXiv Detail & Related papers (2024-01-31T05:13:53Z) - Hybrid Neural Diffeomorphic Flow for Shape Representation and Generation
via Triplane [16.684276798449115]
HNDF is a method that implicitly learns the underlying representation and decomposes intricate dense correspondences into explicitly axis-aligned triplane features.
Unlike conventional approaches that directly generate new 3D shapes, we explore the idea of shape generation with deformed template shape via diffeomorphic flows.
arXiv Detail & Related papers (2023-07-04T23:28:01Z) - Pushing the Limits of 3D Shape Generation at Scale [65.24420181727615]
We present a significant breakthrough in 3D shape generation by scaling it to unprecedented dimensions.
We have developed a model with an astounding 3.6 billion trainable parameters, establishing it as the largest 3D shape generation model to date, named Argus-3D.
arXiv Detail & Related papers (2023-06-20T13:01:19Z) - 3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion
Process [32.3773514247982]
We develop a generalized 3D shape generation prior model tailored for multiple 3D tasks.
Designs jointly equip our proposed 3D shape prior model with high-fidelity, diverse features as well as the capability of cross-modality alignment.
arXiv Detail & Related papers (2023-03-18T12:50:29Z) - Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and
Manipulation [54.09274684734721]
We present a new approach for 3D shape generation, inversion, and manipulation, through a direct generative modeling on a continuous implicit representation in wavelet domain.
Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets.
We may jointly train an encoder network to learn a latent space for inverting shapes, allowing us to enable a rich variety of whole-shape and region-aware shape manipulations.
arXiv Detail & Related papers (2023-02-01T02:47:53Z) - SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation [50.53931728235875]
We present SP-GAN, a new unsupervised sphere-guided generative model for direct synthesis of 3D shapes in the form of point clouds.
Compared with existing models, SP-GAN is able to synthesize diverse and high-quality shapes with fine details.
arXiv Detail & Related papers (2021-08-10T06:49:45Z) - 3D Shape Generation and Completion through Point-Voxel Diffusion [24.824065748889048]
We propose a novel approach for probabilistic generative modeling of 3D shapes.
Point-Voxel Diffusion (PVD) is a unified, probabilistic formulation for unconditional shape generation and conditional, multimodal shape completion.
PVD can be viewed as a series of denoising steps, reversing the diffusion process from observed point cloud data to Gaussian noise, and is trained by optimizing a variational lower bound to the (conditional) likelihood function.
arXiv Detail & Related papers (2021-04-08T10:38:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.