SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation
- URL: http://arxiv.org/abs/2206.12055v1
- Date: Fri, 24 Jun 2022 03:11:28 GMT
- Title: SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation
- Authors: Xin-Yang Zheng and Yang Liu and Peng-Shuai Wang and Xin Tong
- Abstract summary: We present StyleGAN2-based deep learning approach for 3D shape generation, called SDF-StyleGAN.
We extend StyleGAN2 to 3D generation and utilize the implicit signed distance function (SDF) as the 3D shape representation.
We introduce two novel global and local shape discriminators that distinguish real and fake SDF values and gradients.
- Score: 19.21267260770415
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a StyleGAN2-based deep learning approach for 3D shape generation,
called SDF-StyleGAN, with the aim of reducing visual and geometric
dissimilarity between generated shapes and a shape collection. We extend
StyleGAN2 to 3D generation and utilize the implicit signed distance function
(SDF) as the 3D shape representation, and introduce two novel global and local
shape discriminators that distinguish real and fake SDF values and gradients to
significantly improve shape geometry and visual quality. We further complement
the evaluation metrics of 3D generative models with the shading-image-based
Fr\'echet inception distance (FID) scores to better assess visual quality and
shape distribution of the generated shapes. Experiments on shape generation
demonstrate the superior performance of SDF-StyleGAN over the state-of-the-art.
We further demonstrate the efficacy of SDF-StyleGAN in various tasks based on
GAN inversion, including shape reconstruction, shape completion from partial
point clouds, single-view image-based shape generation, and shape style
editing. Extensive ablation studies justify the efficacy of our framework
design. Our code and trained models are available at
https://github.com/Zhengxinyang/SDF-StyleGAN.
Related papers
- NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation [52.772319840580074]
3D shape generation aims to produce innovative 3D content adhering to specific conditions and constraints.
Existing methods often decompose 3D shapes into a sequence of localized components, treating each element in isolation.
We introduce a novel spatial-aware 3D shape generation framework that leverages 2D plane representations for enhanced 3D shape modeling.
arXiv Detail & Related papers (2024-03-27T04:09:34Z) - Mosaic-SDF for 3D Generative Models [41.4585856558786]
When training a diffusion or flow models on 3D shapes a crucial design choice is the shape representation.
We introduce Mosaic-SDF, a simple 3D shape representation that approximates the Signed Distance Function (SDF) of a given shape.
We demonstrate the efficacy of the M-SDF representation by using it to train a 3D generative flow model.
arXiv Detail & Related papers (2023-12-14T18:52:52Z) - Locally Attentional SDF Diffusion for Controllable 3D Shape Generation [24.83724829092307]
We propose a diffusion-based 3D generation framework, to model plausible 3D shapes, via 2D sketch image input.
Our method is built on a two-stage diffusion model. The first stage, named occupancy-diffusion, aims to generate a low-resolution occupancy field to approximate the shape shell.
The second stage, named SDF-diffusion, synthesizes a high-resolution signed distance field within the occupied voxels determined by the first stage to extract fine geometry.
arXiv Detail & Related papers (2023-05-08T05:07:23Z) - gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object
Reconstruction [94.46581592405066]
We exploit the hand structure and use it as guidance for SDF-based shape reconstruction.
We predict kinematic chains of pose transformations and align SDFs with highly-articulated hand poses.
arXiv Detail & Related papers (2023-04-24T10:05:48Z) - StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation [34.01352591390208]
We introduce a high resolution, 3D-consistent image and shape generation technique which we call StyleSDF.
Our method is trained on single-view RGB data only, and stands on the shoulders of StyleGAN2 for image generation.
arXiv Detail & Related papers (2021-12-21T18:45:45Z) - Gradient-SDF: A Semi-Implicit Surface Representation for 3D
Reconstruction [53.315347543761426]
Gradient-SDF is a novel representation for 3D geometry that combines the advantages of implict and explicit representations.
By storing at every voxel both the signed distance field as well as its gradient vector field, we enhance the capability of implicit representations.
We show that (1) the Gradient-SDF allows us to perform direct SDF tracking from depth images, using efficient storage schemes like hash maps, and that (2) the Gradient-SDF representation enables us to perform photometric bundle adjustment directly in a voxel representation.
arXiv Detail & Related papers (2021-11-26T18:33:14Z) - 3DStyleNet: Creating 3D Shapes with Geometric and Texture Style
Variations [81.45521258652734]
We propose a method to create plausible geometric and texture style variations of 3D objects.
Our method can create many novel stylized shapes, resulting in effortless 3D content creation and style-ware data augmentation.
arXiv Detail & Related papers (2021-08-30T02:28:31Z) - 3D Shapes Local Geometry Codes Learning with SDF [8.37542758486152]
A signed distance function (SDF) as the 3D shape description is one of the most effective approaches to represent 3D geometry for rendering and reconstruction.
In this paper, we consider the degeneration problem of reconstruction coming from the capacity decrease of the DeepSDF model.
We propose Local Geometry Code Learning (LGCL), a model that improves the original DeepSDF results by learning from a local shape geometry.
arXiv Detail & Related papers (2021-08-19T09:56:03Z) - Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D
Shapes [77.6741486264257]
We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs.
We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works.
arXiv Detail & Related papers (2021-01-26T18:50:22Z) - DECOR-GAN: 3D Shape Detailization by Conditional Refinement [50.8801457082181]
We introduce a deep generative network for 3D shape detailization, akin to stylization with the style being geometric details.
We demonstrate that our method can refine a coarse shape into a variety of detailed shapes with different styles.
arXiv Detail & Related papers (2020-12-16T18:52:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.