UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusion

• URL: http://arxiv.org/abs/2404.06851v1
• Date: Wed, 10 Apr 2024 09:24:54 GMT
• Title: UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusion
• Authors: Junsheng Zhou, Weiqi Zhang, Baorui Ma, Kanle Shi, Yu-Shen Liu, Zhizhong Han,
• Abstract summary: We present UDiFF, a 3D diffusion model for unsigned distance fields (UDFs) which is capable to generate textured 3D shapes with open surfaces from text conditions or unconditionally. Our key idea is to generate UDFs in spatial-frequency domain with an optimal wavelet transformation, which produces a compact representation space for UDF generation.
• Score: 51.31220416754788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have shown remarkable results for image generation, editing and inpainting. Recent works explore diffusion models for 3D shape generation with neural implicit functions, i.e., signed distance function and occupancy function. However, they are limited to shapes with closed surfaces, which prevents them from generating diverse 3D real-world contents containing open surfaces. In this work, we present UDiFF, a 3D diffusion model for unsigned distance fields (UDFs) which is capable to generate textured 3D shapes with open surfaces from text conditions or unconditionally. Our key idea is to generate UDFs in spatial-frequency domain with an optimal wavelet transformation, which produces a compact representation space for UDF generation. Specifically, instead of selecting an appropriate wavelet transformation which requires expensive manual efforts and still leads to large information loss, we propose a data-driven approach to learn the optimal wavelet transformation for UDFs. We evaluate UDiFF to show our advantages by numerical and visual comparisons with the latest methods on widely used benchmarks. Page: https://weiqi-zhang.github.io/UDiFF.

Related papers

• Gradient Distance Function [52.615859148238464]
  We show that Gradient Distance Functions (GDFs) can be differentiable at the surface while still being able to represent open surfaces. This is done by associating to each 3D point a 3D vector whose norm is taken to be the unsigned distance to the surface. We demonstrate the effectiveness of GDFs on ShapeNet Car, Multi-Garment, and 3D-Scene datasets.
  arXiv  Detail & Related papers  (2024-10-29T18:04:01Z)
• Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models [83.35835521670955]
  Surf-D is a novel method for generating high-quality 3D shapes as Surfaces with arbitrary topologies. We use the Unsigned Distance Field (UDF) as our surface representation to accommodate arbitrary topologies. We also propose a new pipeline that employs a point-based AutoEncoder to learn a compact and continuous latent space for accurately encoding UDF.
  arXiv  Detail & Related papers  (2023-11-28T18:56:01Z)
• Diffusion-SDF: Text-to-Shape via Voxelized Diffusion [90.85011923436593]
  We propose a new generative 3D modeling framework called Diffusion-SDF for the challenging task of text-to-shape synthesis. We show that Diffusion-SDF generates both higher quality and more diversified 3D shapes that conform well to given text descriptions.
  arXiv  Detail & Related papers  (2022-12-06T19:46:47Z)
• NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies [87.06532943371575]
  We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
  arXiv  Detail & Related papers  (2022-11-25T15:21:45Z)
• NeuralODF: Learning Omnidirectional Distance Fields for 3D Shape Representation [7.208066405543874]
  In visual computing, 3D geometry is represented in many different forms including meshes, point clouds, voxel grids, level sets, and depth images. We propose Omni Distance Fields (ODFs), a new 3D shape representation that encodes geometry by storing the depth to the object's surface from any 3D position in any viewing direction.
  arXiv  Detail & Related papers  (2022-06-12T20:59:26Z)
• DUDE: Deep Unsigned Distance Embeddings for Hi-Fidelity Representation of Complex 3D Surfaces [8.104199886760275]
  DUDE is a disentangled shape representation that utilizes an unsigned distance field (uDF) to represent proximity to a surface, and a normal vector field (nVF) to represent surface orientation. We show that a combination of these two (uDF+nVF) can be used to learn high fidelity representations for arbitrary open/closed shapes.
  arXiv  Detail & Related papers  (2020-11-04T22:49:05Z)
• Neural Unsigned Distance Fields for Implicit Function Learning [53.241423815726925]
  We propose Neural Distance Fields (NDF), a neural network based model which predicts the unsigned distance field for arbitrary 3D shapes. NDF represent surfaces at high resolutions as prior implicit models, but do not require closed surface data. NDF can be used for multi-target regression (multiple outputs for one input) with techniques that have been exclusively used for rendering in graphics.
  arXiv  Detail & Related papers  (2020-10-26T22:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.