HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a
Single Image
- URL: http://arxiv.org/abs/2312.04543v1
- Date: Thu, 7 Dec 2023 18:58:09 GMT
- Title: HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a
Single Image
- Authors: Tong Wu, Zhibing Li, Shuai Yang, Pan Zhang, Xinggang Pan, Jiaqi Wang,
Dahua Lin, Ziwei Liu
- Abstract summary: We introduce HyperDreamer, a tool for creating 3D content from a single image.
It is hyper-realistic enough for post-generation usage, as users cannot view, render and edit the resulting 3D content from a full range.
We demonstrate the effectiveness of HyperDreamer in modeling region-aware materials with high-resolution textures and enabling user-friendly editing.
- Score: 94.11473240505534
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: 3D content creation from a single image is a long-standing yet highly
desirable task. Recent advances introduce 2D diffusion priors, yielding
reasonable results. However, existing methods are not hyper-realistic enough
for post-generation usage, as users cannot view, render and edit the resulting
3D content from a full range. To address these challenges, we introduce
HyperDreamer with several key designs and appealing properties: 1) Viewable:
360 degree mesh modeling with high-resolution textures enables the creation of
visually compelling 3D models from a full range of observation points. 2)
Renderable: Fine-grained semantic segmentation and data-driven priors are
incorporated as guidance to learn reasonable albedo, roughness, and specular
properties of the materials, enabling semantic-aware arbitrary material
estimation. 3) Editable: For a generated model or their own data, users can
interactively select any region via a few clicks and efficiently edit the
texture with text-based guidance. Extensive experiments demonstrate the
effectiveness of HyperDreamer in modeling region-aware materials with
high-resolution textures and enabling user-friendly editing. We believe that
HyperDreamer holds promise for advancing 3D content creation and finding
applications in various domains.
Related papers
- InsTex: Indoor Scenes Stylized Texture Synthesis [81.12010726769768]
High-quality textures are crucial for 3D scenes for augmented/virtual reality (ARVR) applications.
Current methods suffer from lengthy processing times and visual artifacts.
We introduce two-stage architecture designed to generate high-quality textures for 3D scenes.
arXiv Detail & Related papers (2025-01-22T08:37:59Z) - ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models [65.22994156658918]
We present a method that learns to generate multi-view images in a single denoising process from real-world data.
We design an autoregressive generation that renders more 3D-consistent images at any viewpoint.
arXiv Detail & Related papers (2024-03-04T07:57:05Z) - En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D
Synthetic Data [36.51674664590734]
We present En3D, an enhanced izable scheme for high-qualityd 3D human avatars.
Unlike previous works that rely on scarce 3D datasets or limited 2D collections with imbalance viewing angles and pose priors, our approach aims to develop a zero-shot 3D capable of producing 3D humans.
arXiv Detail & Related papers (2024-01-02T12:06:31Z) - X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation [61.48050470095969]
X-Dreamer is a novel approach for high-quality text-to-3D content creation.
It bridges the gap between text-to-2D and text-to-3D synthesis.
arXiv Detail & Related papers (2023-11-30T07:23:00Z) - Breathing New Life into 3D Assets with Generative Repainting [74.80184575267106]
Diffusion-based text-to-image models ignited immense attention from the vision community, artists, and content creators.
Recent works have proposed various pipelines powered by the entanglement of diffusion models and neural fields.
We explore the power of pretrained 2D diffusion models and standard 3D neural radiance fields as independent, standalone tools.
Our pipeline accepts any legacy renderable geometry, such as textured or untextured meshes, and orchestrates the interaction between 2D generative refinement and 3D consistency enforcement tools.
arXiv Detail & Related papers (2023-09-15T16:34:51Z) - Fantasia3D: Disentangling Geometry and Appearance for High-quality
Text-to-3D Content Creation [45.69270771487455]
We propose a new method of Fantasia3D for high-quality text-to-3D content creation.
Key to Fantasia3D is the disentangled modeling and learning of geometry and appearance.
Our framework is more compatible with popular graphics engines, supporting relighting, editing, and physical simulation of the generated 3D assets.
arXiv Detail & Related papers (2023-03-24T09:30:09Z) - GET3D: A Generative Model of High Quality 3D Textured Shapes Learned
from Images [72.15855070133425]
We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures.
GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
arXiv Detail & Related papers (2022-09-22T17:16:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.