3D Photo Stylization: Learning to Generate Stylized Novel Views from a
Single Image
- URL: http://arxiv.org/abs/2112.00169v1
- Date: Tue, 30 Nov 2021 23:27:10 GMT
- Title: 3D Photo Stylization: Learning to Generate Stylized Novel Views from a
Single Image
- Authors: Fangzhou Mu, Jian Wang, Yicheng Wu, Yin Li
- Abstract summary: Style transfer and single-image 3D photography as two representative tasks have so far evolved independently.
We propose a deep model that learns geometry-aware content features for stylization from a point cloud representation of the scene.
We demonstrate the superiority of our method via extensive qualitative and quantitative studies.
- Score: 26.71747401875526
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Visual content creation has spurred a soaring interest given its applications
in mobile photography and AR / VR. Style transfer and single-image 3D
photography as two representative tasks have so far evolved independently. In
this paper, we make a connection between the two, and address the challenging
task of 3D photo stylization - generating stylized novel views from a single
image given an arbitrary style. Our key intuition is that style transfer and
view synthesis have to be jointly modeled for this task. To this end, we
propose a deep model that learns geometry-aware content features for
stylization from a point cloud representation of the scene, resulting in
high-quality stylized images that are consistent across views. Further, we
introduce a novel training protocol to enable the learning using only 2D
images. We demonstrate the superiority of our method via extensive qualitative
and quantitative studies, and showcase key applications of our method in light
of the growing demand for 3D content creation from 2D image assets.
Related papers
- Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images [54.56070204172398]
We propose a simple yet effective pipeline for stylizing a 3D scene.
We perform 3D style transfer by refining the source NeRF model using stylized images generated by a style-aligned image-to-image diffusion model.
We demonstrate that our method can transfer diverse artistic styles to real-world 3D scenes with competitive quality.
arXiv Detail & Related papers (2024-06-19T09:36:18Z) - Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generation [12.693847842218604]
We introduce a novel 3D customization method, dubbed Make-Your-3D, that can personalize high-fidelity and consistent 3D content within 5 minutes.
Our key insight is to harmonize the distributions of a multi-view diffusion model and an identity-specific 2D generative model, aligning them with the distribution of the desired 3D subject.
Our method can produce high-quality, consistent, and subject-specific 3D content with text-driven modifications that are unseen in subject image.
arXiv Detail & Related papers (2024-03-14T17:57:04Z) - ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models [65.22994156658918]
We present a method that learns to generate multi-view images in a single denoising process from real-world data.
We design an autoregressive generation that renders more 3D-consistent images at any viewpoint.
arXiv Detail & Related papers (2024-03-04T07:57:05Z) - Towards 4D Human Video Stylization [56.33756124829298]
We present a first step towards 4D (3D and time) human video stylization, which addresses style transfer, novel view synthesis and human animation.
We leverage Neural Radiance Fields (NeRFs) to represent videos, conducting stylization in the rendered feature space.
Our framework uniquely extends its capabilities to accommodate novel poses and viewpoints, making it a versatile tool for creative human video stylization.
arXiv Detail & Related papers (2023-12-07T08:58:33Z) - Guide3D: Create 3D Avatars from Text and Image Guidance [55.71306021041785]
Guide3D is a text-and-image-guided generative model for 3D avatar generation based on diffusion models.
Our framework produces topologically and structurally correct geometry and high-resolution textures.
arXiv Detail & Related papers (2023-08-18T17:55:47Z) - Learning to Stylize Novel Views [82.24095446809946]
We tackle a 3D scene stylization problem - generating stylized images of a scene from arbitrary novel views.
We propose a point cloud-based method for consistent 3D scene stylization.
arXiv Detail & Related papers (2021-05-27T23:58:18Z) - Stylizing 3D Scene via Implicit Representation and HyperNetwork [34.22448260525455]
A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches.
Inspired by the high quality results of the neural radiance fields (NeRF) method, we propose a joint framework to directly render novel views with the desired style.
Our framework consists of two components: an implicit representation of the 3D scene with the neural radiance field model, and a hypernetwork to transfer the style information into the scene representation.
arXiv Detail & Related papers (2021-05-27T09:11:30Z) - 3DSNet: Unsupervised Shape-to-Shape 3D Style Transfer [66.48720190245616]
We propose a learning-based approach for style transfer between 3D objects.
The proposed method can synthesize new 3D shapes both in the form of point clouds and meshes.
We extend our technique to implicitly learn the multimodal style distribution of the chosen domains.
arXiv Detail & Related papers (2020-11-26T16:59:12Z) - Neural Style Transfer for Remote Sensing [0.0]
The purpose of this study is to present a method for creating artistic maps from satellite images, based on the NST algorithm.
This method includes three basic steps (i.e. application of semantic image segmentation on the original satellite image, dividing its content into classes, application of neural style transfer for each class and creation of a collage)
arXiv Detail & Related papers (2020-07-31T09:30:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.