ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization

• URL: http://arxiv.org/abs/2308.12452v2
• Date: Wed, 6 Sep 2023 12:00:04 GMT
• Title: ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization
• Authors: Wenzhao Li, Tianhao Wu, Fangcheng Zhong, Cengiz Oztireli
• Abstract summary: radiance fields style transfer is an emerging field that has recently gained popularity as a means of 3D scene stylization. We highlight a research gap in radiance fields style transfer, the lack of sufficient perceptual controllability. We present ARF-Plus, a 3D neural style transfer framework offering manageable control over perceptual factors.
• Score: 11.841897748330302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The radiance fields style transfer is an emerging field that has recently gained popularity as a means of 3D scene stylization, thanks to the outstanding performance of neural radiance fields in 3D reconstruction and view synthesis. We highlight a research gap in radiance fields style transfer, the lack of sufficient perceptual controllability, motivated by the existing concept in the 2D image style transfer. In this paper, we present ARF-Plus, a 3D neural style transfer framework offering manageable control over perceptual factors, to systematically explore the perceptual controllability in 3D scene stylization. Four distinct types of controls - color preservation control, (style pattern) scale control, spatial (selective stylization area) control, and depth enhancement control - are proposed and integrated into this framework. Results from real-world datasets, both quantitative and qualitative, show that the four types of controls in our ARF-Plus framework successfully accomplish their corresponding perceptual controls when stylizing 3D scenes. These techniques work well for individual style inputs as well as for the simultaneous application of multiple styles within a scene. This unlocks a realm of limitless possibilities, allowing customized modifications of stylization effects and flexible merging of the strengths of different styles, ultimately enabling the creation of novel and eye-catching stylistic effects on 3D scenes.

Related papers

• G3DST: Generalizing 3D Style Transfer with Neural Radiance Fields across Scenes and Styles [45.92812062685523]
  Existing methods for 3D style transfer need extensive per-scene optimization for single or multiple styles. In this work, we overcome the limitations of existing methods by rendering stylized novel views from a NeRF without the need for per-scene or per-style optimization. Our findings demonstrate that this approach achieves a good visual quality comparable to that of per-scene methods.
  arXiv  Detail & Related papers  (2024-08-24T08:04:19Z)
• CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields [7.651502365257349]
  We introduce Controllable Artistic Radiance Fields (CoARF), a novel algorithm for controllable 3D scene stylization. CoARF provides user-specified controllability of style transfer and superior style transfer quality with more precise feature matching.
  arXiv  Detail & Related papers  (2024-04-23T12:22:32Z)
• StylizedGS: Controllable Stylization for 3D Gaussian Splatting [53.0225128090909]
  StylizedGS is an efficient 3D neural style transfer framework with adaptable control over perceptual factors. Our method achieves high-quality stylization results characterized by faithful brushstrokes and geometric consistency with flexible controls.
  arXiv  Detail & Related papers  (2024-04-08T06:32:11Z)
• Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects [84.45345829270626]
  Controllable 3D indoor scene synthesis stands at the forefront of technological progress. Current methods for scene stylization are limited to applying styles to the entire scene. We introduce a unique pipeline designed for synthesis 3D indoor scenes.
  arXiv  Detail & Related papers  (2024-01-24T03:10:36Z)
• DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields [96.0858117473902]
  3D toonification involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture. We propose DeformToon3D, an effective toonification framework tailored for hierarchical 3D GAN. Our approach decomposes 3D toonification into subproblems of geometry and texture stylization to better preserve the original latent space.
  arXiv  Detail & Related papers  (2023-09-08T16:17:45Z)
• S2RF: Semantically Stylized Radiance Fields [1.243080988483032]
  We present our method for transferring style from any arbitrary image(s) to object(s) within a 3D scene. Our primary objective is to offer more control in 3D scene stylization, facilitating the creation of customizable and stylized scene images from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2023-09-03T19:32:49Z)
• OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis [81.70922087960271]
  We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images. Our model can synthesize more preferable identity-preserved 3D heads with compelling dynamic details compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-27T18:36:53Z)
• StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields [52.19291190355375]
  StyleRF (Style Radiance Fields) is an innovative 3D style transfer technique. It employs an explicit grid of high-level features to represent 3D scenes, with which high-fidelity geometry can be reliably restored via volume rendering. It transforms the grid features according to the reference style which directly leads to high-quality zero-shot style transfer.
  arXiv  Detail & Related papers  (2023-03-19T08:26:06Z)
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis [92.25145204543904]
  StyleNeRF is a 3D-aware generative model for high-resolution image synthesis with high multi-view consistency. It integrates the neural radiance field (NeRF) into a style-based generator. It can synthesize high-resolution images at interactive rates while preserving 3D consistency at high quality.
  arXiv  Detail & Related papers  (2021-10-18T02:37:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.