StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on 3D Gaussians

• URL: http://arxiv.org/abs/2504.15281v1
• Date: Mon, 21 Apr 2025 17:59:55 GMT
• Title: StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on 3D Gaussians
• Authors: Cailin Zhuang, Yaoqi Hu, Xuanyang Zhang, Wei Cheng, Jiacheng Bao, Shengqi Liu, Yiying Yang, Xianfang Zeng, Gang Yu, Ming Li,
• Abstract summary: StyleMe3D is a holistic framework for 3D GS style transfer.<n>It integrates multi-modal style conditioning, multi-level semantic alignment, and perceptual quality enhancement.<n>This work bridges photorealistic 3D GS and artistic stylization, unlocking applications in gaming, virtual worlds, and digital art.
• Score: 23.1385740508835
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: 3D Gaussian Splatting (3DGS) excels in photorealistic scene reconstruction but struggles with stylized scenarios (e.g., cartoons, games) due to fragmented textures, semantic misalignment, and limited adaptability to abstract aesthetics. We propose StyleMe3D, a holistic framework for 3D GS style transfer that integrates multi-modal style conditioning, multi-level semantic alignment, and perceptual quality enhancement. Our key insights include: (1) optimizing only RGB attributes preserves geometric integrity during stylization; (2) disentangling low-, medium-, and high-level semantics is critical for coherent style transfer; (3) scalability across isolated objects and complex scenes is essential for practical deployment. StyleMe3D introduces four novel components: Dynamic Style Score Distillation (DSSD), leveraging Stable Diffusion's latent space for semantic alignment; Contrastive Style Descriptor (CSD) for localized, content-aware texture transfer; Simultaneously Optimized Scale (SOS) to decouple style details and structural coherence; and 3D Gaussian Quality Assessment (3DG-QA), a differentiable aesthetic prior trained on human-rated data to suppress artifacts and enhance visual harmony. Evaluated on NeRF synthetic dataset (objects) and tandt db (scenes) datasets, StyleMe3D outperforms state-of-the-art methods in preserving geometric details (e.g., carvings on sculptures) and ensuring stylistic consistency across scenes (e.g., coherent lighting in landscapes), while maintaining real-time rendering. This work bridges photorealistic 3D GS and artistic stylization, unlocking applications in gaming, virtual worlds, and digital art.

Related papers

• Visibility-Uncertainty-guided 3D Gaussian Inpainting via Scene Conceptional Learning [63.94919846010485]
  3D Gaussian inpainting (3DGI) is challenging in effectively leveraging complementary visual and semantic cues from multiple input views. We propose a method that measures the visibility uncertainties of 3D points across different input views and uses them to guide 3DGI. We build a novel 3DGI framework, VISTA, by integrating VISibility-uncerTainty-guided 3DGI with scene conceptuAl learning.
  arXiv  Detail & Related papers  (2025-04-23T06:21:11Z)
• ArtNVG: Content-Style Separated Artistic Neighboring-View Gaussian Stylization [4.362923197888669]
  ArtNVG is an innovative 3D stylization framework that efficiently generates stylized 3D scenes by leveraging reference style images.<n>Our framework realizes high-quality 3D stylization by incorporating two pivotal techniques: Content-Style Separated Control and Attention-based Neighboring-View Alignment.
  arXiv  Detail & Related papers  (2024-12-25T05:19:52Z)
• StyleSplat: 3D Object Style Transfer with Gaussian Splatting [0.3374875022248866]
  Style transfer can enhance 3D assets with diverse artistic styles, transforming creative expression. We introduce StyleSplat, a method for stylizing 3D objects in scenes represented by 3D Gaussians from reference style images. We demonstrate its effectiveness across various 3D scenes and styles, showcasing enhanced control and customization in 3D creation.
  arXiv  Detail & Related papers  (2024-07-12T17:55:08Z)
• StylizedGS: Controllable Stylization for 3D Gaussian Splatting [53.0225128090909]
  StylizedGS is an efficient 3D neural style transfer framework with adaptable control over perceptual factors. Our method achieves high-quality stylization results characterized by faithful brushstrokes and geometric consistency with flexible controls.
  arXiv  Detail & Related papers  (2024-04-08T06:32:11Z)
• HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting [53.6394928681237]
  holistic understanding of urban scenes based on RGB images is a challenging yet important problem. Our main idea involves the joint optimization of geometry, appearance, semantics, and motion using a combination of static and dynamic 3D Gaussians. Our approach offers the ability to render new viewpoints in real-time, yielding 2D and 3D semantic information with high accuracy.
  arXiv  Detail & Related papers  (2024-03-19T13:39:05Z)
• Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects [84.45345829270626]
  Controllable 3D indoor scene synthesis stands at the forefront of technological progress. Current methods for scene stylization are limited to applying styles to the entire scene. We introduce a unique pipeline designed for synthesis 3D indoor scenes.
  arXiv  Detail & Related papers  (2024-01-24T03:10:36Z)
• FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding [11.118857208538039]
  We present Foundation Model Embedded Gaussian Splatting (S), which incorporates vision-language embeddings of foundation models into 3D Gaussian Splatting (GS) Results demonstrate remarkable multi-view semantic consistency, facilitating diverse downstream tasks, beating state-of-the-art methods by 10.2 percent on open-vocabulary language-based object detection. This research explores the intersection of vision, language, and 3D scene representation, paving the way for enhanced scene understanding in uncontrolled real-world environments.
  arXiv  Detail & Related papers  (2024-01-03T20:39:02Z)
• SceneWiz3D: Towards Text-guided 3D Scene Composition [134.71933134180782]
  Existing approaches either leverage large text-to-image models to optimize a 3D representation or train 3D generators on object-centric datasets. We introduce SceneWiz3D, a novel approach to synthesize high-fidelity 3D scenes from text.
  arXiv  Detail & Related papers  (2023-12-13T18:59:30Z)
• TADA! Text to Animatable Digital Avatars [57.52707683788961]
  TADA takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures. We derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map. We render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process.
  arXiv  Detail & Related papers  (2023-08-21T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.