Luce: Relightable Gaussians for 3D Asset Generation

• URL: http://arxiv.org/abs/2608.23943v1
• Date: Tue, 25 Aug 2026 01:03:55 GMT
• Title: Luce: Relightable Gaussians for 3D Asset Generation
• Abstract summary: High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance.<n>We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud.<n>On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline.
• Score: 4.296019358250277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.

Related papers

• LightHarmony3D: Harmonizing Illumination and Shadows for Object Insertion in 3D Gaussian Splatting [90.15229815175593]
  We present LightHarmony3D, a novel framework for illumination-consistent mesh insertion in 3DGS scenes.<n>By leveraging generative priors instead of iterative optimization, our method efficiently captures dominant scene illumination.<n>We introduce the first dedicated benchmark for mesh insertion in 3DGS, providing a standardized evaluation framework for assessing lighting consistency and photorealism.
  arXiv  Detail & Related papers  (2026-03-31T03:26:30Z)
• Large Material Gaussian Model for Relightable 3D Generation [54.10879517395551]
  We introduce a novel framework designed to generate high-quality 3D content with Physically Based Rendering (PBR) materials.<n>Our method not only exhibit greater visual appeal compared to baseline methods but also enhance material modeling, thereby enabling practical downstream rendering applications.
  arXiv  Detail & Related papers  (2025-09-26T09:35:12Z)
• PS-GS: Gaussian Splatting for Multi-View Photometric Stereo [8.560379885301705]
  inverse rendering with multi-view photometric stereo (MVPS) yields more accurate 3D reconstructions than the inverse rendering approaches that rely on fixed environment illumination.<n>We introduce the Gaussian Splatting for Multi-view Photometric Stereo (PS-GS), which efficiently and jointly estimates the geometry, materials, and lighting of the object that is illuminated by diverse directional lights (multi-light)<n> Experiments on both synthetic and real datasets demonstrate that our method outperforms prior works in terms of reconstruction accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-07-24T09:22:02Z)
• Complex-Valued Holographic Radiance Fields [9.050557698554696]
  We reformulate 3D Gaussian splatting with complex-valued Gaussian primitives, expanding support for rendering with light waves.<n>Compared with state-of-the-art methods, our method achieves 30x-10,000x speed improvements while maintaining on-par image quality.
  arXiv  Detail & Related papers  (2025-06-10T02:09:04Z)
• SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering [23.104424058660744]
  3D Gaussian Splatting (3DGS) has demonstrated impressive capabilities for novel view synthesis (NVS) tasks.<n>We present a novel framework called Spatially-vayring Gaussian Inverse Rendering (SVG-IR) aimed at enhancing both NVS and relighting quality.<n>The proposed SVG-IR framework significantly improves rendering quality, outperforming state-of-the-art NeRF-based methods by 2.5 dB in peak signal-to-noise ratio (PSNR) and surpassing existing Gaussian-based techniques by 3.5 dB in relighting tasks.
  arXiv  Detail & Related papers  (2025-04-09T12:11:58Z)
• BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Video [58.97416204208624]
  We present BEAM, a novel pipeline that bridges 4D Gaussian representations with physically-based rendering (PBR) to produce high-quality, relightable videos.<n>By offering realistic, lifelike visualizations under diverse lighting conditions, BEAM opens new possibilities for interactive entertainment, storytelling, and creative visualization.
  arXiv  Detail & Related papers  (2025-02-12T10:58:09Z)
• TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting [48.97819552366636]
  This paper presents TexGaussian, a novel method that uses octant-aligned 3D Gaussian Splatting for rapid PBR material generation.<n>Our method synthesizes more visually pleasing PBR materials and runs faster than previous methods in both unconditional and text-conditional scenarios.
  arXiv  Detail & Related papers  (2024-11-29T12:19:39Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing [21.498078188364566]
  We present a novel differentiable point-based rendering framework to achieve photo-realistic relighting. The proposed framework showcases the potential to revolutionize the mesh-based graphics pipeline with a point-based pipeline enabling editing, tracing, and relighting.
  arXiv  Detail & Related papers  (2023-11-27T18:07:58Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.