LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans

• URL: http://arxiv.org/abs/2507.02861v1
• Date: Thu, 03 Jul 2025 17:59:55 GMT
• Title: LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans
• Authors: Zhening Huang, Xiaoyang Wu, Fangcheng Zhong, Hengshuang Zhao, Matthias Nießner, Joan Lasenby,
• Abstract summary: LiteReality is a novel pipeline that converts RGB-D scans of indoor environments into compact, realistic, and interactive 3D virtual replicas.<n> LiteReality supports key features essential for graphics pipelines -- such as object individuality, articulation, high-quality rendering materials, and physically based interaction.<n>We demonstrate the effectiveness of LiteReality on both real-life scans and public datasets.
• Score: 64.31686158593351
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose LiteReality, a novel pipeline that converts RGB-D scans of indoor environments into compact, realistic, and interactive 3D virtual replicas. LiteReality not only reconstructs scenes that visually resemble reality but also supports key features essential for graphics pipelines -- such as object individuality, articulation, high-quality physically based rendering materials, and physically based interaction. At its core, LiteReality first performs scene understanding and parses the results into a coherent 3D layout and objects with the help of a structured scene graph. It then reconstructs the scene by retrieving the most visually similar 3D artist-crafted models from a curated asset database. Next, the Material Painting module enhances realism by recovering high-quality, spatially varying materials. Finally, the reconstructed scene is integrated into a simulation engine with basic physical properties to enable interactive behavior. The resulting scenes are compact, editable, and fully compatible with standard graphics pipelines, making them suitable for applications in AR/VR, gaming, robotics, and digital twins. In addition, LiteReality introduces a training-free object retrieval module that achieves state-of-the-art similarity performance on the Scan2CAD benchmark, along with a robust material painting module capable of transferring appearances from images of any style to 3D assets -- even under severe misalignment, occlusion, and poor lighting. We demonstrate the effectiveness of LiteReality on both real-life scans and public datasets. Project page: https://litereality.github.io; Video: https://www.youtube.com/watch?v=ecK9m3LXg2c

Related papers

• R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulation [78.26308457952636]
  This paper introduces R3D2, a lightweight, one-step diffusion model designed to overcome limitations in autonomous driving simulation.<n>It enables realistic insertion of complete 3D assets into existing scenes by generating plausible rendering effects-such as shadows and consistent lighting-in real time.<n>We show that R3D2 significantly enhances the realism of inserted assets, enabling use-cases like text-to-3D asset insertion and cross-scene/dataset object transfer.
  arXiv  Detail & Related papers  (2025-06-09T14:50:19Z)
• StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on 3D Gaussians [23.1385740508835]
  StyleMe3D is a holistic framework for 3D GS style transfer.<n>It integrates multi-modal style conditioning, multi-level semantic alignment, and perceptual quality enhancement.<n>This work bridges photorealistic 3D GS and artistic stylization, unlocking applications in gaming, virtual worlds, and digital art.
  arXiv  Detail & Related papers  (2025-04-21T17:59:55Z)
• DRAWER: Digital Reconstruction and Articulation With Environment Realism [42.13130021795826]
  We present DRAWER, a novel framework that converts a video of a static indoor scene into a photorealistic and interactive digital environment.<n>We demonstrate the potential of DRAWER by using it to automatically create an interactive game in Unreal Engine and to enable real-to-sim-to-real transfer for robotics applications.
  arXiv  Detail & Related papers  (2025-04-21T17:59:49Z)
• Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials [108.59709545364395]
  GPT-4V can effectively recognize and describe materials, allowing the construction of a detailed material library. The correctly matched materials are then meticulously applied as reference for the new SVBRDF material generation. Make-it-Real offers a streamlined integration into the 3D content creation workflow.
  arXiv  Detail & Related papers  (2024-04-25T17:59:58Z)
• EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction [53.28220984270622]
  3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time.<n>Our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets.<n>Our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second)
  arXiv  Detail & Related papers  (2023-11-16T11:30:56Z)
• Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives [70.32817882783608]
  We present an approach that produces a simple, compact, and actionable 3D world representation by means of 3D primitives. Unlike existing primitive decomposition methods that rely on 3D input data, our approach operates directly on images. We show that the resulting textured primitives faithfully reconstruct the input images and accurately model the visible 3D points.
  arXiv  Detail & Related papers  (2023-07-11T17:58:31Z)
• Neural Assets: Volumetric Object Capture and Rendering for Interactive Environments [8.258451067861932]
  We propose an approach for capturing real-world objects in everyday environments faithfully and fast. We use a novel neural representation to reconstruct effects, such as translucent object parts, and preserve object appearance. This leads to a seamless integration of the proposed neural assets with existing mesh environments and objects.
  arXiv  Detail & Related papers  (2022-12-12T18:55:03Z)
• PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes [84.66946637534089]
  PhotoScene is a framework that takes input image(s) of a scene and builds a photorealistic digital twin with high-quality materials and similar lighting. We model scene materials using procedural material graphs; such graphs represent photorealistic and resolution-independent materials. We evaluate our technique on objects and layout reconstructions from ScanNet, SUN RGB-D and stock photographs, and demonstrate that our method reconstructs high-quality, fully relightable 3D scenes.
  arXiv  Detail & Related papers  (2022-07-02T06:52:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.