RealMaster: Lifting Rendered Scenes into Photorealistic Video

• URL: http://arxiv.org/abs/2603.23462v1
• Date: Tue, 24 Mar 2026 17:32:42 GMT
• Title: RealMaster: Lifting Rendered Scenes into Photorealistic Video
• Abstract summary: State-of-the-art video generation models produce remarkable photorealism, but they lack the precise control required to align generated content with scene requirements.<n>We present RealMaster, a method that leverages video diffusion models to lift rendered video into photorealistic video while maintaining full alignment with the output of the 3D engine.<n>RealMaster significantly outperforms existing video editing baselines, improving photorealism while preserving the geometry, dynamics, and identity specified by the original 3D control.
• Score: 55.04231137698114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State-of-the-art video generation models produce remarkable photorealism, but they lack the precise control required to align generated content with specific scene requirements. Furthermore, without an underlying explicit geometry, these models cannot guarantee 3D consistency. Conversely, 3D engines offer granular control over every scene element and provide native 3D consistency by design, yet their output often remains trapped in the "uncanny valley". Bridging this sim-to-real gap requires both structural precision, where the output must exactly preserve the geometry and dynamics of the input, and global semantic transformation, where materials, lighting, and textures must be holistically transformed to achieve photorealism. We present RealMaster, a method that leverages video diffusion models to lift rendered video into photorealistic video while maintaining full alignment with the output of the 3D engine. To train this model, we generate a paired dataset via an anchor-based propagation strategy, where the first and last frames are enhanced for realism and propagated across the intermediate frames using geometric conditioning cues. We then train an IC-LoRA on these paired videos to distill the high-quality outputs of the pipeline into a model that generalizes beyond the pipeline's constraints, handling objects and characters that appear mid-sequence and enabling inference without requiring anchor frames. Evaluated on complex GTA-V sequences, RealMaster significantly outperforms existing video editing baselines, improving photorealism while preserving the geometry, dynamics, and identity specified by the original 3D control.

Related papers

• Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Models [53.12111209077268]
  Ink3D is a framework that bridges 3D generation with large-scale video generative models to synthesize complex textures.<n> Texturer is a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies.
  arXiv  Detail & Related papers  (2026-07-01T17:56:39Z)
• Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors [61.34273238077091]
  We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-14T05:35:46Z)
• Lighting-grounded Video Generation with Renderer-based Agent Reasoning [56.50946217758078]
  LiVER is a diffusion-based framework for scene-controllable video generation.<n>We present a novel framework that conditions video synthesis on explicit 3D scene properties.<n>Our method disentangles these properties by rendering control signals from a unified 3D representation.
  arXiv  Detail & Related papers  (2026-04-09T08:29:29Z)
• TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos [65.99602532894241]
  We introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry.<n>We also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting.<n>Results demonstrate that our method outperforms existing approaches in both video consistency and final reconstruction quality.
  arXiv  Detail & Related papers  (2026-03-18T14:02:09Z)
• Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes [22.450051108066216]
  We present C2R (Coarse-to-Real), a generative framework that synthesizes real-style urban crowd videos.<n>Our approach uses coarse 3D renderings to explicitly control scene layout, camera motion, and human trajectories.<n>It produces temporally consistent, controllable, and realistic urban scene videos from minimal 3D input.
  arXiv  Detail & Related papers  (2026-01-29T20:29:04Z)
• ViSA: 3D-Aware Video Shading for Real-Time Upper-Body Avatar Creation [62.86900540547787]
  Current 3D avatar generation methods often suffer from artifacts such as blurry textures and stiff, unnatural motion.<n>We propose a novel approach that combines the strengths of both paradigms.<n>By uniting the geometric stability of 3D reconstruction with the generative capabilities of video models, our method produces high-fidelity digital avatars.
  arXiv  Detail & Related papers  (2025-12-08T17:10:29Z)
• From Gallery to Wrist: Realistic 3D Bracelet Insertion in Videos [8.444819892052958]
  2D diffusion models have shown promise for producing photorealistic edits.<n>Traditional 3D rendering methods excel in spatial and temporal consistency but fall short in achieving photorealistic lighting.<n>This is the first approach to synergize 3D rendering and 2D diffusion for video object insertion.
  arXiv  Detail & Related papers  (2025-07-27T15:49:07Z)
• Constructing a 3D Scene from a Single Image [31.11317559252235]
  SceneFuse-3D is a training-free framework designed to synthesize coherent 3D scenes from a single top-down view.<n>We decompose the input image into overlapping regions and generate each using a pretrained 3D object generator.<n>This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning.
  arXiv  Detail & Related papers  (2025-05-21T17:10:47Z)
• 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models [53.89348957053395]
  We introduce a novel pipeline designed for text-to-4D scene generation. Our method begins by generating a reference video using the video generation model. We then learn the canonical 3D representation of the video using a freeze-time video.
  arXiv  Detail & Related papers  (2024-06-11T17:19:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.