Generative World Renderer

• URL: http://arxiv.org/abs/2604.02329v1
• Date: Thu, 02 Apr 2026 17:59:53 GMT
• Title: Generative World Renderer
• Abstract summary: We introduce a large-scale, dynamic dataset curated from visually complex AAA games.<n>Using a novel dual-screen capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels.<n>This dataset uniquely advances bidirectional generalization: enabling robust in-the-wild geometry and material decomposition.
• Score: 36.885671259896505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.

Related papers

• MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds [19.94963757122156]
  Multi-view inverse rendering aims to recover geometry, materials, and illumination consistently across multiple viewpoints.<n>We introduce a feed-forward multi-view inverse rendering framework that directly predicts spatially varying albedo, metallic, roughness, diffuse shading, and surface normals from sequences of RGB images.<n>Our method achieves state-of-the-art performance in terms of multi-view consistency, material and normal estimation quality, and generalization to real-world imagery.
  arXiv  Detail & Related papers  (2025-12-24T06:59:29Z)
• Progressive Image Restoration via Text-Conditioned Video Generation [6.1671530509662205]
  Text-to-video models have demonstrated strong temporal generation capabilities, yet their potential for image restoration remains underexplored.<n>In this work, we repurpose CogVideo for progressive visual restoration tasks by fine-tuning it to generate restoration trajectories rather than natural video motion.<n>We construct synthetic datasets for super-resolution, deblurring, and low-light enhancement, where each sample depicts a gradual transition from degraded to clean frames.<n>Our model learns to associate temporal progression with restoration quality, producing sequences that improve perceptual metrics such as PSNR, SSIM, and LPIPS across frames.
  arXiv  Detail & Related papers  (2025-12-01T23:37:51Z)
• ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations [33.74746234704817]
  Video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description.<n>This is challenging as it involves deep vision-level understanding, pixel-level dense prediction andtemporal reasoning.<n>We propose bfReferDINO RVOS that inherits region-level vision-text alignment from foundational visual grounding models.
  arXiv  Detail & Related papers  (2025-01-24T16:24:15Z)
• EnvGS: Modeling View-Dependent Appearance with Environment Gaussian [78.74634059559891]
  EnvGS is a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments.<n>To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based reflection that leverages the GPU's RT core for fast rendering.<n>Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections.
  arXiv  Detail & Related papers  (2024-12-19T18:59:57Z)
• A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild [72.0226493284814]
  We propose a unified framework for event-based frame that performs deblurring ad-hoc.<n>Our network consistently outperforms previous state-of-the-art methods on frame, single image deblurring, and the joint task of both.
  arXiv  Detail & Related papers  (2023-01-12T18:19:00Z)
• Fast Non-Rigid Radiance Fields from Monocularized Data [66.74229489512683]
  This paper proposes a new method for full 360deg inward-facing novel view synthesis of non-rigidly deforming scenes. At the core of our method are 1) An efficient deformation module that decouples the processing of spatial and temporal information for accelerated training and inference; and 2) A static module representing the canonical scene as a fast hash-encoded neural radiance field. In both cases, our method is significantly faster than previous methods, converging in less than 7 minutes and achieving real-time framerates at 1K resolution, while obtaining a higher visual accuracy for generated novel views.
  arXiv  Detail & Related papers  (2022-12-02T18:51:10Z)
• Blur Interpolation Transformer for Real-World Motion from Blur [52.10523711510876]
  We propose a encoded blur transformer (BiT) to unravel the underlying temporal correlation in blur. Based on multi-scale residual Swin transformer blocks, we introduce dual-end temporal supervision and temporally symmetric ensembling strategies. In addition, we design a hybrid camera system to collect the first real-world dataset of one-to-many blur-sharp video pairs.
  arXiv  Detail & Related papers  (2022-11-21T13:10:10Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.