BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering

• URL: http://arxiv.org/abs/2606.17049v1
• Date: Mon, 15 Jun 2026 17:58:31 GMT
• Title: BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering
• Abstract summary: Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation.<n>We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering.
• Score: 6.454961474516282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/

Related papers

• Generative Relightable Avatars [54.38436795769187]
  Generative Relightable Avatars (GRA) is a person-specific method for free-view rendering and environment-map relighting of full-body humans.<n>GRA follows a hybrid approach that combines controllable, physics-grounded relighting with probabilistic refinement.
  arXiv  Detail & Related papers  (2026-06-21T23:36:00Z)
• 3DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation [13.662206166615098]
  This work introduces 3DPhysVideo, a novel training-free pipeline that generates physically realistic videos from a single image.<n>We use it as a novel view synthesizer to reconstruct complete 360-degree 3D scene geometry by guiding the image-to-video (I2V) flow model with rendered point clouds.<n>In the diverse experiments including multi-objects, and fluid interaction scenes, our method successfully bridges the gap from single-images to physically plausible videos.
  arXiv  Detail & Related papers  (2026-05-16T03:56:52Z)
• Physical Simulator In-the-Loop Video Generation [96.87054314612142]
  Physical Simulator In-the-loop Video Generation (PSIVG) is a novel framework that integrates a physical simulator into the video diffusion process.<n>PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity.
  arXiv  Detail & Related papers  (2026-03-06T15:48:25Z)
• DrivingGaussian++: Towards Realistic Reconstruction and Editable Simulation for Surrounding Dynamic Driving Scenes [49.23098808629567]
  DrivingGaussian++ is an efficient framework for realistic reconstructing and controllable editing of autonomous driving scenes.<n>It supports training-free controllable editing for dynamic driving scenes, including texture modification, weather simulation, and object manipulation.<n>Our method can automatically generate dynamic object motion trajectories and enhance their realism during the optimization process.
  arXiv  Detail & Related papers  (2025-08-28T16:22:54Z)
• TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfer [47.22201704648345]
  Illumination and texture editing are critical dimensions for world-to-world transfer.<n>Existing techniques generatively re-render the input video to realize the transfer, such as video relighting models and conditioned world generation models.<n>We propose TC-Light, a novel generative computation to overcome these problems.
  arXiv  Detail & Related papers  (2025-06-23T17:59:58Z)
• BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Video [58.97416204208624]
  We present BEAM, a novel pipeline that bridges 4D Gaussian representations with physically-based rendering (PBR) to produce high-quality, relightable videos.<n>By offering realistic, lifelike visualizations under diverse lighting conditions, BEAM opens new possibilities for interactive entertainment, storytelling, and creative visualization.
  arXiv  Detail & Related papers  (2025-02-12T10:58:09Z)
• DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models [83.28670336340608]
  We introduce DiffusionRenderer, a neural approach that addresses the dual problem of inverse and forward rendering.<n>Our model enables practical applications from a single video input--including relighting, material editing, and realistic object insertion.
  arXiv  Detail & Related papers  (2025-01-30T18:59:11Z)
• StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models [76.62929629864034]
  We introduce StreetCrafter, a controllable video diffusion model that utilizes LiDAR point cloud renderings as pixel-level conditions.<n>In addition, the utilization of pixel-level LiDAR conditions allows us to make accurate pixel-level edits to target scenes.<n>Our model enables flexible control over viewpoint changes, enlarging the view for satisfying rendering regions.
  arXiv  Detail & Related papers  (2024-12-17T18:58:55Z)
• UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video [22.613946218766802]
  We present UrbanIR, a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.<n>It accurately infers shape, albedo, visibility, and sun and sky illumination from wide-baseline videos, such as those from car-mounted cameras.<n>UrbanIR addresses these issues with novel losses that reduce errors in inverse graphics inference and rendering artifacts.
  arXiv  Detail & Related papers  (2023-06-15T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.