RenderFormer++: Scalable and Physically Grounded Feed-Forward Neural Rendering

• URL: http://arxiv.org/abs/2606.30380v1
• Date: Mon, 29 Jun 2026 14:39:17 GMT
• Title: RenderFormer++: Scalable and Physically Grounded Feed-Forward Neural Rendering
• Abstract summary: RenderFormer++ is a feed-forward neural rendering framework for global illumination in mesh scenes.<n>We introduce Physics-Informed Transport Guidance (PITG), which embeds rendering-equation inductive biases into the attention mechanism.<n>We also propose Hierarchical Object-Centric Tokenization (H OCT), which aggregates triangle-level features into compact object-level tokens.
• Score: 11.616763064775611
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present RenderFormer++, a scalable and physically grounded feed-forward neural rendering framework for global illumination in mesh scenes. Existing Transformer-based neural rendering methods such as RenderFormer achieve promising cross-scene generalization, but suffer from limited physical consistency and poor scalability due to the quadratic attention complexity of triangle-level tokenization. To address these issues, we introduce Physics-Informed Transport Guidance (PITG), which embeds rendering-equation inductive biases into the attention mechanism and enforces transport consistency loss, enabling physically consistent light transport modeling. We further propose Hierarchical Object-Centric Tokenization (HOCT), which aggregates triangle-level features into compact object-level tokens via cross-attention with learnable queries, substantially reducing computational and memory costs while preserving geometric and radiometric information. Extensive experiments demonstrate that RenderFormer++ achieves scalable, stable, and generalizable feed-forward global illumination rendering across complex large-scale scenes with improved physical accuracy and efficiency over prior neural rendering methods.

Related papers

• RenderFlow: Single-Step Neural Rendering via Flow Matching [17.56739408578129]
  We present a novel end-to-end, deterministic, single-step neural rendering framework, RenderFlow, built upon a flow matching paradigm.<n>Our method significantly accelerates rendering process and enhances both the physical plausibility and overall visual quality of the output.<n>The resulting pipeline achieves near real-time performance with photorealistic rendering quality, effectively bridging the gap between the efficiency of modern generative models and the precision of traditional physically based rendering.
  arXiv  Detail & Related papers  (2026-01-11T14:28:46Z)
• A Convolutional Neural Deferred Shader for Physics Based Rendering [9.933770503395117]
  Recent advances in neural rendering have achieved impressive results on photorealistic shading and relighting.<n>This paper introduces pbnds+: a novel physics-based neural deferred shading pipeline utilizing convolution neural networks to decrease the parameters.
  arXiv  Detail & Related papers  (2025-12-22T16:16:13Z)
• A Generalizable Light Transport 3D Embedding for Global Illumination [30.088406137167997]
  We propose a generalizable 3D light transport embedding that approximates global illumination directly from 3D scene configurations.<n>A scalable transformer models global point-to-point interactions to encode these features into neural primitives.<n>We demonstrate results on diffuse global illumination prediction across diverse indoor scenes with varying layouts, geometry, and materials.
  arXiv  Detail & Related papers  (2025-10-21T00:29:09Z)
• SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modeling [79.56581753856452]
  SparseFlex is a novel sparse-structured isosurface representation that enables differentiable mesh reconstruction at resolutions up to $10243$ directly from rendering losses.<n>By enabling high-resolution, differentiable mesh reconstruction and generation with rendering losses, SparseFlex significantly advances the state-of-the-art in 3D shape representation and modeling.
  arXiv  Detail & Related papers  (2025-03-27T17:46:42Z)
• SMORE: Simultaneous Map and Object REconstruction [66.66729715211642]
  We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR.<n>We take a holistic perspective and optimize a compositional model of a dynamic scene that decomposes the world into rigidly-moving objects and the background.
  arXiv  Detail & Related papers  (2024-06-19T23:53:31Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Inverse Rendering of Translucent Objects using Physical and Neural Renderers [13.706425832518093]
  In this work, we propose an inverse model that estimates 3D shape, spatially-varying reflectance, homogeneous scattering parameters, and an environment illumination jointly from only a pair of captured images of a translucent object. Because two reconstructions are differentiable, we can compute a reconstruction loss to assist parameter estimation. We constructed a large-scale synthetic dataset of translucent objects, which consists of 117K scenes.
  arXiv  Detail & Related papers  (2023-05-15T04:03:11Z)
• Progressively-connected Light Field Network for Efficient View Synthesis [69.29043048775802]
  We present a Progressively-connected Light Field network (ProLiF) for the novel view synthesis of complex forward-facing scenes. ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses.
  arXiv  Detail & Related papers  (2022-07-10T13:47:20Z)
• Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data [73.8970871148949]
  High-fidelity reconstruction of fluids from sparse multiview RGB videos remains a formidable challenge. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting. We present the first method to reconstruct dynamic fluid by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization.
  arXiv  Detail & Related papers  (2022-06-14T03:38:08Z)
• Object-based Illumination Estimation with Rendering-aware Neural Networks [56.01734918693844]
  We present a scheme for fast environment light estimation from the RGBD appearance of individual objects and their local image areas. With the estimated lighting, virtual objects can be rendered in AR scenarios with shading that is consistent to the real scene.
  arXiv  Detail & Related papers  (2020-08-06T08:23:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.