MrGS: Multi-modal Radiance Fields with 3D Gaussian Splatting for RGB-Thermal Novel View Synthesis

• URL: http://arxiv.org/abs/2511.22997v1
• Date: Fri, 28 Nov 2025 09:01:36 GMT
• Title: MrGS: Multi-modal Radiance Fields with 3D Gaussian Splatting for RGB-Thermal Novel View Synthesis
• Authors: Minseong Kweon, Janghyun Kim, Ukcheol Shin, Jinsun Park,
• Abstract summary: We introduce MrGS, a multi-modal radiance field based on 3DGS that simultaneously reconstructs both RGB and thermal 3D scenes.<n>We leverage two physics-based principles to effectively model thermal-domain phenomena.
• Score: 10.127749951026457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) have achieved considerable performance in RGB scene reconstruction. However, multi-modal rendering that incorporates thermal infrared imagery remains largely underexplored. Existing approaches tend to neglect distinctive thermal characteristics, such as heat conduction and the Lambertian property. In this study, we introduce MrGS, a multi-modal radiance field based on 3DGS that simultaneously reconstructs both RGB and thermal 3D scenes. Specifically, MrGS derives RGB- and thermal-related information from a single appearance feature through orthogonal feature extraction and employs view-dependent or view-independent embedding strategies depending on the degree of Lambertian reflectance exhibited by each modality. Furthermore, we leverage two physics-based principles to effectively model thermal-domain phenomena. First, we integrate Fourier's law of heat conduction prior to alpha blending to model intensity interpolation caused by thermal conduction between neighboring Gaussians. Second, we apply the Stefan-Boltzmann law and the inverse-square law to formulate a depth-aware thermal radiation map that imposes additional geometric constraints on thermal rendering. Experimental results demonstrate that the proposed MrGS achieves high-fidelity RGB-T scene reconstruction while reducing the number of Gaussians.

Related papers

• ThermoSplat: Cross-Modal 3D Gaussian Splatting with Feature Modulation and Geometry Decoupling [11.169420448510095]
  ThermoSplat is a novel framework that enables deep spectral-aware reconstruction through active feature modulation and adaptive geometry decoupling.<n>Experiments on the RGBT-Scenes dataset demonstrate that ThermoSplat achieves state-of-the-art rendering quality across both visible and thermal spectrums.
  arXiv  Detail & Related papers  (2026-01-22T12:24:26Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• GOGS: High-Fidelity Geometry and Relighting for Glossy Objects via Gaussian Surfels [0.9392167468538465]
  Inverse rendering of glossy objects from RGB imagery remains fundamentally limited by inherent ambiguity.<n>We propose GOGS, a novel two-stage framework based on 2D Gaussian surfels.<n>We demonstrate state-of-the-art performance in geometry reconstruction, material separation, and photorealistic relighting under novel illuminations.
  arXiv  Detail & Related papers  (2025-08-20T09:35:40Z)
• Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesis [49.67420486373202]
  GRGS is a generalizable and relightable 3D Gaussian framework for high-fidelity human novel view synthesis under diverse lighting conditions.<n>We introduce a Lighting-aware Geometry Refinement (LGR) module trained on synthetically relit data to predict accurate depth and surface normals.
  arXiv  Detail & Related papers  (2025-05-27T17:59:47Z)
• Veta-GS: View-dependent deformable 3D Gaussian Splatting for thermal infrared Novel-view Synthesis [3.1457219084519004]
  3D Gaussian Splatting (3D-GS) based on Thermal Infrared (TIR) imaging has gained attention in novel-view synthesis.<n>We introduce Veta-GS, which leverages a view-dependent deformation field and a Thermal Feature Extractor to capture subtle thermal variations.<n>Our method achieves better performance over existing methods.
  arXiv  Detail & Related papers  (2025-05-25T13:20:45Z)
• GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering [83.69136534797686]
  We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
  arXiv  Detail & Related papers  (2024-11-12T01:51:05Z)
• Thermal3D-GS: Physics-induced 3D Gaussians for Thermal Infrared Novel-view Synthesis [11.793425521298488]
  This paper introduces a physics-induced 3D Gaussian splatting method named Thermal3D-GS. The first large-scale benchmark dataset for this field named Thermal Infrared Novel-view Synthesis dataset (TI-NSD) is created. The results indicate that our method outperforms the baseline method with a 3.03 dB improvement in PSNR.
  arXiv  Detail & Related papers  (2024-09-12T13:46:53Z)
• ThermalGaussian: Thermal 3D Gaussian Splatting [25.536611434289647]
  We propose ThermalGaussian, the first thermal 3DGS approach capable of rendering high-quality images in RGB and thermal modalities.<n>We conduct comprehensive experiments to show that ThermalGaussian achieves photorealistic rendering of thermal images and improves the rendering quality of RGB images.
  arXiv  Detail & Related papers  (2024-09-11T11:45:57Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• ThermoNeRF: Joint RGB and Thermal Novel View Synthesis for Building Facades using Multimodal Neural Radiance Fields [5.66229031510643]
  Thermal scene reconstruction holds great potential for various applications, such as analyzing building energy consumption and performing non-destructive infrastructure testing.<n>Existing methods typically require dense scene measurements and often rely on RGB images for 3D geometry reconstruction, projecting thermal information post-reconstruction.<n>We propose ThermoNeRF, a novel approach based on Neural Radiance Fields that jointly renders new RGB and thermal views of a scene, and ThermoScenes, a dataset of paired RGB+thermal images comprising 8 scenes of building facades and 8 scenes of everyday objects.
  arXiv  Detail & Related papers  (2024-03-18T18:10:34Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.