Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting

• URL: http://arxiv.org/abs/2506.23308v1
• Date: Sun, 29 Jun 2025 15:54:15 GMT
• Title: Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting
• Authors: Yiming Huang, Long Bai, Beilei Cui, Yanheng Li, Tong Chen, Jie Wang, Jinlin Wu, Zhen Lei, Hongbin Liu, Hongliang Ren,
• Abstract summary: Endo-4DGX is a novel reconstruction method with illumination-adaptive Gaussian Splatting.<n>We introduce a region-aware enhancement module to model the sub-area lightness at the Gaussian level.<n>We employ an exposure control loss to restore the appearance from adverse exposure to the normal level for illumination-adaptive optimization.
• Score: 19.767101860583242
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate reconstruction of soft tissue is crucial for advancing automation in image-guided robotic surgery. The recent 3D Gaussian Splatting (3DGS) techniques and their variants, 4DGS, achieve high-quality renderings of dynamic surgical scenes in real-time. However, 3D-GS-based methods still struggle in scenarios with varying illumination, such as low light and over-exposure. Training 3D-GS in such extreme light conditions leads to severe optimization problems and devastating rendering quality. To address these challenges, we present Endo-4DGX, a novel reconstruction method with illumination-adaptive Gaussian Splatting designed specifically for endoscopic scenes with uneven lighting. By incorporating illumination embeddings, our method effectively models view-dependent brightness variations. We introduce a region-aware enhancement module to model the sub-area lightness at the Gaussian level and a spatial-aware adjustment module to learn the view-consistent brightness adjustment. With the illumination adaptive design, Endo-4DGX achieves superior rendering performance under both low-light and over-exposure conditions while maintaining geometric accuracy. Additionally, we employ an exposure control loss to restore the appearance from adverse exposure to the normal level for illumination-adaptive optimization. Experimental results demonstrate that Endo-4DGX significantly outperforms combinations of state-of-the-art reconstruction and restoration methods in challenging lighting environments, underscoring its potential to advance robot-assisted surgical applications. Our code is available at https://github.com/lastbasket/Endo-4DGX.

Related papers

• SAIGFormer: A Spatially-Adaptive Illumination-Guided Network for Low-Light Image Enhancement [58.79901582809091]
  Recent Transformer-based low-light enhancement methods have made promising progress in recovering global illumination.<n>Recent Transformer-based low-light enhancement methods have made promising progress in recovering global illumination.<n>We present a Spatially-Adaptive Illumination-Guided Transformer framework that enables accurate illumination restoration.
  arXiv  Detail & Related papers  (2025-07-21T11:38:56Z)
• EndoFlow-SLAM: Real-Time Endoscopic SLAM with Flow-Constrained Gaussian Splatting [7.7956059927002705]
  We introduce optical flow loss as a geometric constraint, which effectively constrains both the 3D structure of the scene and the camera motion.<n>In addition, to improve scene representation in the SLAM system, we improve the 3DGS refinement strategy by focusing on viewpoints corresponding to Keyframes.<n>Our method outperforms existing state-of-the-art methods in novel view synthesis and pose estimation.
  arXiv  Detail & Related papers  (2025-06-26T16:06:46Z)
• Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesis [49.67420486373202]
  GRGS is a generalizable and relightable 3D Gaussian framework for high-fidelity human novel view synthesis under diverse lighting conditions.<n>We introduce a Lighting-aware Geometry Refinement (LGR) module trained on synthetically relit data to predict accurate depth and surface normals.
  arXiv  Detail & Related papers  (2025-05-27T17:59:47Z)
• Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment [46.60106452798745]
  We introduce Luminance-GS, a novel approach to achieving high-quality novel view synthesis results under challenging lighting conditions using 3DGS.<n>By adopting per-view color matrix mapping and view-adaptive curve adjustments, Luminance-GS achieves state-of-the-art (SOTA) results across various lighting conditions.<n>Compared to previous NeRF- and 3DGS-based baselines, Luminance-GS provides real-time rendering speed with improved reconstruction quality.
  arXiv  Detail & Related papers  (2025-04-02T08:54:57Z)
• LITA-GS: Illumination-Agnostic Novel View Synthesis via Reference-Free 3D Gaussian Splatting and Physical Priors [14.160637565120231]
  LITA-GS is a novel illumination-agnostic novel view synthesis method via reference-free 3DGS and physical priors.<n>We develop the lighting-agnostic structure rendering strategy, which facilitates the optimization of the scene structure and object appearance.<n>We adopt the unsupervised strategy for the training of LITA-GS and extensive experiments demonstrate that LITA-GS surpasses the state-of-the-art (SOTA) NeRF-based method.
  arXiv  Detail & Related papers  (2025-03-31T20:44:39Z)
• D3DR: Lighting-Aware Object Insertion in Gaussian Splatting [48.80431740983095]
  We propose a method, dubbed D3DR, for inserting a 3DGS-parametrized object into 3DGS scenes.<n>We leverage advances in diffusion models, which, trained on real-world data, implicitly understand correct scene lighting.<n>We demonstrate the method's effectiveness by comparing it to existing approaches.
  arXiv  Detail & Related papers  (2025-03-09T19:48:00Z)
• LumiGauss: Relightable Gaussian Splatting in the Wild [15.11759492990967]
  We introduce LumiGauss - a technique that tackles 3D reconstruction of scenes and environmental lighting through 2D Gaussian Splatting.<n>Our approach yields high-quality scene reconstructions and enables realistic lighting synthesis under novel environment maps.<n>We validate our method on the NeRF-OSR dataset, demonstrating superior performance over baseline methods.
  arXiv  Detail & Related papers  (2024-08-06T23:41:57Z)
• Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion [54.197343533492486]
  Event3DGS can reconstruct high-fidelity 3D structure and appearance under high-speed egomotion. Experiments on multiple synthetic and real-world datasets demonstrate the superiority of Event3DGS compared with existing event-based dense 3D scene reconstruction frameworks. Our framework also allows one to incorporate a few motion-blurred frame-based measurements into the reconstruction process to further improve appearance fidelity without loss of structural accuracy.
  arXiv  Detail & Related papers  (2024-06-05T06:06:03Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)
• Low-Light Image Enhancement with Illumination-Aware Gamma Correction and Complete Image Modelling Network [69.96295927854042]
  Low-light environments usually lead to less informative large-scale dark areas. We propose to integrate the effectiveness of gamma correction with the strong modelling capacities of deep networks. Because exponential operation introduces high computational complexity, we propose to use Taylor Series to approximate gamma correction.
  arXiv  Detail & Related papers  (2023-08-16T08:46:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.