Surgical Gaussian Surfels: Highly Accurate Real-time Surgical Scene Rendering

• URL: http://arxiv.org/abs/2503.04079v1
• Date: Thu, 06 Mar 2025 04:33:19 GMT
• Title: Surgical Gaussian Surfels: Highly Accurate Real-time Surgical Scene Rendering
• Authors: Idris O. Sunmola, Zhenjun Zhao, Samuel Schmidgall, Yumeng Wang, Paul Maria Scheikl, Axel Krieger,
• Abstract summary: We introduce Surgical Gaussian Surfels (SGS), which transforms anisotropic point primitives into surface-aligned elliptical splats.<n>We evaluate our method on two in-vivo surgical datasets, where it outperforms current state-of-the-art methods in surface geometry, normal map quality, and rendering efficiency.
• Score: 2.205583950882153
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Accurate geometric reconstruction of deformable tissues in monocular endoscopic video remains a fundamental challenge in robot-assisted minimally invasive surgery. Although recent volumetric and point primitive methods based on neural radiance fields (NeRF) and 3D Gaussian primitives have efficiently rendered surgical scenes, they still struggle with handling artifact-free tool occlusions and preserving fine anatomical details. These limitations stem from unrestricted Gaussian scaling and insufficient surface alignment constraints during reconstruction. To address these issues, we introduce Surgical Gaussian Surfels (SGS), which transforms anisotropic point primitives into surface-aligned elliptical splats by constraining the scale component of the Gaussian covariance matrix along the view-aligned axis. We predict accurate surfel motion fields using a lightweight Multi-Layer Perceptron (MLP) coupled with locality constraints to handle complex tissue deformations. We use homodirectional view-space positional gradients to capture fine image details by splitting Gaussian Surfels in over-reconstructed regions. In addition, we define surface normals as the direction of the steepest density change within each Gaussian surfel primitive, enabling accurate normal estimation without requiring monocular normal priors. We evaluate our method on two in-vivo surgical datasets, where it outperforms current state-of-the-art methods in surface geometry, normal map quality, and rendering efficiency, while remaining competitive in real-time rendering performance. We make our code available at https://github.com/aloma85/SurgicalGaussianSurfels

Related papers

• Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields [66.1612475655465]
  3D reconstruction of deformable surfaces from RGB videos is a challenging problem. Existing methods use deformation models with statistical, neural, or physical priors. We propose ThinShell-SfT, a new method for non-rigid 3D tracking meshes.
  arXiv  Detail & Related papers  (2025-03-25T18:00:46Z)
• Geometric Constrained Non-Line-of-Sight Imaging [8.870147903560023]
  We propose a novel joint albedo-surface reconstruction method, which utilizes the Frobenius norm of the shape operator to control the variation rate of the normal field. By improving the accuracy of the normal field, it enhances detail representation and achieves high-precision reconstruction of hidden object geometry.
  arXiv  Detail & Related papers  (2025-03-23T08:56:00Z)
• MonoGSDF: Exploring Monocular Geometric Cues for Gaussian Splatting-Guided Implicit Surface Reconstruction [84.07233691641193]
  We introduce MonoGSDF, a novel method that couples primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction. To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization. Experiments on real-world datasets outperforms prior methods while maintaining efficiency.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction [18.074890506856114]
  We present SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy. Our approach first initialises a Gaussian point cloud using depth priors, employing binary motion masks to identify pixels with significant depth variations and fusing point clouds from depth maps across frames for initialisation. We use the Flexible Deformation Model to represent dynamic scene and introduce a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction.
  arXiv  Detail & Related papers  (2024-10-11T22:46:46Z)
• Neural Octahedral Field: Octahedral prior for simultaneous smoothing and sharp edge regularization [9.167571374234166]
  We propose to guide surface reconstruction under a new variant of neural field, the octahedral field. By simultaneously fitting and smoothing the octahedral field alongside the implicit geometry, it behaves analogously to bilateral filtering. Our method outperforms various traditional and neural approaches across extensive experiments.
  arXiv  Detail & Related papers  (2024-08-01T06:02:59Z)
• Gaussian Primitives for Deformable Image Registration [9.184092856125067]
  Experimental results on brain MRI, lung CT, and cardiac MRI datasets demonstrate that GaussianDIR outperforms existing DIR methods in both accuracy and efficiency. As a training-free approach, it challenges the stereotype that iterative methods are inherently slow and transcend the limitations of poor generalization.
  arXiv  Detail & Related papers  (2024-06-05T15:44:54Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• Mesh-based Gaussian Splatting for Real-time Large-scale Deformation [58.18290393082119]
  It is challenging for users to directly deform or manipulate implicit representations with large deformations in the real-time fashion. We develop a novel GS-based method that enables interactive deformation. Our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate.
  arXiv  Detail & Related papers  (2024-02-07T12:36:54Z)
• EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction [36.35631592019182]
  We introduce EndoGaussian, a real-time endoscopic scene reconstruction framework built on 3D Gaussian Splatting (3DGS) Our framework significantly boosts the rendering speed to a real-time level. Experiments on public datasets demonstrate our efficacy against prior SOTAs in many aspects.
  arXiv  Detail & Related papers  (2024-01-23T08:44:26Z)
• Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement [50.56517624931987]
  The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions. Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry. This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations.
  arXiv  Detail & Related papers  (2023-09-14T12:05:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.