DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting

• URL: http://arxiv.org/abs/2607.04761v1
• Date: Mon, 06 Jul 2026 07:56:17 GMT
• Title: DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting
• Abstract summary: Existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping.<n>We propose DeGenseGS, Geometric Understandingally and Semantically Decoupled Scene in 4D Splatting, a novel framework that independently models semantic evolution and geometric deformation.<n>Our framework yields enhanced geometric robustness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.
• Score: 17.162560675768642
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time, text-promptable 4D reconstruction is indispensable for autonomous surgical interaction. Severe misalignment between semantic meaning and physical anatomy still persists, largely because existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping. In this paper, we propose DeGenseGS, Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting, a novel framework that independently models semantic evolution and geometric deformation. Specifically, we propose a HexPlane-based spatiotemporal entanglement module that uses shared kinematic latents to synchronize semantic mutations with scene dynamics, while explicitly disentangling semantic updates from geometric deformation. To further ensure robustness against reconstruction artifacts, we devise a Rasterization-Native Semantic Extraction mechanism that infers semantics from topologically continuous feature maps. Additionally, we incorporate an angular-aligned optimization strategy that conforms to the native hyperspherical latent space, thereby preventing semantic distortion. Extensive evaluations on the CholecSeg8k and EndoVis18 datasets demonstrate that DeGenseGS achieves state-of-the-art performance. Our framework yields enhanced geometric completeness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.

Related papers

• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer [64.5765465030666]
  We propose IGGT4D, a streaming instance-grounded geometry Transformer for online 4D scene understanding.<n> IGGT4D processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally updates a unified representation of camera motion, geometry, and object identity.<n> Experiments on 3D reconstruction, pose estimation, instance spatial tracking, and open-vocabulary segmentation demonstrate that IGGT4D outperforms existing streaming baselines.
  arXiv  Detail & Related papers  (2026-07-21T16:00:01Z)
• Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images [62.31892851760248]
  COVScene is a pose-free semantic Gaussian framework that couples renderable primitives with a dense semantic occupancy field.<n>We show that COVScene maintains competitive rendering quality, improves open-vocabulary segmentation, and achieves stronger semantic occupancy prediction than the self-supervised baseline.
  arXiv  Detail & Related papers  (2026-07-02T03:00:23Z)
• OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention [80.51557267896938]
  We present OpenGaFF, a novel framework for open-vocabulary 3D scene understanding built upon 3D Gaussian Splatting.<n>At the core of our method is a Gaussian Feature Field that models semantics as a continuous function of Gaussian geometry and appearance.<n>To further enforce object-level semantic consistency, we introduce a structured codebook that serves as a set of shared semantic primitives.
  arXiv  Detail & Related papers  (2026-05-07T12:10:07Z)
• Video Analysis and Generation via a Semantic Progress Function [75.07700965101854]
  We introduce a one-dimensional representation that captures how the meaning of a given sequence evolves over time.<n>For each frame, we compute distances between semantic embeddings and fit a smooth curve that reflects the cumulative semantic shift across the sequence.<n>Building on this insight, we propose a semantic linearization procedure that re parameterizes the sequence so that semantic change unfolds at a constant rate.
  arXiv  Detail & Related papers  (2026-04-24T13:48:51Z)
• \textit{4DSurf}: High-Fidelity Dynamic Scene Surface Reconstruction [53.82595920033838]
  This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS)<n>We propose textit4DSurf'', a novel and unified framework for generic dynamic surface reconstruction.<n>Our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance.
  arXiv  Detail & Related papers  (2026-03-30T06:09:37Z)
• SegSplat: Feed-forward Gaussian Splatting and Open-Set Semantic Segmentation [114.57192386025373]
  SegSplat is a novel framework designed to bridge the gap between rapid, feed-forward 3D reconstruction and rich, open-vocabulary semantic understanding.<n>This work represents a significant step towards practical, on-the-fly generation of semantically aware 3D environments.
  arXiv  Detail & Related papers  (2025-11-23T10:26:38Z)
• AlignGS: Aligning Geometry and Semantics for Robust Indoor Reconstruction from Sparse Views [18.361136390711415]
  The demand for semantically rich 3D models of indoor scenes is rapidly growing, driven by applications in augmented reality, virtual reality, and robotics.<n>Existing methods often treat semantics as a passive feature painted on an already-formed, and potentially flawed, geometry.<n>This paper introduces AlignGS, a novel framework that actualizes this vision by pioneering a synergistic, end-to-end optimization of geometry and semantics.
  arXiv  Detail & Related papers  (2025-10-09T06:30:20Z)
• Self-Supervised Multi-Part Articulated Objects Modeling via Deformable Gaussian Splatting and Progressive Primitive Segmentation [23.18517560629462]
  We introduce DeGSS, a unified framework that encodes articulated objects as deformable 3D Gaussian fields, embedding geometry, appearance, and motion in one compact representation.<n>To evaluate generalization and realism, we enlarge the synthetic PartNet-Mobility benchmark and release RS-Art, a real-to-sim dataset that pairs RGB captures with accurately reverse-engineered 3D models.
  arXiv  Detail & Related papers  (2025-06-11T12:32:16Z)
• Geometry-Editable and Appearance-Preserving Object Compositon [67.98806888489385]
  General object composition (GOC) aims to seamlessly integrate a target object into a background scene with desired geometric properties.<n>Recent approaches derive semantic embeddings and integrate them into advanced diffusion models to enable geometry-editable generation.<n>We introduce a Disentangled Geometry-editable and Appearance-preserving Diffusion model that first leverages semantic embeddings to implicitly capture desired geometric transformations.
  arXiv  Detail & Related papers  (2025-05-27T09:05:28Z)
• NF3DM: Combining Neural Fields and Deformation Models for 3D Non-Rigid Motion Reconstruction [6.851721795186258]
  We introduce a novel, data-driven approach for reconstructing temporally coherent 3D motion from unstructured observations of non-rigidly deforming shapes.<n>Our goal is to achieve high-fidelity motion reconstructions for shapes that undergo near-isometric deformations, such as humans wearing loose clothing.<n>Our method outperforms state-of-the-art approaches, as demonstrated by its application to human and animal motion sequences reconstructed from monocular depth videos.
  arXiv  Detail & Related papers  (2024-12-11T16:24:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.