LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction

• URL: http://arxiv.org/abs/2406.16073v1
• Date: Sun, 23 Jun 2024 10:49:39 GMT
• Title: LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction
• Authors: Hengyu Liu, Yifan Liu, Chenxin Li, Wuyang Li, Yixuan Yuan,
• Abstract summary: We introduce a Lightweight 4D Gaussian Splatting framework (LGS) for dynamic endoscopic reconstruction. To minimize the redundancy of Gaussian quantities, we propose Deformation-Aware Pruning. We also simplify the representation of textures and lighting in non-crucial areas by pruning the dimensions of Gaussian attributes.
• Score: 33.794584735264884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of 3D Gaussian Splatting (3D-GS) techniques and their dynamic scene modeling variants, 4D-GS, offers promising prospects for real-time rendering of dynamic surgical scenarios. However, the prerequisite for modeling dynamic scenes by a large number of Gaussian units, the high-dimensional Gaussian attributes and the high-resolution deformation fields, all lead to serve storage issues that hinder real-time rendering in resource-limited surgical equipment. To surmount these limitations, we introduce a Lightweight 4D Gaussian Splatting framework (LGS) that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction. Specifically, to minimize the redundancy of Gaussian quantities, we propose Deformation-Aware Pruning by gauging the impact of each Gaussian on deformation. Concurrently, to reduce the redundancy of Gaussian attributes, we simplify the representation of textures and lighting in non-crucial areas by pruning the dimensions of Gaussian attributes. We further resolve the feature field redundancy caused by the high resolution of 4D neural spatiotemporal encoder for modeling dynamic scenes via a 4D feature field condensation. Experiments on public benchmarks demonstrate efficacy of LGS in terms of a compression rate exceeding 9 times while maintaining the pleasing visual quality and real-time rendering efficiency. LGS confirms a substantial step towards its application in robotic surgical services.

Related papers

• 4DRGS: 4D Radiative Gaussian Splatting for Efficient 3D Vessel Reconstruction from Sparse-View Dynamic DSA Images [49.170407434313475]
  Existing methods often produce suboptimal results or require excessive computation time. We propose 4D radiative Gaussian splatting (4DRGS) to achieve high-quality reconstruction efficiently. 4DRGS achieves impressive results in 5 minutes training, which is 32x faster than the state-of-the-art method.
  arXiv  Detail & Related papers  (2024-12-17T13:51:56Z)
• Temporally Compressed 3D Gaussian Splatting for Dynamic Scenes [46.64784407920817]
  Temporally Compressed 3D Gaussian Splatting (TC3DGS) is a novel technique designed specifically to compress dynamic 3D Gaussian representations. Our experiments across multiple datasets demonstrate that TC3DGS achieves up to 67$times$ compression with minimal or no degradation in visual quality.
  arXiv  Detail & Related papers  (2024-12-07T17:03:09Z)
• 4D Scaffold Gaussian Splatting for Memory Efficient Dynamic Scene Reconstruction [27.455934322535853]
  This paper proposes a 4D anchor-based framework that retains visual quality and rendering speed of 4D Gaussians while reducing storage costs. Experimental results demonstrate that our method achieves state-of-the-art visual quality and 97.8% storage reduction over 4DGS.
  arXiv  Detail & Related papers  (2024-11-26T02:22:07Z)
• MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes [49.36091070642661]
  This paper introduces a memory-efficient framework for 4DGS. It achieves a storage reduction by approximately 190$times$ and 125$times$ on the Technicolor and Neural 3D Video datasets. It maintains comparable rendering speeds and scene representation quality, setting a new standard in the field.
  arXiv  Detail & Related papers  (2024-10-17T14:47:08Z)
• L3DG: Latent 3D Gaussian Diffusion [74.36431175937285]
  L3DG is the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. We employ a sparse convolutional architecture to efficiently operate on room-scale scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time.
  arXiv  Detail & Related papers  (2024-10-17T13:19:32Z)
• EfficientGS: Streamlining Gaussian Splatting for Large-Scale High-Resolution Scene Representation [29.334665494061113]
  'EfficientGS' is an advanced approach that optimize 3DGS for high-resolution, large-scale scenes. We analyze the densification process in 3DGS and identify areas of Gaussian over-proliferation. We propose a selective strategy, limiting Gaussian increase to key redundant primitives, thereby enhancing the representational efficiency.
  arXiv  Detail & Related papers  (2024-04-19T10:32:30Z)
• SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer [57.506654943449796]
  We propose an efficient, sparse-controlled video-to-4D framework named SC4D that decouples motion and appearance. Our method surpasses existing methods in both quality and efficiency. We devise a novel application that seamlessly transfers motion onto a diverse array of 4D entities.
  arXiv  Detail & Related papers  (2024-04-04T18:05:18Z)
• Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstruction [89.53963284958037]
  We propose a novel motion-aware enhancement framework for dynamic scene reconstruction. Specifically, we first establish a correspondence between 3D Gaussian movements and pixel-level flow. For the prevalent deformation-based paradigm that presents a harder optimization problem, a transient-aware deformation auxiliary module is proposed.
  arXiv  Detail & Related papers  (2024-03-18T03:46:26Z)
• EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction [36.35631592019182]
  We introduce EndoGaussian, a real-time endoscopic scene reconstruction framework built on 3D Gaussian Splatting (3DGS) Our framework significantly boosts the rendering speed to a real-time level. Experiments on public datasets demonstrate our efficacy against prior SOTAs in many aspects.
  arXiv  Detail & Related papers  (2024-01-23T08:44:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.