GSDeformer: Direct, Real-time and Extensible Cage-based Deformation for 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2405.15491v3
• Date: Thu, 10 Apr 2025 10:29:02 GMT
• Title: GSDeformer: Direct, Real-time and Extensible Cage-based Deformation for 3D Gaussian Splatting
• Authors: Jiajun Huang, Shuolin Xu, Hongchuan Yu, Tong-Yee Lee,
• Abstract summary: We present GSDeformer, a method that enables cage-based deformation on 3D Gaussian Splatting (3DGS).<n>Our approach bridges cage-based deformation and 3DGS by using a proxy point-cloud representation.<n>Experiments demonstrate that GSDeformer delivers superior deformation results compared to existing methods, is robust under extreme deformations, requires no retraining for editing, runs in real-time, and can be extended to other 3DGS variants.
• Score: 8.727176816793179
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GSDeformer, a method that enables cage-based deformation on 3D Gaussian Splatting (3DGS). Our approach bridges cage-based deformation and 3DGS by using a proxy point-cloud representation. This point cloud is generated from 3D Gaussians, and deformations applied to the point cloud are translated into transformations on the 3D Gaussians. To handle potential bending caused by deformation, we incorporate a splitting process to approximate it. Our method does not modify or extend the core architecture of 3D Gaussian Splatting, making it compatible with any trained vanilla 3DGS or its variants. Additionally, we automate cage construction for 3DGS and its variants using a render-and-reconstruct approach. Experiments demonstrate that GSDeformer delivers superior deformation results compared to existing methods, is robust under extreme deformations, requires no retraining for editing, runs in real-time, and can be extended to other 3DGS variants. Project Page: https://jhuangbu.github.io/gsdeformer/

Related papers

• CAGE-GS: High-fidelity Cage Based 3D Gaussian Splatting Deformation [7.218737495375119]
  CAGE-GS is a cage-based 3DGS deformation method that seamlessly aligns a source 3DGS scene with a user-defined target shape. Our approach learns a deformation cage from the target, which guides the geometric transformation of the source scene. Our method is highly flexible, accommodating various target shape representations, including texts, images, point clouds, meshes and 3DGS models.
  arXiv  Detail & Related papers  (2025-04-17T10:00:15Z)
• 3DGS-to-PC: Convert a 3D Gaussian Splatting Scene into a Dense Point Cloud or Mesh [0.552480439325792]
  3DGS-to-PC is capable of transforming 3DGS scenes into dense, high-accuracy point clouds. This package is highly customisable and capability of simple integration into existing 3DGS pipelines.
  arXiv  Detail & Related papers  (2025-01-13T16:52:28Z)
• Sketch-guided Cage-based 3D Gaussian Splatting Deformation [19.39701978836869]
  3D Gaussian Splatting (GS) is one of the most promising novel 3D representations. We present a sketch-guided 3D GS deformation system that allows users to intuitively modify the geometry of a 3D GS model.
  arXiv  Detail & Related papers  (2024-11-19T02:18:19Z)
• 3D Gaussian Editing with A Single Image [19.662680524312027]
  We introduce a novel single-image-driven 3D scene editing approach based on 3D Gaussian Splatting. Our method learns to optimize the 3D Gaussians to align with an edited version of the image rendered from a user-specified viewpoint. Experiments show the effectiveness of our method in handling geometric details, long-range, and non-rigid deformation.
  arXiv  Detail & Related papers  (2024-08-14T13:17:42Z)
• WildGaussians: 3D Gaussian Splatting in the Wild [80.5209105383932]
  We introduce WildGaussians, a novel approach to handle occlusions and appearance changes with 3DGS. We demonstrate that WildGaussians matches the real-time rendering speed of 3DGS while surpassing both 3DGS and NeRF baselines in handling in-the-wild data.
  arXiv  Detail & Related papers  (2024-07-11T12:41:32Z)
• GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction [52.04103235260539]
  We present a diffusion model approach based on Gaussian Splatting representation for 3D object reconstruction from a single view. The model learns to generate 3D objects represented by sets of GS ellipsoids. The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views.
  arXiv  Detail & Related papers  (2024-07-05T03:43:08Z)
• Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh [44.57625460339714]
  We propose a triangular mesh to manipulate 3DGS directly with self-adaptation. Our approach is capable of handling large deformations, local manipulations, and soft body simulations while keeping high-fidelity rendering.
  arXiv  Detail & Related papers  (2024-05-28T04:13:21Z)
• DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus [56.45194233357833]
  We propose DoGaussian, a method that trains 3DGS distributedly. Our method accelerates the training of 3DGS by 6+ times when evaluated on large-scale scenes.
  arXiv  Detail & Related papers  (2024-05-22T19:17:58Z)
• SAGS: Structure-Aware 3D Gaussian Splatting [53.6730827668389]
  We propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene. SAGS reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets.
  arXiv  Detail & Related papers  (2024-04-29T23:26:30Z)
• 3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis [49.352765055181436]
  We propose a 3D geometry-aware deformable Gaussian Splatting method for dynamic view synthesis. Our solution achieves 3D geometry-aware deformation modeling, which enables improved dynamic view synthesis and 3D dynamic reconstruction.
  arXiv  Detail & Related papers  (2024-04-09T12:47:30Z)
• Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting [9.90835990611019]
  3D Gaussian Splatting (3DGS) provides fast and high-quality novel view synthesis. It is a natural extension to deform a canonical 3DGS to multiple frames for representing a dynamic scene. Previous works fail to accurately reconstruct complex dynamic scenes.
  arXiv  Detail & Related papers  (2024-04-04T17:34:41Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)
• Mesh-based Gaussian Splatting for Real-time Large-scale Deformation [58.18290393082119]
  It is challenging for users to directly deform or manipulate implicit representations with large deformations in the real-time fashion. We develop a novel GS-based method that enables interactive deformation. Our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate.
  arXiv  Detail & Related papers  (2024-02-07T12:36:54Z)
• SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.80822249039235]
  3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis. We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS. Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
  arXiv  Detail & Related papers  (2024-01-31T14:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.