GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions

• URL: http://arxiv.org/abs/2311.16037v2
• Date: Wed, 24 Jul 2024 13:16:38 GMT
• Title: GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions
• Authors: Junjie Wang, Jiemin Fang, Xiaopeng Zhang, Lingxi Xie, Qi Tian,
• Abstract summary: We propose a systematic framework, named GaussianEditor, to edit 3D scenes delicately via 3D Gaussians with text instructions. Our framework can achieve more delicate and precise editing of 3D scenes than previous methods while enjoying much faster training speed.
• Score: 90.38892097863814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, impressive results have been achieved in 3D scene editing with text instructions based on a 2D diffusion model. However, current diffusion models primarily generate images by predicting noise in the latent space, and the editing is usually applied to the whole image, which makes it challenging to perform delicate, especially localized, editing for 3D scenes. Inspired by recent 3D Gaussian splatting, we propose a systematic framework, named GaussianEditor, to edit 3D scenes delicately via 3D Gaussians with text instructions. Benefiting from the explicit property of 3D Gaussians, we design a series of techniques to achieve delicate editing. Specifically, we first extract the region of interest (RoI) corresponding to the text instruction, aligning it to 3D Gaussians. The Gaussian RoI is further used to control the editing process. Our framework can achieve more delicate and precise editing of 3D scenes than previous methods while enjoying much faster training speed, i.e. within 20 minutes on a single V100 GPU, more than twice as fast as Instruct-NeRF2NeRF (45 minutes -- 2 hours).

Related papers

• Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting [55.14822004410817]
  We introduce DYG, an effective 3D drag-based editing method for 3D Gaussian Splatting. It enables precise control over the extent of editing through the input of 3D masks and pairs of control points. DYG integrates the strengths of the implicit triplane representation to establish the geometric scaffold of the editing results.
  arXiv  Detail & Related papers  (2025-01-30T18:51:54Z)
• GaussianAvatar-Editor: Photorealistic Animatable Gaussian Head Avatar Editor [53.58201080040624]
  We introduce an innovative framework for text-driven editing of animatable Gaussian head avatars. GaussianAvatar-Editor can be fully controlled in expression, pose, and viewpoint.
  arXiv  Detail & Related papers  (2025-01-17T06:40:20Z)
• 3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting [31.98493679748211]
  We propose 3DSceneEditor, a fully 3D-based paradigm for real-time, precise editing of 3D scenes using Gaussian Splatting. Unlike conventional methods, 3DSceneEditor operates through a streamlined 3D pipeline, enabling direct manipulation of Gaussians for efficient, high-quality edits.
  arXiv  Detail & Related papers  (2024-12-02T15:03:55Z)
• GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization [11.170354299559998]
  We propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. We introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model.
  arXiv  Detail & Related papers  (2024-11-15T08:25:14Z)
• DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation [57.406031264184584]
  DragGaussian is a 3D object drag-editing framework based on 3D Gaussian Splatting. Our contributions include the introduction of a new task, the development of DragGaussian for interactive point-based 3D editing, and comprehensive validation of its effectiveness through qualitative and quantitative experiments.
  arXiv  Detail & Related papers  (2024-05-09T14:34:05Z)
• DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing [72.54566271694654]
  We consider the problem of editing 3D objects and scenes based on open-ended language instructions. A common approach to this problem is to use a 2D image generator or editor to guide the 3D editing process. This process is often inefficient due to the need for iterative updates of costly 3D representations.
  arXiv  Detail & Related papers  (2024-04-29T17:59:30Z)
• GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting [66.08674785436612]
  3D editing plays a crucial role in many areas such as gaming and virtual reality. Traditional 3D editing methods, which rely on representations like meshes and point clouds, often fall short in realistically depicting complex scenes. Our paper presents GaussianEditor, an innovative and efficient 3D editing algorithm based on Gaussian Splatting (GS), a novel 3D representation.
  arXiv  Detail & Related papers  (2023-11-24T14:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.