Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation

• URL: http://arxiv.org/abs/2508.20471v1
• Date: Thu, 28 Aug 2025 06:39:53 GMT
• Title: Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation
• Abstract summary: G2Editor is a framework designed for imprecise and precise object editing in driving videos.<n>A scene-level 3D bounding box layout is employed to reconstruct occluded areas of non-target objects.<n>Experiments demonstrate that G2Editor effectively supports object repositioning, insertion and deletion within a unified framework.
• Score: 12.982001613987315
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Corner cases are crucial for training and validating autonomous driving systems, yet collecting them from the real world is often costly and hazardous. Editing objects within captured sensor data offers an effective alternative for generating diverse scenarios, commonly achieved through 3D Gaussian Splatting or image generative models. However, these approaches often suffer from limited visual fidelity or imprecise pose control. To address these issues, we propose G^2Editor, a framework designed for photorealistic and precise object editing in driving videos. Our method leverages a 3D Gaussian representation of the edited object as a dense prior, injected into the denoising process to ensure accurate pose control and spatial consistency. A scene-level 3D bounding box layout is employed to reconstruct occluded areas of non-target objects. Furthermore, to guide the appearance details of the edited object, we incorporate hierarchical fine-grained features as additional conditions during generation. Experiments on the Waymo Open Dataset demonstrate that G^2Editor effectively supports object repositioning, insertion, and deletion within a unified framework, outperforming existing methods in both pose controllability and visual quality, while also benefiting downstream data-driven tasks.

Related papers

• EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning [59.36059865997641]
  We propose EditVerse3D, a novel 3D editing framework that enables high-quality object editing under coarse guidance.<n>Our approach takes as input a 3D object to be edited, a coarse 3D bounding box indicating the target region, and a reference 2D image describing the desired modification.<n>To facilitate this editing, we introduce a novel region-aware adaptive loss that emphasizes hard-to-learn regions.
  arXiv  Detail & Related papers  (2026-07-08T09:21:22Z)
• VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction [59.303842406260124]
  VGGT-Edit is a feed-forward framework for text-conditioned native 3D scene editing.<n>We introduce depth-synchronized text injection to align semantic guidance with the backbone's spatial poses.<n> VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed.
  arXiv  Detail & Related papers  (2026-05-14T17:59:04Z)
• SpatialEdit: Benchmarking Fine-Grained Image Spatial Editing [85.49998134609665]
  Image spatial editing performs geometry-driven transformations, allowing precise control over object layout and camera viewpoints.<n>Current models are insufficient for fine-grained spatial manipulations, motivating a dedicated assessment suite.<n>We introduce SpatialEdit-Bench, a complete benchmark that evaluates spatial editing.<n>We construct SpatialEdit-500k, a synthetic dataset generated with a controllable Blender pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:54:42Z)
• ObjectMorpher: 3D-Aware Image Editing via Deformable 3DGS Models [31.723421257306384]
  We present ObjectMorpher, a unified, interactive framework that converts ambiguous 2D edits into geometry-grounded operations.<n>Users drag control points; a graph-based non-rigid deformation with as-rigid-as-possible (ARAP) constraints ensures physically sensible shape and pose changes.<n>Across diverse categories, ObjectMorpher delivers fine-grained, photorealistic edits with superior controllability and efficiency.
  arXiv  Detail & Related papers  (2026-03-30T08:15:29Z)
• HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles [63.88996084630768]
  Controllable driving scene generation is critical for realistic and scalable autonomous driving simulation.<n>We introduce HorizonForge, a unified framework that reconstructs scenes as editable Gaussian Splats and Meshes.<n>Experiments show that Gaussian-Mesh representation delivers substantially higher fidelity than alternative 3D representations.
  arXiv  Detail & Related papers  (2026-02-24T20:03:47Z)
• Free-Form Scene Editor: Enabling Multi-Round Object Manipulation like in a 3D Engine [83.0145525456509]
  We present FFSE, a 3D-aware framework designed to enable intuitive, physically-consistent object editing on real-world images.<n>Unlike previous approaches that either operate in image space or require slow and error-prone 3D reconstruction, FFSE models editing as a sequence of learned 3D transformations.<n>To support learning of multi-round 3D-aware object manipulation, we introduce 3DObjectEditor.
  arXiv  Detail & Related papers  (2025-11-17T18:57:39Z)
• Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting [55.14822004410817]
  We introduce DYG, an effective 3D drag-based editing method for 3D Gaussian Splatting.<n>It enables precise control over the extent of editing through the input of 3D masks and pairs of control points.<n>DYG integrates the strengths of the implicit triplane representation to establish the geometric scaffold of the editing results.
  arXiv  Detail & Related papers  (2025-01-30T18:51:54Z)
• DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes [23.215760822443194]
  DriveEditor is a diffusion-based framework for object editing in driving videos.<n>It offers a unified framework for comprehensive object editing operations, including repositioning, replacement, deletion, and insertion.
  arXiv  Detail & Related papers  (2024-12-27T04:49:36Z)
• 3D Gaussian Editing with A Single Image [19.662680524312027]
  We introduce a novel single-image-driven 3D scene editing approach based on 3D Gaussian Splatting. Our method learns to optimize the 3D Gaussians to align with an edited version of the image rendered from a user-specified viewpoint. Experiments show the effectiveness of our method in handling geometric details, long-range, and non-rigid deformation.
  arXiv  Detail & Related papers  (2024-08-14T13:17:42Z)
• DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation [57.406031264184584]
  DragGaussian is a 3D object drag-editing framework based on 3D Gaussian Splatting. Our contributions include the introduction of a new task, the development of DragGaussian for interactive point-based 3D editing, and comprehensive validation of its effectiveness through qualitative and quantitative experiments.
  arXiv  Detail & Related papers  (2024-05-09T14:34:05Z)
• View-Consistent 3D Editing with Gaussian Splatting [50.6460814430094]
  View-consistent Editing (VcEdit) is a novel framework that seamlessly incorporates 3DGS into image editing processes.<n>By incorporating consistency modules into an iterative pattern, VcEdit proficiently resolves the issue of multi-view inconsistency.
  arXiv  Detail & Related papers  (2024-03-18T15:22:09Z)
• GSEdit: Efficient Text-Guided Editing of 3D Objects via Gaussian Splatting [10.527349772993796]
  We present GSEdit, a pipeline for text-guided 3D object editing based on Gaussian Splatting models. Our method enables the editing of the style and appearance of 3D objects without altering their main details, all in a matter of minutes on consumer hardware.
  arXiv  Detail & Related papers  (2024-03-08T08:42:23Z)
• GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting [66.08674785436612]
  3D editing plays a crucial role in many areas such as gaming and virtual reality. Traditional 3D editing methods, which rely on representations like meshes and point clouds, often fall short in realistically depicting complex scenes. Our paper presents GaussianEditor, an innovative and efficient 3D editing algorithm based on Gaussian Splatting (GS), a novel 3D representation.
  arXiv  Detail & Related papers  (2023-11-24T14:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.