ES3D: Embedding Semantics into 3D Space for Component-Aware Editing

• URL: http://arxiv.org/abs/2608.15749v1
• Date: Sun, 16 Aug 2026 13:58:08 GMT
• Title: ES3D: Embedding Semantics into 3D Space for Component-Aware Editing
• Abstract summary: ES3D is a framework that embeds semantics directly into 3D space.<n>It retrieves editable regions based on semantic cues and uses multiple images as conditions.<n> ES3D produces geometrically consistent and semantically coherent edits.
• Score: 31.12252706485155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing 3D editing methods have made notable progress in controllability, yet they remain limited in several important ways. Most approaches rely on text-driven editing, which struggles to express fine-grained visual changes intended by the user. Moreover, many methods require manually supplied 3D masks or introduce unintended changes to regions that should remain untouched. These limitations largely arise from the absence of fine-grained semantic understanding, making it difficult for existing models to retrieve or modify specific 3D components. We introduce ES3D, a framework that embeds semantics directly into 3D space, enabling component-aware retrieval and editing of a 3D asset conditioned on multiple local reference images and optional text queries. We first construct a 3D semantic embedding by projecting multi-view semantic features into the voxelized space of the asset. We then perform 3D component retrieval by computing feature similarity between the 3D semantic embedding and the semantic embeddings of image or text queries. For editing, we employ a pretrained 3D generative model with an inpainting mechanism to modify the retrieved components guided by user-provided images while preserving the rest of the asset. Overall, ES3D is a 3D editing framework that retrieves editable regions based on semantic cues and uses multiple images as conditions. Extensive experiments demonstrate that ES3D produces geometrically consistent and semantically coherent edits, enabling robust image-based and text-assisted control for 3D editing.

Related papers

• Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models [57.59888019135289]
  Hash-Atlas network reformulates 3D scene editing as operations on 2D atlas images.<n>We introduce a dialogue-based 3D scene editing approach, termed CE3D++.<n>Results demonstrate that CE3D++ effectively integrates multiple visual models to achieve diverse visual editing effects.
  arXiv  Detail & Related papers  (2026-08-29T08:25:58Z)
• EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning [59.36059865997641]
  We propose EditVerse3D, a novel 3D editing framework that enables high-quality object editing under coarse guidance.<n>Our approach takes as input a 3D object to be edited, a coarse 3D bounding box indicating the target region, and a reference 2D image describing the desired modification.<n>To facilitate this editing, we introduce a novel region-aware adaptive loss that emphasizes hard-to-learn regions.
  arXiv  Detail & Related papers  (2026-07-08T09:21:22Z)
• Vinedresser3D: Agentic Text-guided 3D Editing [26.81659566314386]
  Vinedresser3D is an agentic framework for high-quality text-guided 3D editing.<n>It operates directly in the latent space of a native 3D generative model.<n>Experiments on diverse 3D edits demonstrate that Vinedresser3D outperforms prior baselines in both automatic metrics and human preference studies.
  arXiv  Detail & Related papers  (2026-02-23T06:30:36Z)
• Free-Form Scene Editor: Enabling Multi-Round Object Manipulation like in a 3D Engine [83.0145525456509]
  We present FFSE, a 3D-aware framework designed to enable intuitive, physically-consistent object editing on real-world images.<n>Unlike previous approaches that either operate in image space or require slow and error-prone 3D reconstruction, FFSE models editing as a sequence of learned 3D transformations.<n>To support learning of multi-round 3D-aware object manipulation, we introduce 3DObjectEditor.
  arXiv  Detail & Related papers  (2025-11-17T18:57:39Z)
• Towards Scalable and Consistent 3D Editing [32.16698854719098]
  3D editing has wide applications in immersive content creation, digital entertainment, and AR/VR.<n>Unlike 2D editing, it remains challenging due to the need for cross-view consistency, structural fidelity, and fine-grained controllability.<n>We introduce 3DEditVerse, the largest paired 3D editing benchmark to date, comprising 116,309 high-quality training pairs and 1,500 curated test pairs.<n>On the model side, we propose 3DEditFormer, a 3D-structure-preserving conditional transformer.
  arXiv  Detail & Related papers  (2025-10-03T13:34:55Z)
• 3D-LATTE: Latent Space 3D Editing from Textual Instructions [64.77718887666312]
  We propose a training-free editing method that operates within the latent space of a native 3D diffusion model.<n>We guide the edit synthesis by blending 3D attention maps from the generation with the source object.
  arXiv  Detail & Related papers  (2025-08-29T22:51:59Z)
• Chat-Edit-3D: Interactive 3D Scene Editing via Text Prompts [76.73043724587679]
  We propose a dialogue-based 3D scene editing approach, termed CE3D. Hash-Atlas represents 3D scene views, which transfers the editing of 3D scenes onto 2D atlas images. Results demonstrate that CE3D effectively integrates multiple visual models to achieve diverse editing visual effects.
  arXiv  Detail & Related papers  (2024-07-09T13:24:42Z)
• Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts [38.94299662658179]
  We propose a general framework named Progressive3D, which decomposes the entire generation into a series of locally progressive editing steps. We constrain the content change to only occur in regions determined by user-defined region prompts in each editing step. Extensive experiments demonstrate that the proposed Progressive3D framework generates precise 3D content for prompts with complex semantics.
  arXiv  Detail & Related papers  (2023-10-18T08:23:14Z)
• Directional Texture Editing for 3D Models [51.31499400557996]
  ITEM3D is designed for automatic textbf3D object editing according to the text textbfInstructions. Leveraging the diffusion models and the differentiable rendering, ITEM3D takes the rendered images as the bridge of text and 3D representation.
  arXiv  Detail & Related papers  (2023-09-26T12:01:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.