Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing

• URL: http://arxiv.org/abs/2603.17583v1
• Date: Wed, 18 Mar 2026 10:46:42 GMT
• Title: Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing
• Abstract summary: Edit-As-Act is a framework that performs open-vocabulary scene editing as goal-regressive planning in 3D space.<n>A language-driven planner proposes actions, and a validator enforces goal-directedness, monotonicity, and physical feasibility.<n>On E2A-Bench, our benchmark of 63 editing tasks across 9 indoor environments, Edit-As-Act significantly outperforms prior approaches.
• Score: 20.022591860394012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Editing a 3D indoor scene from natural language is conceptually straightforward but technically challenging. Existing open-vocabulary systems often regenerate large portions of a scene or rely on image-space edits that disrupt spatial structure, resulting in unintended global changes or physically inconsistent layouts. These limitations stem from treating editing primarily as a generative task. We take a different view. A user instruction defines a desired world state, and editing should be the minimal sequence of actions that makes this state true while preserving everything else. This perspective motivates Edit-As-Act, a framework that performs open-vocabulary scene editing as goal-regressive planning in 3D space. Given a source scene and free-form instruction, Edit-As-Act predicts symbolic goal predicates and plans in EditLang, a PDDL-inspired action language that we design with explicit preconditions and effects encoding support, contact, collision, and other geometric relations. A language-driven planner proposes actions, and a validator enforces goal-directedness, monotonicity, and physical feasibility, producing interpretable and physically coherent transformations. By separating reasoning from low-level generation, Edit-As-Act achieves instruction fidelity, semantic consistency, and physical plausibility - three criteria that existing paradigms cannot satisfy together. On E2A-Bench, our benchmark of 63 editing tasks across 9 indoor environments, Edit-As-Act significantly outperforms prior approaches across all edit types and scene categories.

Related papers

• What Makes a 3D Scene Editable? A Factorized Benchmark of Fidelity, Locality, Consistency, and Preservation [0.0]
  We introduce EditBench3D, a representation-agnostic benchmark that treats editing as controlled information replacement.<n>We evaluate eight representative NeRF, 3D Gaussian Splatting, hybrid, and proxy-based editors on 240 scene-edit pairs.
  arXiv  Detail & Related papers  (2026-09-14T01:38:38Z)
• Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models [57.59888019135289]
  Hash-Atlas network reformulates 3D scene editing as operations on 2D atlas images.<n>We introduce a dialogue-based 3D scene editing approach, termed CE3D++.<n>Results demonstrate that CE3D++ effectively integrates multiple visual models to achieve diverse visual editing effects.
  arXiv  Detail & Related papers  (2026-08-29T08:25:58Z)
• DesignAgent3D: Interactive 3D Scene Editing via Designer-like Multimodal Reasoning [32.91291969659809]
  Text guided 3D scene editing provides an intuitive interface for modifying reconstructed environments.<n>Existing methods typically formulate the task as one-shot conditional generation from a single prompt.<n>We present DesignAgent3D, an interactive multimodal agentic framework that reformulates 3D scene editing as a designer-like Plan-Perceive-Act paradigm.
  arXiv  Detail & Related papers  (2026-08-18T01:16:13Z)
• Geometry-Instructed Video Editing [54.86319853573801]
  GIVE is a geometry-instructed video editing framework that represents edits through a unified object-state formulation.<n>We build a scalable graphics-engine pipeline that executes object-level edit programs and renders controlled before/after pairs.<n> Experimental results demonstrate that GIVE produces faithful geometric edits with temporal coherence and consistent secondary effects.
  arXiv  Detail & Related papers  (2026-06-23T07:11:45Z)
• TASE: Truncation-Aware Semantic Embeddings for 3D Scene Understanding and Editing [25.304236821424198]
  High-fidelity semantic 3D scene representations are crucial for numerous applications, including robotics, autonomous driving, and simulation.<n>Current approaches provide limited support for controllable editing.<n>We introduce TASE, a method that projects pretrained 2D semantic features into a truncation-aware embedding space.
  arXiv  Detail & Related papers  (2026-06-02T08:25:53Z)
• VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction [59.303842406260124]
  VGGT-Edit is a feed-forward framework for text-conditioned native 3D scene editing.<n>We introduce depth-synchronized text injection to align semantic guidance with the backbone's spatial poses.<n> VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed.
  arXiv  Detail & Related papers  (2026-05-14T17:59:04Z)
• CEI-3D: Collaborative Explicit-Implicit 3D Reconstruction for Realistic and Fine-Grained Object Editing [51.73433734209541]
  Existing 3D editing methods often produce unrealistic and unrefined results due to the deeply integrated nature of their reconstruction networks.<n>This paper introduces CEI-3D, an editing-oriented reconstruction pipeline designed to facilitate realistic and fine-grained editing.<n>Our approach achieves more realistic and fine-grained editing results than the state-of-the-art (SOTA) methods while requiring less editing time.
  arXiv  Detail & Related papers  (2026-03-12T11:15:11Z)
• InterCoG: Towards Spatially Precise Image Editing with Interleaved Chain-of-Grounding Reasoning [60.799998743918955]
  We propose a novel text-vision Interleaved Chain-of-Grounding reasoning framework for fine-grained image editing in complex real-world scenes.<n>The key insight of InterCoG is to first perform object position reasoning solely within text.<n>We also propose two auxiliary training modules: multimodal grounding reconstruction supervision and multimodal grounding reasoning alignment.
  arXiv  Detail & Related papers  (2026-03-02T08:13:16Z)
• I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing [59.434028565445885]
  I2E is a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment.<n>I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions.<n>I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.
  arXiv  Detail & Related papers  (2026-01-07T09:29:57Z)
• ReSpace: Text-Driven 3D Indoor Scene Synthesis and Editing with Preference Alignment [8.954070942391603]
  ReSpace is a generative framework for text-driven 3D indoor scene synthesis and editing.<n>We leverage a dual-stage training approach combining supervised fine-tuning and preference alignment.<n>For scene editing, we employ a zero-shot LLM to handle object removal and prompts for addition.
  arXiv  Detail & Related papers  (2025-06-03T05:22:04Z)
• EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing [114.14164860467227]
  We propose EditRoom, a framework capable of executing a variety of layout edits through natural language commands.<n>Specifically, EditRoom leverages Large Language Models (LLMs) for command planning and generates target scenes.<n>We have developed an automatic pipeline to augment existing 3D scene datasets and introduced EditRoom-DB, a large-scale dataset with 83k editing pairs.
  arXiv  Detail & Related papers  (2024-10-03T17:42:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.