POCI-Diff: Position Objects Consistently and Interactively with 3D-Layout Guided Diffusion

• URL: http://arxiv.org/abs/2601.14056v1
• Date: Tue, 20 Jan 2026 15:13:43 GMT
• Title: POCI-Diff: Position Objects Consistently and Interactively with 3D-Layout Guided Diffusion
• Abstract summary: We propose a diffusion-based approach for Text-to-Image (T2I) generation with consistent and interactive 3D layout control and editing.<n>We introduce a framework for Positioning Objects Consistently and Interactively (POCI-Diff)<n>Our method enables explicit per-object semantic control by binding individual text descriptions to specific 3D bounding boxes.
• Score: 46.97254555348757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a diffusion-based approach for Text-to-Image (T2I) generation with consistent and interactive 3D layout control and editing. While prior methods improve spatial adherence using 2D cues or iterative copy-warp-paste strategies, they often distort object geometry and fail to preserve consistency across edits. To address these limitations, we introduce a framework for Positioning Objects Consistently and Interactively (POCI-Diff), a novel formulation for jointly enforcing 3D geometric constraints and instance-level semantic binding within a unified diffusion process. Our method enables explicit per-object semantic control by binding individual text descriptions to specific 3D bounding boxes through Blended Latent Diffusion, allowing one-shot synthesis of complex multi-object scenes. We further propose a warping-free generative editing pipeline that supports object insertion, removal, and transformation via regeneration rather than pixel deformation. To preserve object identity and consistency across edits, we condition the diffusion process on reference images using IP-Adapter, enabling coherent object appearance throughout interactive 3D editing while maintaining global scene coherence. Experimental results demonstrate that POCI-Diff produces high-quality images consistent with the specified 3D layouts and edits, outperforming state-of-the-art methods in both visual fidelity and layout adherence while eliminating warping-induced geometric artifacts.

Related papers

• Geometry-Instructed Video Editing [54.86319853573801]
  GIVE is a geometry-instructed video editing framework that represents edits through a unified object-state formulation.<n>We build a scalable graphics-engine pipeline that executes object-level edit programs and renders controlled before/after pairs.<n> Experimental results demonstrate that GIVE produces faithful geometric edits with temporal coherence and consistent secondary effects.
  arXiv  Detail & Related papers  (2026-06-23T07:11:45Z)
• Direct 3D-Aware Object Insertion via Decomposed Visual Proxies [101.76006356727015]
  DIRECT (Decomposed Injection for Reference Composition and Target-integration) is a novel framework that integrates interactive pose manipulation with high-fidelity 2D image synthesis.<n>Our method decomposes the insertion conditions into three complementary components: appearance guidance capturing visual details from the reference object, geometry guidance derived from the user-adjusted 3D proxy, and context guidance from the target background.<n>Experiments show that DIRECT outperforms previous methods in both geometric controllability and visual quality.
  arXiv  Detail & Related papers  (2026-06-04T18:00:34Z)
• DreamEdit3D: Personalization of Multi-View Diffusion Models for 3D Editing [47.437239634403426]
  We present a novel personalization method for text-guided 3D editing that enables compositional, object-level control through natural language.<n>Our method successfully transfers the flexibility of 2D personalization to 3D, achieving state-of-the-art edit faithfulness and identity preservation.
  arXiv  Detail & Related papers  (2026-05-16T13:21:22Z)
• VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction [59.303842406260124]
  VGGT-Edit is a feed-forward framework for text-conditioned native 3D scene editing.<n>We introduce depth-synchronized text injection to align semantic guidance with the backbone's spatial poses.<n> VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed.
  arXiv  Detail & Related papers  (2026-05-14T17:59:04Z)
• View-Consistent 3D Scene Editing via Dual-Path Structural Correspondense and Semantic Continuity [11.663455227298122]
  Cross-view inconsistency remains a major bottleneck in text-driven 3D scene editing.<n>We recast multi-view consistent 3D editing from a distributional perspective.<n>We introduce a dual-path consistency mechanism consisting of projection-guided structural guidance and patch-level semantic propagation.
  arXiv  Detail & Related papers  (2026-04-20T04:47:15Z)
• TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking [38.40189748889072]
  Mesh-guided 3DGS editing framework achieves automated, high-fidelity scene transformation.<n>By anchoring video diffusion with explicit 3D geometry, TRACE enables fine-grained, part-level manipulatio.
  arXiv  Detail & Related papers  (2026-04-01T17:51:00Z)
• CEI-3D: Collaborative Explicit-Implicit 3D Reconstruction for Realistic and Fine-Grained Object Editing [51.73433734209541]
  Existing 3D editing methods often produce unrealistic and unrefined results due to the deeply integrated nature of their reconstruction networks.<n>This paper introduces CEI-3D, an editing-oriented reconstruction pipeline designed to facilitate realistic and fine-grained editing.<n>Our approach achieves more realistic and fine-grained editing results than the state-of-the-art (SOTA) methods while requiring less editing time.
  arXiv  Detail & Related papers  (2026-03-12T11:15:11Z)
• Interp3D: Correspondence-aware Interpolation for Generative Textured 3D Morphing [63.141976759536625]
  We propose Interp3D, a training-free framework for textured 3D morphing.<n>It harnesses generative priors and adopts a progressive alignment principle to ensure both geometric fidelity and texture coherence.<n>For comprehensive evaluations, we construct a dedicated dataset, Interp3DData, with graded difficulty levels and assess generation results from fidelity, transition smoothness, and plausibility.
  arXiv  Detail & Related papers  (2026-01-20T16:03:22Z)
• Dragging with Geometry: From Pixels to Geometry-Guided Image Editing [42.176957681367185]
  We propose a novel geometry-guided drag-based image editing method - GeoDrag.<n>Built upon a unified displacement field that jointly encodes 3D geometry and 2D spatial priors, GeoDrag enables coherent, high-fidelity, and structure-consistent editing.
  arXiv  Detail & Related papers  (2025-09-30T03:53:11Z)
• FreeInsert: Personalized Object Insertion with Geometric and Style Control [26.088650452374726]
  We propose a training-free framework that customizes object insertion into arbitrary scenes by leveraging 3D geometric information.<n>The rendered image, serving as geometric control, is combined with style and content control achieved through diffusion adapters.
  arXiv  Detail & Related papers  (2025-09-25T05:26:10Z)
• Training-free Geometric Image Editing on Diffusion Models [53.38549950608886]
  We tackle the task of geometric image editing, where an object within an image is repositioned, reoriented, or reshaped.<n>We propose a decoupled pipeline that separates object transformation, source region inpainting, and target region refinement.<n>Both inpainting and refinement are implemented using a training-free diffusion approach, FreeFine.
  arXiv  Detail & Related papers  (2025-07-31T07:36:00Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• 3DOT: Texture Transfer for 3DGS Objects from a Single Reference Image [31.972069558992946]
  3D texture swapping allows for the customization of 3D object textures.<n>No dedicated method exists, but adapted 2D editing and text-driven 3D editing approaches can serve this purpose.<n>We introduce 3DSwapping, a 3D texture swapping method that integrates progressive generation, view-consistency gradient guidance, and prompt-tuned gradient guidance.
  arXiv  Detail & Related papers  (2025-03-24T16:31:52Z)
• Advancing 3D Gaussian Splatting Editing with Complementary and Consensus Information [4.956066467858058]
  We present a novel framework for enhancing the visual fidelity and consistency of text-guided 3D Gaussian Splatting (3DGS) editing.<n>Our method demonstrates superior performance in rendering quality and view consistency compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-03-14T17:15:26Z)
• Diffusion-Based Attention Warping for Consistent 3D Scene Editing [55.2480439325792]
  We present a novel method for 3D scene editing using diffusion models.<n>Our approach leverages attention features extracted from a single reference image to define the intended edits.<n>Injecting these warped features into other viewpoints enables coherent propagation of edits.
  arXiv  Detail & Related papers  (2024-12-10T23:57:18Z)
• SyncNoise: Geometrically Consistent Noise Prediction for Text-based 3D Scene Editing [58.22339174221563]
  We propose SyncNoise, a novel geometry-guided multi-view consistent noise editing approach for high-fidelity 3D scene editing. SyncNoise synchronously edits multiple views with 2D diffusion models while enforcing multi-view noise predictions to be geometrically consistent. Our method achieves high-quality 3D editing results respecting the textual instructions, especially in scenes with complex textures.
  arXiv  Detail & Related papers  (2024-06-25T09:17:35Z)
• Vox-E: Text-guided Voxel Editing of 3D Objects [14.88446525549421]
  Large scale text-guided diffusion models have garnered significant attention due to their ability to synthesize diverse images. We present a technique that harnesses the power of latent diffusion models for editing existing 3D objects.
  arXiv  Detail & Related papers  (2023-03-21T17:36:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.