SceneCraft: Interactive System for Image Editing via Scene Graph

• URL: http://arxiv.org/abs/2606.16103v1
• Date: Mon, 15 Jun 2026 01:34:22 GMT
• Title: SceneCraft: Interactive System for Image Editing via Scene Graph
• Abstract summary: SceneCraft is a novel framework that bridges user intent and model execution by representing images as editable scene graphs.<n>Instead of guessing text prompts through trial and error, users interact directly with a visual graph to perform complex spatial and relational operations.<n>We show that SceneCraft provides a more intuitive control mechanism, significantly reducing the cognitive burden of manual prompt engineering.
• Score: 17.007784605446716
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in generative AI have enabled natural language-driven image editing, yet existing systems often fail in complex scenes with multiple interacting objects because they rely heavily on users crafting precise text prompts. To address the absence of structured control, we propose SceneCraft, a novel interactive framework that bridges user intent and model execution by representing images as editable scene graphs. Instead of guessing text prompts through trial and error, users interact directly with a visual graph to perform complex spatial and relational operations. These graph modifications are automatically translated into precise, context-aware editing prompts, effectively eliminating linguistic ambiguity. To ensure robust and diverse results, structured prompts are dispatched to multiple state-of-the-art generative models. Evaluations across diverse editing scenarios show that SceneCraft provides a more intuitive control mechanism, significantly reducing the cognitive burden of manual prompt engineering while generating outputs that users consistently rate as higher in quality and fidelity.

Related papers

• Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generation [19.404516863003057]
  INSET is a unified generation model that seamlessly embeds images as native vocabulary within textual instructions.<n>We introduce a scalable data engine that synthesizes 15M high-quality interleaved samples from standard image and video datasets.<n>Results on InterleaveBench demonstrate that INSET significantly outperforms state-of-the-art methods in multi-image consistency and text alignment.
  arXiv  Detail & Related papers  (2026-05-12T15:54:49Z)
• ScribbleEdit: Synthetic Data for Image Editing with Scribbles and Text [26.951462580463883]
  We introduce ScribbleEdit, a large-scale synthetic dataset designed to bridge this gap.<n>We evaluate and finetune both diffusion-based and autoregressive unified multimodal image editing models.<n>Our experiments reveal that while off-the-shelf models struggle with abstract scribble inputs, finetuning on our synthetic dataset significantly improves their ability to generate spatially aligned and semantically consistent edits.
  arXiv  Detail & Related papers  (2026-05-01T22:26:42Z)
• I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing [59.434028565445885]
  I2E is a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment.<n>I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions.<n>I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.
  arXiv  Detail & Related papers  (2026-01-07T09:29:57Z)
• Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing [32.36623938012829]
  Chain-of-Thought (CoT) is limited in its ability to represent intricate visual layouts.<n>We propose Multimodal Reasoning Edit (MURE) to shift the visual editing process from purely text-based reasoning to a series of interleaved rationales.<n>We release the first CoT-Edit-14K dataset, comprising 14K high-quality editing examples.
  arXiv  Detail & Related papers  (2025-10-09T12:36:51Z)
• Text-Driven Image Editing via Learnable Regions [74.45313434129005]
  We introduce a method for region-based image editing driven by textual prompts, without the need for user-provided masks or sketches. We show that this simple approach enables flexible editing that is compatible with current image generation models. Experiments demonstrate the competitive performance of our method in manipulating images with high fidelity and realism that correspond to the provided language descriptions.
  arXiv  Detail & Related papers  (2023-11-28T02:27:31Z)
• Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing [23.00202969969574]
  We propose Dynamic Prompt Learning (DPL) to force cross-attention maps to focus on correct noun words in the text prompt. We show improved prompt editing results for Word-Swap, Prompt Refinement, and Attention Re-weighting, especially for complex multi-object scenes.
  arXiv  Detail & Related papers  (2023-09-27T13:55:57Z)
• Prompt-to-Prompt Image Editing with Cross Attention Control [41.26939787978142]
  We present an intuitive prompt-to-prompt editing framework, where the edits are controlled by text only. We show our results over diverse images and prompts, demonstrating high-quality synthesis and fidelity to the edited prompts.
  arXiv  Detail & Related papers  (2022-08-02T17:55:41Z)
• Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors [58.71128866226768]
  Recent text-to-image generation methods have incrementally improved the generated image fidelity and text relevancy. We propose a novel text-to-image method that addresses these gaps by (i) enabling a simple control mechanism complementary to text in the form of a scene. Our model achieves state-of-the-art FID and human evaluation results, unlocking the ability to generate high fidelity images in a resolution of 512x512 pixels.
  arXiv  Detail & Related papers  (2022-03-24T15:44:50Z)
• SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning [61.57887011165744]
  multimodal Transformers have made great progress in the task of Visual Commonsense Reasoning. We propose a Scene Graph Enhanced Image-Text Learning framework to incorporate visual scene graphs in commonsense reasoning.
  arXiv  Detail & Related papers  (2021-12-16T03:16:30Z)
• HairCLIP: Design Your Hair by Text and Reference Image [100.85116679883724]
  This paper proposes a new hair editing interaction mode, which enables manipulating hair attributes individually or jointly. We encode the image and text conditions in a shared embedding space and propose a unified hair editing framework. With the carefully designed network structures and loss functions, our framework can perform high-quality hair editing.
  arXiv  Detail & Related papers  (2021-12-09T18:59:58Z)
• Adjusting Image Attributes of Localized Regions with Low-level Dialogue [83.06971746641686]
  We develop a task-oriented dialogue system to investigate low-level instructions for NLIE. Our system grounds language on the level of edit operations, and suggests options for a user to choose from. An analysis shows that users generally adapt to utilizing the proposed low-level language interface.
  arXiv  Detail & Related papers  (2020-02-11T20:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.