Dependency-Aware Discrete Diffusion for Scene Graph Generation

• URL: http://arxiv.org/abs/2605.09065v1
• Date: Sat, 09 May 2026 17:16:20 GMT
• Title: Dependency-Aware Discrete Diffusion for Scene Graph Generation
• Abstract summary: We introduce a dependency-aware, hierarchically constrained discrete diffusion model for scene graph generation.<n>Our approach decouples structure and semantics across the forward and reverse processes, enabling the model to capture conditional dependencies.<n>When fed to downstream image generation, our approach yields improved compositional alignment compared to text-to-image models.
• Score: 8.87774679281067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scene graphs (SGs) represent objects and their relationships as structured graphs, enabling applications in image generation, robotics, and 3D understanding. Recent work suggests that conditioning image generation on scene graphs improves compositional fidelity compared to text-only prompting. However, since users typically provide text rather than structured graphs, a key challenge is to generate scene graphs from natural language. Prior work on discrete diffusion has demonstrated success in generating generic graphs such as molecules and circuits, but fails to account for the hierarchical structure and strong dependencies between objects, edges, and relations in scene graphs. We address this limitation by introducing a dependency-aware, hierarchically constrained discrete diffusion model for scene graph generation. Our approach decouples structure and semantics across the forward and reverse processes, enabling the model to capture conditional dependencies. At inference time, we perform training-free conditioning to sample text-aligned scene graphs. We evaluate our method on standard SG benchmarks and demonstrate improvements over both continuous and discrete graph generation baselines across graph and layout metrics. When fed to downstream image generation, our approach yields improved compositional alignment compared to text-to-image models, particularly in multi-object scenarios.

Related papers

• Semantic-Structural Alignment for Generative Pictorial Charts [59.45629259524998]
  We present a generative framework for the automated synthesis of pictorial charts.<n>We frame the problem as a dual-conditioned generation task guided by two parallel external control signals.<n>Our method produces pictorial charts that are both artistically compelling and structurally consistent.
  arXiv  Detail & Related papers  (2026-05-05T05:20:46Z)
• SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance [46.77060502803466]
  We introduce the Scene Graph Adapter(SG-Adapter), leveraging the structured representation of scene graphs to rectify inaccuracies in the original text embeddings. The SG-Adapter's explicit and non-fully connected graph representation greatly improves the fully connected, transformer-based text representations.
  arXiv  Detail & Related papers  (2024-05-24T08:00:46Z)
• From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models [81.92098140232638]
  Scene graph generation (SGG) aims to parse a visual scene into an intermediate graph representation for downstream reasoning tasks. Existing methods struggle to generate scene graphs with novel visual relation concepts. We introduce a new open-vocabulary SGG framework based on sequence generation.
  arXiv  Detail & Related papers  (2024-04-01T04:21:01Z)
• Image Synthesis with Graph Conditioning: CLIP-Guided Diffusion Models for Scene Graphs [0.0]
  We introduce a novel approach to generate images from scene graphs. We leverage pre-trained text-to-image diffusion models and CLIP guidance to translate graph knowledge into images. Elaborate experiments reveal that our method outperforms existing methods on standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-25T11:46:31Z)
• Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models [35.22309759777998]
  We introduce a framework for joint grounded scene graph - image generation.<n> DiffuseSG is a novel diffusion model that jointly models the heterogeneous node and edge attributes.<n>Our model outperforms existing methods in grounded scene graph generation on the VG and COCO-Stuff datasets.
  arXiv  Detail & Related papers  (2024-01-02T10:10:29Z)
• Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training [112.94542676251133]
  We propose to learn scene graph embeddings by directly optimizing their alignment with images. Specifically, we pre-train an encoder to extract both global and local information from scene graphs. The resulting method, called SGDiff, allows for the semantic manipulation of generated images by modifying scene graph nodes and connections.
  arXiv  Detail & Related papers  (2022-11-21T01:11:19Z)
• Scene Graph Modification as Incremental Structure Expanding [61.84291817776118]
  We focus on scene graph modification (SGM), where the system is required to learn how to update an existing scene graph based on a natural language query. We frame SGM as a graph expansion task by introducing the incremental structure expanding (ISE) We construct a challenging dataset that contains more complicated queries and larger scene graphs than existing datasets.
  arXiv  Detail & Related papers  (2022-09-15T16:26:14Z)
• Iterative Scene Graph Generation [55.893695946885174]
  Scene graph generation involves identifying object entities and their corresponding interaction predicates in a given image (or video) Existing approaches to scene graph generation assume certain factorization of the joint distribution to make the estimation iteration feasible. We propose a novel framework that addresses this limitation, as well as introduces dynamic conditioning on the image.
  arXiv  Detail & Related papers  (2022-07-27T10:37:29Z)
• Unconditional Scene Graph Generation [72.53624470737712]
  We develop a deep auto-regressive model called SceneGraphGen which can learn the probability distribution over labelled and directed graphs. We show that the scene graphs generated by SceneGraphGen are diverse and follow the semantic patterns of real-world scenes.
  arXiv  Detail & Related papers  (2021-08-12T17:57:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.