StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation

• URL: http://arxiv.org/abs/2604.12575v1
• Date: Tue, 14 Apr 2026 10:55:43 GMT
• Title: StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation
• Abstract summary: StructDiff is a generative framework based on a single-scale diffusion model for single-image generation.<n>It incorporates 3D positional encoding (PE) as a spatial prior, allowing flexible control over positions, scale, and local details of generated objects.<n>It also demonstrates broad applicability across downstream tasks, such as text-guided image generation, image editing, outpainting, and paint-to-image synthesis.
• Score: 72.84181869780627
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces StructDiff, a generative framework based on a single-scale diffusion model for single-image generation. Single-image generation aims to synthesize diverse samples with similar visual content to the source image by capturing its internal statistics, without relying on external data. However, existing methods often struggle to preserve the structural layout, especially for images with large rigid objects or strict spatial constraints. Moreover, most approaches lack spatial controllability, making it difficult to guide the structure or placement of generated content. To address these challenges, StructDiff introduces an \textit{adaptive receptive field} module to maintain both global and local distributions. Building on this foundation, StructDiff incorporates 3D positional encoding (PE) as a spatial prior, allowing flexible control over positions, scale, and local details of generated objects. To our knowledge, this spatial control capability represents the first exploration of PE-based manipulation in single-image generation. Furthermore, we propose a novel evaluation criterion for single-image generation based on large language models (LLMs). This criterion specifically addresses the limitations of existing objective metrics and the high labor costs associated with user studies. StructDiff also demonstrates broad applicability across downstream tasks, such as text-guided image generation, image editing, outpainting, and paint-to-image synthesis. Extensive experiments demonstrate that StructDiff outperforms existing methods in structural consistency, visual quality, and spatial controllability. The project page is available at https://butter-crab.github.io/StructDiff/.

Related papers

• Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generation [3.3692935883125954]
  Current flow matching models learn to transport the source i.i.d. Gaussian noise into the target distribution of natural images.<n>Images are fundamentally local since nearby pixels are strongly correlated.<n>We propose StructFlow which encodes spatial locality directly into the source by having the pixels within a small region share a common noise component.
  arXiv  Detail & Related papers  (2026-08-16T00:03:21Z)
• StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling [112.49355973930555]
  Multi-reference image generation aims to synthesize images by integrating attributes from multiple reference images under textual instructions.<n>Existing approaches, which rely solely on natural language instruction, often fail to capture complex intentions precisely.<n>We propose StructGen, which employs a structured, dictionary-like format to encode multiple reference images.
  arXiv  Detail & Related papers  (2026-07-17T04:35:36Z)
• Property-Informed Diffusion-Based Text-to-Microstructure Generation [25.492080916045012]
  A property-informed diffusion-based network is proposed that enables the generation of 3D microstructures directly from textual descriptions.<n>We show that the model is capable of generating semantically meaningful and physically plausible structures across a wide range of material categories.<n>Our approach has good potential for interactive microstructure design and opens up new directions for combining language-based interfaces with inverse material discovery.
  arXiv  Detail & Related papers  (2026-06-06T13:00:34Z)
• HetScene: Heterogeneity-Aware Diffusion for Dense Indoor Scene Generation [25.696780353190253]
  HetScene is a framework to generate controllable and physically plausible indoor scenes for embodied AI.<n>It decomposes objects into primary objects and secondary objects according to their roles in shaping a scene.<n>It generates globally coherent structural layouts with only primary objects conditioned on text descriptions, top-down binary room masks, and spatial relation graphs.
  arXiv  Detail & Related papers  (2026-05-13T14:21:51Z)
• HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation [11.087309945227826]
  We propose a textbfHierarchical textbfControllable (HiCo) diffusion model for layout-to-image generation. Our key insight is to achieve spatial disentanglement through hierarchical modeling of layouts. To evaluate the performance of multi-objective controllable layout generation in natural scenes, we introduce the HiCo-7K benchmark.
  arXiv  Detail & Related papers  (2024-10-18T09:36:10Z)
• A Structure-Guided Diffusion Model for Large-Hole Image Completion [85.61681358977266]
  We develop a structure-guided diffusion model to fill large holes in images. Our method achieves a superior or comparable visual quality compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-18T18:59:01Z)
• Image Inpainting via Conditional Texture and Structure Dual Generation [26.97159780261334]
  We propose a novel two-stream network for image inpainting, which models the structure-constrained texture synthesis and texture-guided structure reconstruction. To enhance the global consistency, a Bi-directional Gated Feature Fusion (Bi-GFF) module is designed to exchange and combine the structure and texture information. Experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2021-08-22T15:44:37Z)
• Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE [74.29384873537587]
  We propose a two-stage model for diverse inpainting, where the first stage generates multiple coarse results each of which has a different structure, and the second stage refines each coarse result separately by augmenting texture. Experimental results on CelebA-HQ, Places2, and ImageNet datasets show that our method not only enhances the diversity of the inpainting solutions but also improves the visual quality of the generated multiple images.
  arXiv  Detail & Related papers  (2021-03-18T05:10:49Z)
• MOGAN: Morphologic-structure-aware Generative Learning from a Single Image [59.59698650663925]
  Recently proposed generative models complete training based on only one image. We introduce a MOrphologic-structure-aware Generative Adversarial Network named MOGAN that produces random samples with diverse appearances. Our approach focuses on internal features including the maintenance of rational structures and variation on appearance.
  arXiv  Detail & Related papers  (2021-03-04T12:45:23Z)
• Person-in-Context Synthesiswith Compositional Structural Space [59.129960774988284]
  We propose a new problem, textbfPersons in Context Synthesis, which aims to synthesize diverse person instance(s) in consistent contexts. The context is specified by the bounding box object layout which lacks shape information, while pose of the person(s) by keypoints which are sparsely annotated. To handle the stark difference in input structures, we proposed two separate neural branches to attentively composite the respective (context/person) inputs into shared compositional structural space'' This structural space is then decoded to the image space using multi-level feature modulation strategy, and learned in a self
  arXiv  Detail & Related papers  (2020-08-28T14:33:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.