Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generation

• URL: http://arxiv.org/abs/2605.31266v1
• Date: Fri, 29 May 2026 13:00:00 GMT
• Title: Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generation
• Abstract summary: We propose a representation-driven framework that disentangles semantics from primitives for robust few-shot adaptation.<n>Semantic Anchoring aggregates categorical semantics into anchors for stable identity, while Primitive Imbuing models recomposable primitives for robust local detail modeling.<n>Experiments demonstrate consistent improvements in the 5-shot regime over state-of-the-art L2I methods in both visual fidelity and alignment across diverse domains.
• Score: 11.600113323546404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The layout-to-image (L2I) task enables fine-grained control over image generation via object categories and spatial layouts. However, existing L2I methods yield fragmented and distorted generations under few-shot atypical settings. We term this failure as representation fragmentation, arising from a granularity mismatch that entangles semantic identity with visual details. To address this issue, we propose a representation-driven framework that disentangles semantics from primitives for robust few-shot adaptation. Specifically, Semantic Anchoring aggregates categorical semantics into anchors for stable identity, while Primitive Imbuing models recomposable primitives for robust local detail modeling. Conceptual Steering further regulates optimization with a saliency-aware objective to preserve foreground semantic consistency. Extensive experiments demonstrate consistent improvements in the 5-shot regime over state-of-the-art L2I methods in both visual fidelity and alignment across diverse atypical domains. The source code is publicly available at https://github.com/iCVTEAM/DSP.

Related papers

• Timage: A Generative Text-in-Image Paradigm for Fine-Tuning Vision-Language Models [22.803631190636313]
  We introduce Timage, a paradigm that recasts multimodal understanding as an alignment problem solved at the input.<n>The study positions deliberate input reconstruction as a powerful, architecture-neutral lever for strengthening multimodal reasoning.
  arXiv  Detail & Related papers  (2026-06-18T08:40:41Z)
• STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model [4.685207648946472]
  STEDiff is a training-free method designed to enhance semantic representations directly within the text-embedding space.<n>Our method notably improves semantic consistency and generation in complex scenarios.
  arXiv  Detail & Related papers  (2026-06-09T09:59:09Z)
• HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning [30.51174005122181]
  Remote sensing image change captioning (RSICC) aims to achieve high-level semantic understanding of genuine changes occurring between bi-temporal images.<n>Existing methods are fundamentally limited by a shared modeling assumption.<n>We propose a novel hierarchical semantic disentangling network (HiSem) that explicitly disentangles semantic representations of different granularities.
  arXiv  Detail & Related papers  (2026-05-14T16:23:16Z)
• UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation [65.53694602893042]
  We propose a unified visual conditioning framework that fuses ViT and VAE features before VLM encoding.<n>Experiments on two multi-reference generation benchmarks demonstrate that UniCustom consistently improves subject consistency, instruction following, and compositional fidelity.
  arXiv  Detail & Related papers  (2026-05-12T13:10:05Z)
• Concept-Guided Fine-Tuning: Steering ViTs away from Spurious Correlations to Improve Robustness [16.541207762213272]
  Vision Transformers (ViTs) often degrade under distribution shifts because they rely on spurious correlations.<n>We introduce a novel finetuning framework that steers model reasoning toward concept-level semantics.<n>Our method improves robustness across multiple ViT-based models.
  arXiv  Detail & Related papers  (2026-03-09T12:31:14Z)
• DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition [69.10628479553709]
  We introduce DRC, a novel personalized image generation framework that enhances Large Multimodal Models (LMMs)<n> DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively.<n>It involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation.
  arXiv  Detail & Related papers  (2025-04-24T08:10:10Z)
• DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization [31.960807999301196]
  We propose a novel plug-in method, called DreamMatcher, which reformulates T2I personalization as semantic matching. Specifically, DreamMatcher replaces the target values with reference values aligned by semantic matching, while leaving the structure path unchanged. We also introduce a semantic-consistent masking strategy to isolate the personalized concept from irrelevant regions introduced by the target prompts.
  arXiv  Detail & Related papers  (2024-02-15T09:21:16Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• Progressively Dual Prior Guided Few-shot Semantic Segmentation [57.37506990980975]
  Few-shot semantic segmentation task aims at performing segmentation in query images with a few annotated support samples. We propose a progressively dual prior guided few-shot semantic segmentation network.
  arXiv  Detail & Related papers  (2022-11-20T16:19:47Z)
• Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions [78.71745819446176]
  Refign is a generic extension to self-training-based UDA methods which leverages cross-domain correspondences. Refign consists of two steps: (1) aligning the normal-condition image to the corresponding adverse-condition image using an uncertainty-aware dense matching network, and (2) refining the adverse prediction with the normal prediction using an adaptive label correction mechanism. The approach introduces no extra training parameters, minimal computational overhead -- during training only -- and can be used as a drop-in extension to improve any given self-training-based UDA method.
  arXiv  Detail & Related papers  (2022-07-14T11:30:38Z)
• Context-Aware Layout to Image Generation with Enhanced Object Appearance [123.62597976732948]
  A layout to image (L2I) generation model aims to generate a complicated image containing multiple objects (things) against natural background (stuff) Existing L2I models have made great progress, but object-to-object and object-to-stuff relations are often broken. We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.
  arXiv  Detail & Related papers  (2021-03-22T14:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.