Self-Creative Text-to-Object Generation using Semantic-Aware Spatial Weighting

• URL: http://arxiv.org/abs/2605.19554v1
• Date: Tue, 19 May 2026 08:52:18 GMT
• Title: Self-Creative Text-to-Object Generation using Semantic-Aware Spatial Weighting
• Abstract summary: Current text-to-image (T2I) models are largely optimized for literal text-image alignment with their data distribution.<n>We propose a Self-Creative Diffusion (SCDiff) model for meaningful T2I generations featuring two core modules.<n>Our model substantially improves creativity, semantic alignment, and visual coherence, offering a simple yet powerful framework for generating creative objects.
• Score: 24.62226758388138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instilling creativity in text-to-image (T2I) generation presents a significant challenge, as it requires synthesized images to exhibit not only visual novelty and surprise, but also artistic value. Current T2I models, however, are largely optimized for literal text-image alignment with their data distribution, and their noise prediction networks constrain the generation to high-probability regions, consequently generating outputs that lack authentic creativity. To address this, we propose a Self-Creative Diffusion (SCDiff) model for meaningful T2I generations featuring two core modules: a learnable spatial weighting (LSW) module and a visual-semantic mixing loss (VSML). The LSW module designs a parametric Kaiser-Bessel window to reinforce central image features, fostering novel and surprising generation. The VSML module introduces a dual loss function: a similarity loss constrains that the new images align with its textual description, while a diversity loss maximizes its distinction from the original image, enhancing both semantic value and visual novelty. Extensive experiments demonstrate that our model substantially improves creativity, semantic alignment, and visual coherence, offering a simple yet powerful framework for generating creative objects.

Related papers

• Multimodal Large Language Models for Multi-Subject In-Context Image Generation [56.20395856287325]
  Music is the first MLLM specifically designed for textbfMUlti-textbfSubject textbfIn-textbfContext image generation.<n>We enhance the model's understanding of multi-subject semantic relationships through a vision chain-of-thought mechanism.<n>By incorporating complex subject images during training, we improve the model's capacity for chained reasoning.
  arXiv  Detail & Related papers  (2026-04-08T15:37:42Z)
• Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation [63.50827603618498]
  We propose Lavida-O, a unified Masked Diffusion Model (MDM) for multimodal understanding and generation.<n>Lavida-O presents a single framework that enables image-level understanding, object grounding, image editing, and high-resolution text-to-image synthesis.<n>Lavida-O achieves state-of-the-art performance on a wide range of benchmarks including RefCOCO object grounding, GenEval text-to-image generation, and ImgEdit image editing.
  arXiv  Detail & Related papers  (2025-09-23T17:05:46Z)
• LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generation [1.124958340749622]
  Vision-Language Models (LVLMs) have demonstrated powerful capabilities in cross-modal understanding and instruction following.<n>LumiGen is a novel LVLM-enhanced iterative framework designed to elevate T2I model performance.<n>LumiGen achieves a superior average score of 3.08, outperforming state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-05T20:53:43Z)
• Boosting Generative Image Modeling via Joint Image-Feature Synthesis [15.133906625258797]
  We introduce a novel generative image modeling framework that seamlessly bridges the gap by leveraging a diffusion model to jointly model low-level image latents.<n>Our latent-semantic diffusion approach learns to generate coherent image-feature pairs from pure noise.<n>By eliminating the need for complex distillation objectives, our unified design simplifies training and unlocks a powerful new inference strategy: Representation Guidance.
  arXiv  Detail & Related papers  (2025-04-22T17:41:42Z)
• ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement [68.05833403672274]
  Existing unified models have struggled to handle the three fundamental capabilities in a unified model: understanding, generation, and editing.<n>ILLUME+ introduces a unified dual visual tokenizer, DualViTok, which preserves fine-grained textures and text-aligned semantics.<n>We also employ a diffusion model as the image detokenizer for enhanced generation quality and efficient super-resolution.
  arXiv  Detail & Related papers  (2025-04-02T17:45:00Z)
• Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model [69.09404597939744]
  Seedream 2.0 is a native Chinese-English bilingual image generation foundation model.<n>It adeptly manages text prompt in both Chinese and English, supporting bilingual image generation and text rendering.<n>It is integrated with a self-developed bilingual large language model as a text encoder, allowing it to learn native knowledge directly from massive data.
  arXiv  Detail & Related papers  (2025-03-10T17:58:33Z)
• Towards Enhanced Image Generation Via Multi-modal Chain of Thought in Unified Generative Models [52.84391764467939]
  Unified generative models have shown remarkable performance in text and image generation.<n>We introduce Chain of Thought (CoT) into unified generative models to address the challenges of complex image generation.<n>Experiments show that FoX consistently outperforms existing unified models on various T2I benchmarks.
  arXiv  Detail & Related papers  (2025-03-03T08:36:16Z)
• Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models [54.052963634384945]
  We introduce the Image Regeneration task to assess text-to-image models. We use GPT4V to bridge the gap between the reference image and the text input for the T2I model. We also present ImageRepainter framework to enhance the quality of generated images.
  arXiv  Detail & Related papers  (2024-11-14T13:52:43Z)
• MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models [51.1034358143232]
  We introduce component-controllable personalization, a new task that enables users to customize and reconfigure individual components within concepts.<n>This task faces two challenges: semantic pollution, where undesired elements disrupt the target concept, and semantic imbalance, which causes disproportionate learning of the target concept and component.<n>We design MagicTailor, a framework that uses Dynamic Masked Degradation to adaptively perturb unwanted visual semantics and Dual-Stream Balancing for more balanced learning of desired visual semantics.
  arXiv  Detail & Related papers  (2024-10-17T09:22:53Z)
• Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models [20.19571676239579]
  We introduce a novel diffusion-based framework to enhance the alignment of generated images with their corresponding descriptions. Our framework is built upon a comprehensive analysis of inconsistency phenomena, categorizing them based on their manifestation in the image. We then integrate a state-of-the-art controllable image generation model with a visual text generation module to generate an image that is consistent with the original prompt.
  arXiv  Detail & Related papers  (2024-06-24T06:12:16Z)
• DreamCreature: Crafting Photorealistic Virtual Creatures from Imagination [140.1641573781066]
  We introduce a novel task, Virtual Creatures Generation: Given a set of unlabeled images of the target concepts, we aim to train a T2I model capable of creating new, hybrid concepts. We propose a new method called DreamCreature, which identifies and extracts the underlying sub-concepts. The T2I thus adapts to generate novel concepts with faithful structures and photorealistic appearance.
  arXiv  Detail & Related papers  (2023-11-27T01:24:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.