Text to Sketch Generation with Multi-Styles

• URL: http://arxiv.org/abs/2511.04123v1
• Date: Thu, 06 Nov 2025 07:13:56 GMT
• Title: Text to Sketch Generation with Multi-Styles
• Authors: Tengjie Li, Shikui Tu, Lei Xu,
• Abstract summary: We propose a training-free framework based on diffusion models that enables explicit style guidance.<n>We incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism.<n>Our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control.
• Score: 17.309370958875785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at https://github.com/CMACH508/M3S.

Related papers

• Multi-Level Conditioning by Pairing Localized Text and Sketch for Fashion Image Generation [14.962452069195544]
  We present LOcalized Text and Sketch with multi-level guidance (LOTS)<n>LOTS combines global sketch guidance with multiple localized sketch-text pairs.<n>We develop Sketchy, the first fashion dataset where multiple text-sketch pairs are provided per image.
  arXiv  Detail & Related papers  (2026-02-20T16:07:31Z)
• VideoSketcher: Video Models Prior Enable Versatile Sequential Sketch Generation [73.23035143627598]
  Most generative models treat sketches as static images, overlooking the temporal structure that underlies creative drawing.<n>We present a data-efficient approach for sequential sketch generation that adapts pretrained text-to-video diffusion models.<n>Our method generates high-quality sketches that closely follow text-specified orderings while exhibiting rich visual detail.
  arXiv  Detail & Related papers  (2026-02-17T18:55:03Z)
• PokeFusion Attention: Enhancing Reference-Free Style-Conditioned Generation [0.0]
  This paper studies reference-free style-conditioned character generation in text-to-image diffusion models.<n>Existing approaches rely on text-only prompting, or introduce reference-based adapters that depend on external images at inference time.<n>We propose PokeFusion Attention, a lightweight decoder-level cross-attention mechanism.
  arXiv  Detail & Related papers  (2026-02-03T07:44:01Z)
• Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration [57.02757226679549]
  We introduce a training-free framework that reformulates style-guided synthesis as an in-context learning task.<n>We propose a Dynamic Semantic-Style Integration (DSSI) mechanism that reweights attention between semantic and style visual tokens.<n>Experiments show that our approach achieves high-fidelity stylization with superior semantic-style balance and visual quality.
  arXiv  Detail & Related papers  (2026-01-10T16:01:14Z)
• SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation [57.47730473674261]
  We introduce SwiftSketch, a model for image-conditioned vector sketch generation that can produce high-quality sketches in less than a second.<n>SwiftSketch operates by progressively denoising stroke control points sampled from a Gaussian distribution.<n>ControlSketch is a method that enhances SDS-based techniques by incorporating precise spatial control through a depth-aware ControlNet.
  arXiv  Detail & Related papers  (2025-02-12T18:57:12Z)
• ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model [73.95608242322949]
  Stylized Text-to-Image Generation (STIG) aims to generate images from text prompts and style reference images. We present ArtWeaver, a novel framework that leverages pretrained Stable Diffusion to address challenges such as misinterpreted styles and inconsistent semantics.
  arXiv  Detail & Related papers  (2024-05-24T07:19:40Z)
• Customize StyleGAN with One Hand Sketch [0.0]
  We propose a framework to control StyleGAN imagery with a single user sketch. We learn a conditional distribution in the latent space of a pre-trained StyleGAN model via energy-based learning. Our model can generate multi-modal images semantically aligned with the input sketch.
  arXiv  Detail & Related papers  (2023-10-29T09:32:33Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• Learning Graph Neural Networks for Image Style Transfer [131.73237185888215]
  State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching. In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization.
  arXiv  Detail & Related papers  (2022-07-24T07:41:31Z)
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval [119.03470556503942]
  Crossmodal matching problem is typically solved by learning a joint embedding space where semantic content shared between photo and sketch modalities are preserved. An effective model needs to explicitly account for this style diversity, crucially, to unseen user styles. Our model can not only disentangle the cross-modal shared semantic content, but can adapt the disentanglement to any unseen user style as well, making the model truly agnostic.
  arXiv  Detail & Related papers  (2021-03-29T15:44:19Z)
• Sketch-to-Art: Synthesizing Stylized Art Images From Sketches [23.75420342238983]
  We propose a new approach for synthesizing fully detailed art-stylized images from sketches. Given a sketch, with no semantic tagging, and a reference image of a specific style, the model can synthesize meaningful details with colors and textures.
  arXiv  Detail & Related papers  (2020-02-26T19:02:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.