SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space

• URL: http://arxiv.org/abs/2608.21659v1
• Date: Fri, 21 Aug 2026 22:02:23 GMT
• Title: SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space
• Abstract summary: We present SketchFlow, a novel generative framework rooted in Optimal Transport (OT) theory and flow matching.<n>By leveraging pre-trained CLIP models, we formulate cross-modal alignment as a continuous mapping problem.<n>Experiments demonstrate that SketchFlow substantially outperforms existing baselines in visual quality and adherence to natural human drawing styles.
• Score: 9.378039371407114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vector sketches remain one of the most concise and immediate mediums for abstract human expression. However, generating high-quality vector strokes that exhibit human-like drawing styles remains an open challenge due to the severe scarcity of fine-grained, high-quality text-to-sketch paired data. Existing text-conditioned generation methods often rely on unstable, time-consuming optimization or struggle to generalize to unseen categories in a zero-shot manner. To address these limitations, we present SketchFlow, a novel generative framework rooted in Optimal Transport (OT) theory and flow matching. By leveraging pre-trained CLIP models to bypass labor-intensive image-level text annotations, we formulate cross-modal alignment as a continuous mapping problem directly within the CLIP latent space. To bridge the inevitable modality gap between discrete text concepts and continuous sketch features, we first inject noise into discrete category embeddings to construct a continuous Gaussian Mixture Model (GMM) prior. We then utilize an Optimal Transport Conditional Flow Matching (OT-CFM) model to learn a deterministic vector field mapping from this continuous GMM prior to the target sketch feature distribution. Finally, a Hybrid Diffusion Decoder, fusing 1D U-Net and Transformer architectures, is designed to decode these features into fast and high-fidelity stroke trajectories. Extensive experiments demonstrate that SketchFlow substantially outperforms existing baselines in visual quality and adherence to natural human drawing styles. Furthermore, our geometry-preserving framework demonstrates promising local zero-shot synthesis for prompts beyond the QuickDraw training vocabulary, including unseen concept labels and semantic modifiers, while enabling smooth, continuous semantic interpolation between distinct concepts. Source code is available at: https://github.com/QiuHong-1202/SketchFlow.

Related papers

• SeamFlow: Structure-Aware Flow Matching on Edge Probabilities for Artist-Like UV Unwrapping [49.50766584180635]
  We introduce SeamFlow, a novel generative framework for 3D surface cutting.<n>SeamFlow achieves exceptional semantic coherence and remarkably low parameterization distortion.
  arXiv  Detail & Related papers  (2026-09-04T05:31:09Z)
• XYZFlow:Scaling Multi dimensional Shortcut Flows for Efficient Generative Modeling [24.78695434475162]
  XYZFlow is a framework that rethinks efficient generation through multidimensional scaling of flow matching.<n>Next Shortcut Prediction delivers superior quality-latency trade-offs over model scaling or step reduction.
  arXiv  Detail & Related papers  (2026-08-12T17:15:42Z)
• DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation [53.011370226020695]
  We propose DINOde, an ODE-based framework that aligns CLIP text embeddings with the DINO visual manifold.<n>Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which refines the holistic image representation carried by DINO's CLS token.<n>By modeling alignment as a continuous trajectory, DINOde avoids the entanglement inherent in discrete projections and yields more robust cross-modal alignment.
  arXiv  Detail & Related papers  (2026-07-23T14:37:27Z)
• Single-Line Drawing Generation via Semantics-Driven Optimization [51.26529682867565]
  We present the first semantics-driven method for automatically generating single-line drawings in vector format.<n>Our approach leverages score distillation sampling to optimize the parameters of a uniform rational B-spline (URBS) curve.<n>Because our method generates a vectorized curve, it directly supports downstream fabrication processes such as embroidery, laser engraving and wire bending.
  arXiv  Detail & Related papers  (2026-06-01T08:46:22Z)
• StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusion [13.862427684807486]
  StrokeFusion is a two-stage framework for vector sketch generation.<n>It contains a dual-modal sketch feature learning network that maps strokes into a high-quality latent space.<n>It exploits a stroke-level latent diffusion model that simultaneously adjusts stroke position, scale, and trajectory during generation.
  arXiv  Detail & Related papers  (2025-03-31T06:03:03Z)
• SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation [57.47730473674261]
  We introduce SwiftSketch, a model for image-conditioned vector sketch generation that can produce high-quality sketches in less than a second.<n>SwiftSketch operates by progressively denoising stroke control points sampled from a Gaussian distribution.<n>ControlSketch is a method that enhances SDS-based techniques by incorporating precise spatial control through a depth-aware ControlNet.
  arXiv  Detail & Related papers  (2025-02-12T18:57:12Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models [33.6615688030998]
  DiffSketcher is an innovative algorithm that creates textitvectorized free-hand sketches using natural language input. Our experiments show that DiffSketcher achieves greater quality than prior work.
  arXiv  Detail & Related papers  (2023-06-26T13:30:38Z)
• B\'ezierSketch: A generative model for scalable vector sketches [132.5223191478268]
  We present B'ezierSketch, a novel generative model for fully vector sketches that are automatically scalable and high-resolution. We first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B'ezier curve. This enables us to treat sketches as short sequences of paramaterized strokes and thus train a recurrent sketch generator with greater capacity for longer sketches.
  arXiv  Detail & Related papers  (2020-07-04T21:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.