Reason-SVG: Hybrid Reward RL for Aha-Moments in Vector Graphics Generation

• URL: http://arxiv.org/abs/2505.24499v1
• Date: Fri, 30 May 2025 11:57:58 GMT
• Title: Reason-SVG: Hybrid Reward RL for Aha-Moments in Vector Graphics Generation
• Authors: Ximing Xing, Yandong Guan, Jing Zhang, Dong Xu, Qian Yu,
• Abstract summary: We introduce Reason-SVG, a framework designed to enhance Large Language Models (LLMs) reasoning for SVG generation.<n>Reason-SVG pioneers the "Drawing-with-Thought" (DwT) paradigm, in which models generate both SVG code and explicit design rationales.<n>We introduce the SVGX-DwT-10k dataset, a high-quality corpus of 10,000 SVG-DwT pairs, where each SVG code is generated based on explicit DwT reasoning.
• Score: 29.418375886989992
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating high-quality Scalable Vector Graphics (SVGs) is challenging for Large Language Models (LLMs), as it requires advanced reasoning for structural validity, semantic faithfulness, and visual coherence -- capabilities in which current LLMs often fall short. In this work, we introduce Reason-SVG, a novel framework designed to enhance LLM reasoning for SVG generation. Reason-SVG pioneers the "Drawing-with-Thought" (DwT) paradigm, in which models generate both SVG code and explicit design rationales, mimicking the human creative process. Reason-SVG adopts a two-stage training strategy: First, Supervised Fine-Tuning (SFT) trains the LLM on the DwT paradigm to activate foundational reasoning abilities. Second, Reinforcement Learning (RL), utilizing Group Relative Policy Optimization (GRPO), empowers the model to generate both DwT and SVGs rationales through refined, reward-driven reasoning. To facilitate reasoning-driven SVG generation, we design a Hybrid Reward function that evaluates the presence and utility of DwT reasoning, along with structural validity, semantic alignment, and visual quality. We also introduce the SVGX-DwT-10k dataset, a high-quality corpus of 10,000 SVG-DwT pairs, where each SVG code is generated based on explicit DwT reasoning. By integrating DwT, SFT, and Hybrid Reward-guided RL, Reason-SVG significantly improves LLM performance in generating accurate and visually compelling SVGs, potentially fostering "Aha moments" in design.

Related papers

• Rendering-Aware Reinforcement Learning for Vector Graphics Generation [15.547843461605746]
  We introduce RLRF(Reinforcement Learning from Rendering Feedback), an RL method that enhances SVG generation in vision-language models (VLMs)<n>Given an input image, the model generates SVG roll-outs that are rendered and compared to the original image to compute a reward.<n>This visual fidelity feedback guides the model toward producing more accurate, efficient, and semantically coherent SVGs.
  arXiv  Detail & Related papers  (2025-05-27T06:56:00Z)
• OmniSVG: A Unified Scalable Vector Graphics Generation Model [69.59073636922287]
  We propose OmniSVG, a unified framework that leverages pre-trained Vision-Language Models for end-to-end multimodal SVG generation.<n>By parameterizing SVG commands and coordinates into discrete tokens, OmniSVG decouples structural logic from low-level geometry for efficient training while maintaining the synthesis of complex SVG structure.<n>We introduce MMSVG-2M, a multimodal dataset with two million annotated SVG assets, along with a standardized evaluation protocol for conditional SVG generation tasks.
  arXiv  Detail & Related papers  (2025-04-08T17:59:49Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformers [5.921625661186367]
  This paper introduces a component-based, autoregressive model for generating high-quality colored SVGs from textual input.<n>It significantly reduces computational overhead and improves efficiency compared to traditional methods.<n>To address the limitations of existing SVG datasets and support our research, we introduce ColorSVG-100K, the first large-scale dataset of colored SVGs.
  arXiv  Detail & Related papers  (2024-12-13T15:24:11Z)
• Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models [14.917583676464266]
  Chat2SVG is a hybrid framework that combines Large Language Models and image diffusion models for text-to-SVG generation. Our system enables intuitive editing through natural language instructions, making professional vector graphics creation accessible to all users.
  arXiv  Detail & Related papers  (2024-11-25T17:31:57Z)
• Visually Descriptive Language Model for Vector Graphics Reasoning [76.42082386029206]
  We propose the Visually Descriptive Language Model (VDLM) to bridge the gap between low-level visual perception and high-level language reasoning. We show that VDLM significantly improves state-of-the-art LMMs like GPT-4o on various multimodal perception and reasoning tasks.
  arXiv  Detail & Related papers  (2024-04-09T17:30:18Z)
• VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models [82.93345261434943]
  We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches.
  arXiv  Detail & Related papers  (2022-11-21T10:04:27Z)
• Towards Layer-wise Image Vectorization [57.26058135389497]
  We propose Layerwise Image Vectorization, namely LIVE, to convert images to SVGs and simultaneously maintain its image topology. Live generates compact forms with layer-wise structures that are semantically consistent with human perspective. Live initiates human editable SVGs for both designers and can be used in other applications.
  arXiv  Detail & Related papers  (2022-06-09T17:55:02Z)
• SVG-Net: An SVG-based Trajectory Prediction Model [67.68864911674308]
  Anticipating motions of vehicles in a scene is an essential problem for safe autonomous driving systems. To this end, the comprehension of the scene's infrastructure is often the main clue for predicting future trajectories. Most of the proposed approaches represent the scene with averse averseized format and some of the more recent approaches leverage custom vectorized formats.
  arXiv  Detail & Related papers  (2021-10-07T18:00:08Z)
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation [217.86315551526235]
  We propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and manipulation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool.
  arXiv  Detail & Related papers  (2020-07-22T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.