Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs

• URL: http://arxiv.org/abs/2608.28678v1
• Date: Wed, 26 Aug 2026 00:51:10 GMT
• Title: Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs
• Abstract summary: We evaluate whether inference-time methods can unlock SVG generation capabilities in off-the-shelf vision-language models (VLMs)<n>Our results highlight both the promise and current limitations of using inference-time methods to adapt general-purpose VLMs for SVG generation.
• Score: 50.541990269155484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scalable Vector Graphics (SVGs) power much of the modern visual ecosystem, yet state-of-the-art generative models focus almost entirely on rasterized images. We explore whether inference-time methods can unlock SVG generation capabilities in off-the-shelf vision-language models (VLMs). We systematically evaluate a constrained iterative refinement harness that combines visual feedback, structured editing, and constrained decoding to characterize the capabilities and limitations of current VLMs for SVG generation. Across multiple VLMs and generation settings, we find that constrained decoding improves compilation success rates, while iterative refinement reveals a deficit in visual reasoning and self-correction. Our results highlight both the promise and current limitations of using inference-time methods to adapt general-purpose VLMs for SVG generation.

Related papers

• SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation [62.7676301168359]
  We introduce textbfours, a human-aligned evaluation framework for text-to-SVG generation.<n>We first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations.<n>We then introduce a human-annotated dataset for textitSemantic Alignment, measuring how faithfully a generated SVG reflects its caption.
  arXiv  Detail & Related papers  (2026-09-03T13:12:37Z)
• RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation [2.1124849856688064]
  RefineSVG is a single-step closed-loop visual feedback framework.<n>It enables multimodal large language models (MLLMs) to perform high-fidelity image-to-SVG generation through self-correction.
  arXiv  Detail & Related papers  (2026-07-30T05:30:14Z)
• Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback [29.19392406217364]
  We propose a novel generation paradigm that reformulates SVG synthesis as a step-wise, visual-context-aware process.<n>By rendering intermediate code states into a cumulative canvas, the model explicitly observes the evolving visual context at each step.<n>We show that applying this visual loop naively to off-the-shelf models is suboptimal due to their inability to leverage incremental visual-code mappings.
  arXiv  Detail & Related papers  (2026-04-22T16:15:28Z)
• IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework [20.964700751378547]
  Existing text-to-SVG generation methods do not incorporate visual perception of the final rendered image.<n>We propose an Introspective SVG Generation Framework (IntroSVG)<n>We show that our method achieves state-of-the-art performance across several key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-10T07:44:51Z)
• Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure [57.89872230703339]
  We introduce a framework that recovers the semantic structure required for reliable SVG animation.<n>By reorganizing SVGs into semantic groups, our approach enables VLMs to produce animations with far greater coherence.
  arXiv  Detail & Related papers  (2025-12-16T12:03:46Z)
• DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance [48.98604326855894]
  We introduce DuetSVG, a unified multimodal model that jointly generates image tokens and corresponding SVG tokens in an end-to-end manner.<n>At inference, we apply a novel test-time scaling strategy that leverages the model's native visual predictions as guidance to improve SVG decoding quality.
  arXiv  Detail & Related papers  (2025-12-11T18:23:03Z)
• Visual Generation Tuning [84.50113837230333]
  We propose VGT, Visual Generation Tuning, a novel paradigm designed to stimulate the underlying capabilities of visual generation within vision language models.<n>In image reconstruction tasks, we achieve 26.67 PSNR and 0.50 rFID at a 28x compression ratio, outperforming specialized VAEs.<n>Our proposed VGT showcases significant scaling promise and is versatile for endowing any VLMs trained for multimodal understanding with the capabilities of visual generation.
  arXiv  Detail & Related papers  (2025-11-28T18:57:13Z)
• SVGen: Interpretable Vector Graphics Generation with Large Language Models [61.62816031675714]
  We introduce SVG-1M, a large-scale dataset of high-quality SVGs paired with natural language descriptions.<n>We create well-aligned Text to SVG training pairs, including a subset with Chain of Thought annotations for enhanced semantic guidance.<n>Based on this dataset, we propose SVGen, an end-to-end model that generates SVG code from natural language inputs.
  arXiv  Detail & Related papers  (2025-08-06T15:00:24Z)
• Rendering-Aware Reinforcement Learning for Vector Graphics Generation [15.547843461605746]
  We introduce RLRF(Reinforcement Learning from Rendering Feedback), an RL method that enhances SVG generation in vision-language models (VLMs)<n>Given an input image, the model generates SVG roll-outs that are rendered and compared to the original image to compute a reward.<n>This visual fidelity feedback guides the model toward producing more accurate, efficient, and semantically coherent SVGs.
  arXiv  Detail & Related papers  (2025-05-27T06:56:00Z)
• Leveraging Large Language Models For Scalable Vector Graphics Processing: A Review [0.0]
  Traditional vectorization techniques suffer from long processing times and excessive output complexity.<n>The advent of large language models (LLMs) has opened new possibilities for the generation, editing, and analysis of vector graphics.
  arXiv  Detail & Related papers  (2025-03-06T21:23:17Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.