SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

• URL: http://arxiv.org/abs/2609.03806v1
• Date: Thu, 03 Sep 2026 13:12:37 GMT
• Title: SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation
• Abstract summary: We introduce textbfours, a human-aligned evaluation framework for text-to-SVG generation.<n>We first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations.<n>We then introduce a human-annotated dataset for textitSemantic Alignment, measuring how faithfully a generated SVG reflects its caption.
• Score: 62.7676301168359
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \textbf{\ours}, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf Vision-Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated dataset for \textit{Semantic Alignment}, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.

Related papers

• SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation [49.00672742747506]
  Multimodal large models are increasingly used to generate scalable vector graphics (SVG)<n>Existing protocols are often code-centric or borrow-image metrics after rendering SVGs.<n>We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment.
  arXiv  Detail & Related papers  (2026-08-03T09:41:18Z)
• DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance [48.98604326855894]
  We introduce DuetSVG, a unified multimodal model that jointly generates image tokens and corresponding SVG tokens in an end-to-end manner.<n>At inference, we apply a novel test-time scaling strategy that leverages the model's native visual predictions as guidance to improve SVG decoding quality.
  arXiv  Detail & Related papers  (2025-12-11T18:23:03Z)
• SVGThinker: Instruction-Aligned and Reasoning-Driven Text-to-SVG Generation [47.390332111383294]
  We present SVGThinker, a reasoning-driven framework that aligns the production of SVG code with the visualization process.<n>Our pipeline first renders each primitive in sequence and uses a multimodal model to annotate the image and code.<n> Experiments against state-of-the-art baselines show that SVGThinker produces more stable, editable, and higher-quality SVGs.
  arXiv  Detail & Related papers  (2025-09-29T05:25:00Z)
• SVGauge: Towards Human-Aligned Evaluation for SVG Generation [5.565553009801627]
  We introduce SVGauge, the first human-aligned, reference based metric for text-to-SVG generation.<n> SVGauge measures (i) visual fidelity, obtained by extracting SigLIP image embeddings and refining them with PCA and whitening for domain alignment.<n> Evaluation on the proposed SHE benchmark shows that SVGauge attains the highest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-09-08T18:28:31Z)
• SVGen: Interpretable Vector Graphics Generation with Large Language Models [61.62816031675714]
  We introduce SVG-1M, a large-scale dataset of high-quality SVGs paired with natural language descriptions.<n>We create well-aligned Text to SVG training pairs, including a subset with Chain of Thought annotations for enhanced semantic guidance.<n>Based on this dataset, we propose SVGen, an end-to-end model that generates SVG code from natural language inputs.
  arXiv  Detail & Related papers  (2025-08-06T15:00:24Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• Visually Descriptive Language Model for Vector Graphics Reasoning [76.42082386029206]
  We propose the Visually Descriptive Language Model (VDLM) to bridge the gap between low-level visual perception and high-level language reasoning.<n>We show that VDLM significantly improves state-of-the-art LMMs like GPT-4o on various multimodal perception and reasoning tasks.
  arXiv  Detail & Related papers  (2024-04-09T17:30:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.