Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis

• URL: http://arxiv.org/abs/2604.08809v1
• Date: Thu, 09 Apr 2026 22:50:41 GMT
• Title: Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis
• Abstract summary: We introduce element-level leave-one-out (LOO) analysis, inspired by the classic jackknife estimator.<n>LOO scores per element through LOO scoring that enable zero-shot artifact detection.<n>We validate these metrics on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.
• Score: 7.246398319376824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scalable Vector Graphics (SVG) represent visual content as structured, editable code. Each element (path, shape, or text node) can be individually inspected, transformed, or removed. This structural editability is a main motivation for SVG generation, yet prevailing evaluation protocols primarily reduce the output to a single similarity score against a reference image or input texts, measuring how faithfully the result reproduces an image or follows the instructions, but not how well it preserves the structural properties that make SVG valuable. In particular, existing metrics cannot determine which generated elements contribute positively to overall visual quality, how visual concepts map to specific parts of the code, or whether the generated output supports meaningful downstream editing. We introduce element-level leave-one-out (LOO) analysis, inspired by the classic jackknife estimator. The procedure renders the SVG with and without each element, measures the resulting visual change, and derives a suite of structural quality metrics. Despite its simplicity, the jackknife's capacity to decompose an aggregate statistic into per-sample contributions translates directly to this setting. From a single mechanism, we obtain: (1) quality scores per element through LOO scoring that enable zero-shot artifact detection; (2) concept-element attribution that maps each element to the visual concept it serves; and (3) four structural metrics, purity, coverage, compactness, and locality, that quantify SVG modularity from complementary perspectives. We validate these metrics on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.

Related papers

• Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing [18.53393579768945]
  We propose a VLM-driven agentic framework for semantic compositional generation.<n>Our pipeline parses visual scenes into semantic and geometric hierarchies via top-down decomposition.<n>Experiments show that our predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability.
  arXiv  Detail & Related papers  (2026-09-13T16:50:07Z)
• SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation [62.7676301168359]
  We introduce textbfours, a human-aligned evaluation framework for text-to-SVG generation.<n>We first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations.<n>We then introduce a human-annotated dataset for textitSemantic Alignment, measuring how faithfully a generated SVG reflects its caption.
  arXiv  Detail & Related papers  (2026-09-03T13:12:37Z)
• SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation [49.00672742747506]
  Multimodal large models are increasingly used to generate scalable vector graphics (SVG)<n>Existing protocols are often code-centric or borrow-image metrics after rendering SVGs.<n>We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment.
  arXiv  Detail & Related papers  (2026-08-03T09:41:18Z)
• Multimodal OCR: Parse Anything from Documents [72.69545534962234]
  dots.mocr treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets.<n>It reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction.<n>It supports end-to-end training over heterogeneous document elements.
  arXiv  Detail & Related papers  (2026-03-13T14:42:21Z)
• SVG360: Multi-View SVG Generation with Geometric and Color Consistency from a Single SVG [1.558477573183958]
  We present a three-stage framework that produces multi-view consistent SVGs from a single SVG input.<n>The resulting SVGs exhibit strong color consistency across views, significantly reduce redundant paths, and retain fine structural details.<n>This work bridges generative modeling and structured vector representation, providing a scalable route to single input and object level multi-view generation.
  arXiv  Detail & Related papers  (2025-11-20T19:33:57Z)
• SVGauge: Towards Human-Aligned Evaluation for SVG Generation [5.565553009801627]
  We introduce SVGauge, the first human-aligned, reference based metric for text-to-SVG generation.<n> SVGauge measures (i) visual fidelity, obtained by extracting SigLIP image embeddings and refining them with PCA and whitening for domain alignment.<n> Evaluation on the proposed SHE benchmark shows that SVGauge attains the highest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-09-08T18:28:31Z)
• SVGen: Interpretable Vector Graphics Generation with Large Language Models [61.62816031675714]
  We introduce SVG-1M, a large-scale dataset of high-quality SVGs paired with natural language descriptions.<n>We create well-aligned Text to SVG training pairs, including a subset with Chain of Thought annotations for enhanced semantic guidance.<n>Based on this dataset, we propose SVGen, an end-to-end model that generates SVG code from natural language inputs.
  arXiv  Detail & Related papers  (2025-08-06T15:00:24Z)
• OmniSVG: A Unified Scalable Vector Graphics Generation Model [69.59073636922287]
  We propose OmniSVG, a unified framework that leverages pre-trained Vision-Language Models for end-to-end multimodal SVG generation.<n>By parameterizing SVG commands and coordinates into discrete tokens, OmniSVG decouples structural logic from low-level geometry for efficient training while maintaining the synthesis of complex SVG structure.<n>We introduce MMSVG-2M, a multimodal dataset with two million annotated SVG assets, along with a standardized evaluation protocol for conditional SVG generation tasks.
  arXiv  Detail & Related papers  (2025-04-08T17:59:49Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• SVGCraft: Beyond Single Object Text-to-SVG Synthesis with Comprehensive Canvas Layout [14.824205628841158]
  This work introduces a novel end-to-end framework for the creation of vector graphics depicting entire scenes from textual descriptions. SVGCraft is optimized using a pre-trained encoder and LPIPS loss with opacity modulation to maximize similarity. It is demonstrated to surpass prior works in abstraction, recognizability, and detail.
  arXiv  Detail & Related papers  (2024-03-30T16:43:40Z)
• Beyond Pixels: Exploring Human-Readable SVG Generation for Simple Images with Vision Language Models [19.145503353922038]
  We introduce our method, Simple-SVG-Generation (Stextsuperscript2VGtextsuperscript2). Our method focuses on producing SVGs that are both accurate and simple, aligning with human readability and understanding. With simple images, we evaluate our method with reasoning tasks together with advanced language models, the results show a clear improvement over previous SVG generation methods.
  arXiv  Detail & Related papers  (2023-11-27T05:20:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.